The butterfly effect, a concept originating in chaos theory, describes how small changes can have a nonlinear impact on a complex system, such as the flap of a butterfly's wings causing a typhoon thousands of miles away.
The idea here is that small changes—the burgeoning amount of unstructured data being generated in the enterprise, the interminable event fatigue problem created by false positives in security information and event management (SIEM) solutions, and the global talent shortage in cybersecurity that makes finding affordable security operations center (SOC) analysts difficult—are ushering in a big change as the sun begins to set on SIEM technology.
While organizations historically sent device logs, security control events, and operating system and application logs to central log servers or SIEMs that end up eventually slowing the SIEM down due to over-burdened SQL databases, they are now rethinking this approach by instead sending logs and events into their existing data lakes for analysis by machine learning-powered analytics platforms.
Indeed, the sun is setting on the SIEM as a technology for monitoring the disparate events of security controls and application, operating system, and device logs in the enterprise. Chief information security officers (CISOs) want fewer cybersecurity controls that add to the existing noise and instead want something that can do a better job of finding the signal in the noise of what they are currently generating.
Enter security analytics platforms from companies such as SAS. Security analytics platforms apply machine learning models instead of patterns and signatures to find that signal in the noise within data at rest and leverage data streaming in real time using solutions such as StreamSets. Using machine learning, security analytics platforms take data at rest or in real time in enterprise data lakes and perform analytics on it in order to find that signal, addressing many of the pain points of legacy SIEM technology.
What, you don't think that unstructured data and its prevalence is that big of an issue?
According to IBM, 90% of the world’s data has been created in the last two years alone, and according to IDC, by 2025, 80% of the world's data will be unstructured. And for those not in the know, unstructured data is unable to be stored in traditional SQL databases, creating a challenge for organizations to search through it, edit it, and analyze it. The fact is, the world is moving to unstructured data. Organizations are now creating exabytes of unstructured data in their own data lakes that require a migration to NOSQL databases, such as Hadoop, Elasticsearch, and Cloudera—companies incontrovertibly capitalizing on this data challenge.
In recent interviews with enterprise CISOs, it's become clear to me as an industry analyst that the winds of change are blowing in the SIEM market. CISOs are increasingly divesting in their existing SIEM platforms in an attempt to look for a better tool for the job. Their goal is to remove human error as much as possible in the event analysis process through automated response applied by security orchestration, automation, and response (SOAR), while also sprinkling in security analytics platforms to chew away at their growing data lakes.
What's my industry forecast? I believe that, in the short term, SIEM and SOAR platforms will merge, creating a new hybrid product that combines both SIEM and SOAR functionality in a single platform. While that’s a controversial position, I believe that the SIEM industry will eventually go away, and organizations will send all of their event data and logs to large data lakes that security analytics platforms automatically analyze and respond to. Maybe SOAR companies will begin partnering with companies such as SAS—companies that they can rely on for automated response via their playbooks—to integrate with their analytics platforms, but only time will tell.
As we head into the holiday season and close out 2019, I will be wrapping up my report on all of the security analytics solutions in the market as well as reports on individual vendor solutions, such as SAS Cybersecurity, Elasticsearch, Arcadia Data (powered by the open-source Apache Spot project), and more.
Like and Share
The best way you can support me in my continued content development and influencer efforts in cybersecurity is to like and share my article.
Subscribe and Follow
Subscribe to my YouTube channel to get notifications of my VLOG and Vodcast episodes uploaded weekly and follow me on Twitter. To view my latest content calendar, monitor it in real-time on my homepage at alissaknight.com