Having the time and inclination to look at the entire SIEM market is something I’ve been afforded over the last few years. While certainly I’ve had more exposure to some vendors more than others I’ve been very lucky to have had very in-depth discussions with most, if not all of them over the last few years. Nothing I’m saying will violate NDA or even simple common sense but there are some interesting general threads happening across the SIEM market (and by proxy Log Management) and perhaps the entire Security Industry.
For the most part I’ve found that vendors will create exactly what they think users require of them to the best of their abilities (financially and technically). A thought that is both empowering and terrifying at the same time given both the vendor’s ability to listen and the user’s ability to articulate requirements. The result is at least in my mind a market that should be 3 years further along than it currently is both in terms of industry consolidation and functionality.
In short, everyone approached the “problem” from their initial perspective. Andrew Hay of The 451 Group has a recent post that describes this pretty well. You have Network, Host, Operations, Logging and Analytical limbs of the SIEM tree you can point to as a heritage for most of the products in the space today. As a result you have varying levels of expertise and limitations based on where in the tree your product hails from. The environmental dimension of their evolution was the customer’s they were listening to as they started development and refined features. Those driven by SMB may have done well with ease of use, but probably lacked scalability and flexibility. Those that were scalable and flexible probably had to contend with usability. To this day, there is no perfect solution, there are a few good ones, some others with potential and a lot of dead limbs waiting to be pruned.
In the beginning there were “Speeds and Feeds”. You know the claims of “x” EPS and “y” devices supported. What did it mean? In reality, not a whole lot. Why do vendors still push this in their marketing? In reality so that they can claim relevance alongside their big brothers. It’s a marketing game, currently the high water mark is at 200K EPS and 500+ devices supported. How much do you need to be successful in your organization? Good question, please answer it before you contact the vendors.
Initially, the only real requirement we related to the vendors was having a place to store the logs and search them, some offered a lot more than that, some not so much but there was an entire market (or two) around Eating, Storing and Regurgitating.
Tainted Apple (misguided passion)
The requirements were comprehended by vendors as “make my life easier, automate the analysis, identify everything bad”. So the vendors initially asked about “Use-Cases” and developed features to accommodate those use-cases. The problem was the influencers were the top 1% of the field and as a result the solutions developed were so complex that only a handful of people could effectively utilize them. The features were in reality solutions that might actually have been years ahead of the mainstream consumer so figuring out how to apply them caused confusion. In general this led to two things:
1.) Adding features that were so complex that they hurt performance/reliability and
2.) an opening for even more competition because of complexity, performance, reliability, and other assumed badness.
Competition is good for a market to a certain extent, how many players do you need to validate a market? Certainly the answer to that question can be the current answer (over 20).
So now we come to the phase in each Vendor’s lifecycle where they get to address reliability and stabilty commonly known as bulletproofing. Some hit this in 2004, Some in 2010. The fact remains that every solution hits a phase in its evolution where they need to stop new features and make their current solution more stable and robust to accommodate current needs and future vision. Sometimes if the vendor is lucky enough to survive long enough it happens multiple times. Sometimes it happens at a point in the market where other providers can capitalize on it and make a name for themselves. It’s an important step that takes anywhere from 12-24 months to complete. Ideally, it would happen prior to or immediately after an acquisition. No such luck so far.
Plagues: Blood, Frogs and Locust
Then came Compliance (Specifically SOX, PCI and HIPAA) and all security innovation ceased. Well maybe that is a bit harsh, but it did derail certain elements of where functionality was headed, but on the other hand compliance kept everyone in business. Even many who really should not be in business today. So the market made a decision - we want to believe in SIEM (and security in general) and Compliance provided us money to invest in it. The thought was “We’ll find a way to make the investment work for us later.” We can use Compliance to make “ROI” (I’m shuddering inside) possible.
So the requirement was mandated as “Make me Compliant” and as it was said so it was done… Vendors put together reports and other content, packaged it up and sold it back at a premium and the world was saved. Birds sang, people stopped dying and there was no longer evil occurring anywhere.
Ok so back in reality we soon found out that it was unreasonable to expect check box security to do anything meaningful (we are at that point, aren’t we?!?!) and we may have sacrificed actual security in the name of creating a nearly meaningless lowest common denominator. But hey everyone is still in business and we all have jobs.
SIEM functionality is all over the map, though if you look at analyst rankings everyone is a leader. I took a fun poke at this in my “Olympics Preview” post back in Jan. It was meant to be tongue-in-cheek, I had no idea it would be taken as seriously as it was by anyone.
A simple fact remains true, each of the vendors addresses a subset of the problem from their perspective and that perspective has been broadened and lengthened by their customers over time. Your requirements will probably not match exactly with anyone else’s but there are at least some criteria you will need to define prior to heading to vendor selection. When I do an evaluation of a product I have 150+ specific items I look at not as a positive or negative, but I seek to understand how they solve that problem. Then I try and make it relevent to the specific customer’s situation over the next 12 months. Some times it’s as easy as log management (eat, store, regurgitate) other times it is very complex. The worst thing is settling for a solution without knowing your requirements first. No product can solve that problem.
In the next post I’ll describe where I think SIEM is headed based on the gaps I see today, the innovation that IS being driven across the industry and the requirements that I think should exist, such as:
- Adding perspectives: Context such as User, Network, Host, Application, Virtualization, and Cloud Services.
- Integrating Platforms (SIEM, Log Management and others)
- and of course meaningful and timely Content.