Protect your network from zero-day attacks

Selection criteria for pragmatic Log Management

As we wrap up our 6-month tour of Pragmatic Log Management, let’s focus on what are some of the important buying criteria that you should consider when looking at log management offerings. Ultimately, a lot of the vendors in the space have done a good job of making all the products sound the same. So really deciphering what differentiates one product versus another is an art form.

As you remember from the piece on Buying Log Management offerings, the objective is not to necessarily find the product that knocks all the other competitors out with a first round TKO. If that happens, all the better, but in order to get maximum negotiating leverage, as well as to optimize your selection you want to have a couple of companies on the long list. You also want to evaluate a handful of offerings, and ultimately start to negotiate with at least two.

I’m going to present a set of fairly generic criteria for your purchase. Ultimately these are just guidelines because what is important to you will likely be different. Or you will weigh some of the criteria differently. After studying this market from a number of different perspectives, I’ve simplified pages and pages of features into a couple of buckets that you absolutely need to be focused on:

  1. Connectivity/Integration
  2. Performance
  3. Data retrieval and analysis
  4. Reporting


Log management clearly has limited value if you are only able to log data from a subset of your networks, systems and/or applications. So first and foremost, your log management platform needs to easily support the data sources you need to log. Here are some of the considerations, in no specific order:

  • Syslog is the lowest common denominator of log formats. Basically almost everything I’ve come across supports some flavor of syslog-based logging. So at a minimum, you will look for your platform to be able to take syslog records.
  • Connectors – There are some products where you need to get a more granular level of information that can’t be provided by generic syslog records. You certainly don’t want to build (or support) these integrations yourself. So you should expect your log management vendor to provide a handful of the connectors that are most important to you.
  • Agents – Finally, when it’s impractical to build a connector, it may make more sense to have a little software that runs on the target system to pull the log information and transform it into a format that your log management platform is going to understand.

What’s going to differentiate solutions relative to connectivity/integration? It’s basically ease of integration and the breadth of support for the specific target systems that you need. How easy is it for you to add a new data source? How flexible is the data mapping, so you can put the right information in the right place? Don’t be fooled by a vendor crowing about having 10,000 connectors because that doesn’t matter if all of the leaders support the 25 targets that you really need.


The secret to effective log management is to gather ALL of the data. The aspects of LM that add value to investigations and compliance are seriously minimized if the integrity or completeness of the data can be questioned. Thus, it’s pretty important that your log management NOT drop data all over the floor. So you need a solution that will scale to your requirements. But here is the trick: you need to understand what your requirements REALLY are.

Everyone wants to think they need 100,000 rps (records per second). But do you? Really? The way to figure this out is through the evaluation. Maybe put up a syslog server and start directing traffic there to get a feel for the data flow and quantity. Of course, you’ll be taking a rough guess, and that’s OK. The point of the eval is to make sure the guess isn’t too rough and that you don’t end up with an environment that cannot scale to peak load.

The other thing to be wary of is that most vendors are a little optimistic in terms of their real throughput numbers. DO NOT make buying decisions based on the scalability numbers written on marketing materials. You need to try it for yourself, or suffer the consequences when you need double the number of logging devices to meet your needs.

Data Retrieval and Analysis

Data analysis is where some of the vendors start to really differentiate themselves. Basically, if you think back to the 6 months of the Pragmatic Log Management Series, you know there are 3 different aspects of log management: Operations, Investigations and Compliance.

In terms of operations, you want to make sure you can take the data from the LM system and gather some operational data from it. During the eval, make sure you understand how you’d pinpoint an issue (REACT FASTER) within each vendor’s interface and figure out how to find the areas for deeper investigation to isolate the problem.

You want to see the pre-built dashboards and play around a bit with the custom views that can be built within the interface. Make sure you are comfortable with the customization process and ensure it’ll be easy for you to do, since once you buy the product – don’t expect the vendor to spend a lot of time customizing it for you (unless you bring your checkbook for lots of additional consulting).

Since investigations are also a key part of LM, you want to make sure the system provides adequate filtering, correlation and drill down capabilities on historical data. It’s important to store the data in a forensically sound format, so scrutinize the approach the vendor has for that and make sure that it’ll stand up in court. The idea of being able to “play back” a scenario based on log data is a very nice feature that can help during an active investigation.

This is also when you want to check with references about REAL live instances of a customer using the LM system to isolate and find an attack or incident. You also want to talk to a customer that has used the data in both an audit and a legal proceeding. If the vendor hasn’t done that (or can’t provide it for you), then they probably aren’t the right vendor.


Finally, the last aspect of LM selection is making sure you have the reports you need, both for operations, as well as compliance. What does that mean? In a nutshell, you want to be able to clearly show how deployed controls meet the requirements of certain standards. This is most easily facilitated if the reports are grouped by functional category and then the data is mapped to those categories. To manage expectations, no product is going to pump out a generic report that exactly meets your needs. But you want to have 80% of those requirements met, as opposed to 20%.

It’s also very helpful for the vendors to take those categories (mentioned above) and associate them with specific regulations. For example, you should have a PCI oriented report that distinctly shows how the firewall records substantiate that you have a firewall in place to protect access to cardholder data.

Another key requirement from the reporting engine is trending analysis. From an operational standpoint, you want to be able to see your baseline behavior and then compare different timeframes to that baseline. This will help you to REACT FASTER to emerging issues.

Finally, you will likely need to customize the reports. So similar to getting comfortable with the process to customize the interface, you need to understand and like the process to tune your reports. The vendor will provide a set of pre-built reports and that’s a start, but if you need a PhD to generate a new report, it may not be the right product for you.

So that’s it, I think those are the 4 key selection criteria for a log management platform. To reiterate, your specific requirements will vary a bit, but if you pay attention to the Big 4, you’ll likely make a log management decision that you’ll be happy with for many years.

Industry News

Data loss prevention trends to watch in 2008
No doubt about it, 2007 was the year that high profile data breaches splashed across the front pages with as much sensation as paint on a Jackson Pollock canvas. Experts say this is just the tip of the iceberg…

Nugache worm kicking up a storm 

Although the infamous Storm worm enters 2008 with a reputation as the world’s most dangerous botnet, security experts say there’s an up-and-comer called Nugache that could give it a run for its money.

Protect your network from zero-day attacks

EventTracker offers a unique combination of event log analysis and change detection for a comprehensive SIEM solution that is capable of detecting all forms of cyber attacks including:
• Those that are readily recognized (100 login failures between 2-3 am)
• Those that are known but not easily recognized or obvious (http traffic from a server that should not have any)
• Zero-day attacks that are so new that threat profiles have not yet been created (such as the Storm and Nugache worms doing the rounds these days)