Index now, understand later

Late binding is a computer programming mechanism in which the method being called upon an object or the function being called with arguments is looked up by name at runtime. This contrasts with early binding, where everything must be known in advance. This method is favored in object-oriented languages and is efficient but incredibly restrictive. After all, how can everything be known in advance?

In EventTracker, late binding allows us to continue learning and leveraging new understanding instead of getting stuck in whatever was sensible at the time of indexing. The upside is that it is very easy to ingest data into EventTracker without knowing much (or anything) about its meaning or organization. Use any one of several common formats/protocols, and voila, data is indexed and available for searching/reporting.

As understanding improves, users can create a “Knowledge Pack” to describe the indexed data in reports, search output, dashboards, co-relation rules, behavior rules, etc. There is no single, forced “normalized” schema and thus no connectors to transform incoming data to the fixed schema.

As your understanding improves, the knowledge pack improves and so does the resulting output. And oh by the way, since the same data can be viewed by two different roles in very different ways, this is easily accommodated in the Knowledge Pack. Thus the same data (e.g., Login failures) can be viewed in one way by the Security team (in real time, as an alert, with trends) and in an entirely different way by the Compliance team (as a report covering a time-span with annotation to show due care).

SIEM: What are you searching for?

Search engines are now well established as a vital feature of IT and applications continue to evolve in breadth and depth at dizzying rates.  It is tempting to try and reduce any and all problems to one of query construction against an index. Can Security Information and Event Management or SIEM be (force) fitted into the search paradigm?

The answer depends on what you are looking to do and your skill with query construction.

If you are an expert with detailed knowledge of log formats and content, you may find it easy to construct a suitable query. When launched against a suitably indexed log collection, results can be gratifyingly fast and accurate. This is however a limited use-case in the SIEM universe of use-cases. This model usually applies when Administrators are seeking to resolve Operational problems.

Security analysts however are usually searching for behavior and not simple text searches. While this is the holy grail of search engines, attempts from Excite (1996) to Accoona (RIP Oct 2008) never made the cut. In the SIEM world, the context problem is compounded by myriad formats and the lack of any standard to assign meaning to logs even within one vendor’s products and versions of a product.

All is not lost, SIEM vendors do offer solutions by way of pre-packaged reports and the best ones offer users the ability to perform analysis of behavior within a certain context (as opposed to simple text search). By way of example – show me all failed logins after 6PM; from this set, show only those that failed on SERVER57; from this set show me those for User4; now go back and show me all User4 activity after 6PM on all machines.

Don’t try this with a “simple” text search engine….or like John Wayne in The Searchers, you may become bitter and middle aged.

– Ananth