Search engines are now well established as a vital feature of IT and applications continue to evolve in breadth and depth at dizzying rates. It is tempting to try and reduce any and all problems to one of query construction against an index. Can Security Information and Event Management or SIEM be (force) fitted into the search paradigm?
The answer depends on what you are looking to do and your skill with query construction.
If you are an expert with detailed knowledge of log formats and content, you may find it easy to construct a suitable query. When launched against a suitably indexed log collection, results can be gratifyingly fast and accurate. This is however a limited use-case in the SIEM universe of use-cases. This model usually applies when Administrators are seeking to resolve Operational problems.
Security analysts however are usually searching for behavior and not simple text searches. While this is the holy grail of search engines, attempts from Excite (1996) to Accoona (RIP Oct 2008) never made the cut. In the SIEM world, the context problem is compounded by myriad formats and the lack of any standard to assign meaning to logs even within one vendor’s products and versions of a product.
All is not lost, SIEM vendors do offer solutions by way of pre-packaged reports and the best ones offer users the ability to perform analysis of behavior within a certain context (as opposed to simple text search). By way of example – show me all failed logins after 6PM; from this set, show only those that failed on SERVER57; from this set show me those for User4; now go back and show me all User4 activity after 6PM on all machines.
Don’t try this with a “simple” text search engine….or like John Wayne in The Searchers, you may become bitter and middle aged.