Do Smart Systems mark the end of SIEM?

IBM recently introduced the IBM PureSystems line of intelligent expert integrated systems. Available in a number of versions, they are pre-configured with various levels of embedded automation and intelligence depending upon whether the customer wants these capabilities implemented with a focus on infrastructure, platform or application levels. Depending on what is purchased, IBM PureSystems can include server, network, storage and management capabilities. These are ‘smart’ systems that include the ability to monitor and adapt automatically to optimize performance and resource allocation based on pre-defined criteria.

These systems will significantly impact IT operations and staff in multiple ways, and raise the question of whether or not automated, integrated intelligence in monitoring and management threaten the future of SIEM.

The evolution and pace of change in services and variability in user demand for those services strains IT staff resources as they must monitor, manage and adjust infrastructure allocation. SIEM complements and improves overall systems management. IT staff must have an integrated view of operations, business needs and service delivery. They need significant help to be able to provision, configure, adapt and allocate available computing infrastructure assets (servers, network, storage and applications) to meet changing needs of the workload and the business environment. IT cannot succeed if they must rely on manual methods to apply policy-based expertise to change, release, provisioning, configuration and event management.

IT’s success depends upon their ability to be freed from a focus on the idiosyncrasies of the physical infrastructure. They need integrated, automated management solutions that allow them to concentrate on how to create new services and extract value from that infrastructure. Their responsibility is to get the best out of the infrastructure to address the problem or to create a new service. One of the long term arguments for the benefit of computerized systems is the promise of automating and consolidating operations, management and maintenance functions where it makes sense and is feasible. Typically, this has been done by in-house projects, scripts, manual instructions and directions that have been refined and passed along from expert to expert.

What’s new now is that vendors have taken on the task of integrating and embedding management and operational intelligence based on experience, best practices and expertise into pre-configured systems. The idea leverages a far broader and deeper knowledge base. At the same time, policies, technologies, processes, operating conditions are neither consistent nor identical for all users. Therefore, they have to be able to easily change and modify the embedded expertise over time and to make use of the wisdom of local staff. Therefore, we believe these systems increase the value to be derived from SIEM solutions and enable IT staff to more effectively leverage the data and insight they obtain from SIEM solutions.

Looking Forward

Let’s consider what this means. We have been approaching the limits of exploiting the speed and computation of hardware. With the emergence and embrace of virtualization, we’ve seen how manipulation by software can improve the utilization and performance of infrastructures. There is a lot more to be gained as we get smarter about such manipulations. The announcement of intelligent, integrated expert systems is a significant step forward toward eliminating the hard line that divides hardware and software as independent entities. We also know that it isn’t the technology and infrastructure that is the most critical for enterprise operations – it’s the workload or service to the user that is most important.

The implementation of intelligent systems, along with the evolution of the cloud architecture and efforts directed at defining interoperability standards for applications moves us along the path to an operating environment where the service/workload interact with the infrastructure to adapt automatically to meet the delivery goals of the enterprise that is providing and/or consuming the service.

Finally, it is the ingenuity and knowledge acquired from data available to the expert user that translates into the wisdom of successful operations. It all starts with the data, and it is only with and through that data that successful management is possible.

Learning from JPMorgan

The single most revealing moment in the coverage of JPMorgan’s multibillion dollar debacle can be found in this take-your-breath-away passage from The Wall Street Journal: On April 30, associates who were gathered in a conference room handed Mr. Dimon summaries and analyses of the losses. But there were no details about the trades themselves. “I want to see the positions!” he barked, throwing down the papers, according to attendees. “Now! I want to see everything!”

When Mr. Dimon saw the numbers, these people say, he couldn’t breathe.

Only when he saw the actual trades — the raw data — did Mr. Dimon realize the full magnitude of his company’s situation. The horrible irony: The very detail-oriented systems (and people) Dimon had put in place had obscured rather than surfaced his bank’s horrible hedge.

This underscores the new trust versus due diligence dilemma outlined by Michael Schrage. Raw data can have enormous impact on executive perceptions that pre-chewed analytics lack.   This is not to minimize or marginalize the importance of analysis and interpretation; but nothing creates situational awareness faster than seeing with your own eyes what your experts are trying to synthesize and summarize.

There’s a reason why great chefs visit the farms and markets that source their restaurants:   the raw ingredients are critical to success — or failure.

We have spent a lot of energy in building dashboards for critical log data and recognize the value of these summaries; but while we should trust our data, we also need to do the due diligence.