Are you a Data Scientist?

The advent of the big data era means that analyzing large, messy, unstructured data will increasingly form part of everyone’s work. Managers and business analysts will often be called upon to conduct data-driven experiments, to interpret data, and to create innovative data-based products and services. To thrive in this world, many will require additional skills. In a new Avanade survey, more than 60 percent of respondents said their employees need to develop new skills to translate big data into insights and business value.

Are you:

Ready and willing to experiment with your log and SIEM data? Managers and security analysts must be able to apply the principles of scientific experimentation to their log and SIEM data. They must know how to construct intelligent hypotheses. They also need to understand the principles of experimental testing and design, including population selection and sampling, in order to evaluate the validity of data analyses. As randomized testing and experimentation become more commonplace, a background in scientific experimental design will be particularly valued.

Adept at mathematical reasoning? How many of your IT staff today are really “numerate” — competent in the interpretation and use of numeric data? It’s a skill that’s going to become increasingly critical. IT Staff members don’t need to be statisticians, but they need to understand the proper usage of statistical methods. They should understand how to interpret data, metrics and the results of statistical models.

Able to see the big (data) picture? You might call this “data literacy,” or competence in finding, manipulating, managing, and interpreting data, including not just numbers but also text and images. Data literacy skills should be widespread within the IT function, and become an integral aspect of every function and activity.

Jeanne Harris blogging in the Harvard Business Review writes, “Tomorrow’s leaders need to ensure that their people have these skills, along with the culture, support and accountability to go with it. In addition, they must be comfortable leading organizations in which many employees, not just a handful of IT professionals and PhDs in statistics, are up to their necks in the complexities of analyzing large, unstructured and messy data.

“Ensuring that big data creates big value calls for a reskilling effort that is at least as much about fostering a data-driven mindset and analytical culture as it is about adopting new technology. Companies leading the revolution already have an experiment-focused, numerate, data-literate workforce.”

If this presents a challenge, then co-sourcing the function may be an option. The EventTracker Control Center here at Prism offers SIEM Simplified, a service where trained and expert IT staff perform the heavy lifting associated with big data analysis, as it relates to SIEM data. By removing the outliers and bringing patterns to your attention at greater efficiencies because of scale, focus and expertise, you can focus on the interpretation and associated actions.

SIEM and the Appalachian Trail

The Appalachian Trail is a marked hiking trail in the eastern United States extending between Georgia and Maine. It is approximately 2,181 miles long and takes about six months to complete. It is not a particularly difficult journey from start to finish; yet even so, completing the trail requires more from the hiker than just enthusiasm, endurance and will.

Likewise, SIEM implementation can take from one to six months to complete (depending on the level of customization) and like the Trail, appears deceptively simple.   It too, can be filled with challenges that reduce even the most experienced IT manager to despair, and there is no shortage of implementations that have been abandoned or uncompleted.   As with the Trail, SIEM implementation requires thoughtful consideration.

1) The Reasons Why

It doesn’t take too many nights scurrying to find shelter in a lightning storm, or days walking in adverse conditions before a hiker wonders: Why am I doing this again? Similarly, when implementing any IT project, SIEM included, it doesn’t take too many inter-departmental meetings, technical gotchas, or budget discussions before this same question presents itself: Why are we doing this again?

  All too often, we don’t have a compelling answer, or we have forgotten it. If you are considering a half year long backpacking trip through the woods, there is a really good reason for it.   In the same way, one embarks on a SIEM project with specific goals, such as regulatory compliance, IT security improvement or to control operating costs.   Define the answer to this question before you begin the project and refer to it when the implementation appears to be derailing. This is the compass that should guide your way.   Make adjustments as necessary.

2) The Virginia Blues

Daily trials can include anything from broken bones to homesickness, a circumstance that occurs on the Appalachian Trail about four to eight weeks into the journey, within the state lines of Virginia. Getting through requires not just perseverance but also an ability to adapt.

For a SIEM project, staff turnover, false positives, misconfigurations or unplanned explosions of data can potentially derail the project. But pushing harder in the face of distress is a recipe for failure. Step back, remind yourself of the reasons why this project is underway, and look at the problems from a fresh perspective. Can you be flexible? Can you make find new avenues to go around the problems?

  3) A Fresh Perspective

In the beginning, every day is chock full of excitement, every summit view or wild animal encounter is exciting.   But life in the woods will become the routine and exhilaration eventually fades into frustration.

In  much the same way, after the initial thrill of installation and its challenges, the SIEM project devolves into a routine of discipline and daily observation across the infrastructure for signs of something amiss.

This is where boredom can set in, but the best defense against the lull that comes along with the end of the implementation is the expectation of it. The journey’s going to end.   Completing it does not occur when the project is implemented.   Rather, when the installation is done, the real journey and the hard work begins.

Will the cloud take my job?

Nearly every analyst has made aggressive predictions that outsourcing to the cloud will continue to grow rapidly. It’s clear that servers and applications are migrating to the cloud as fast as possible, but according to an article in The Economist, the tradeoff is efficiency vs. sovereignty.   The White House announced that the federal government will shut down 178 duplicative data centers in 2012, adding to the 195 that will be closed by the end of this year.

Businesses need motivation and capability to recognize business problems, solutions that can improve the enterprise, and ways to implement those solutions.   There is clearly a role for outsourced solutions and it is one that enterprises are embracing.

For an engineer, however, the response to outsourcing can be one of frustration, and concerns about short-sighted decisions by management that focus on short term gains at the risk of long term security. But there is also an argument why in-sourcing isn’t necessarily the better business decision:   a recent Gartner report noted that IT departments often center too much of their attention on technology and not enough on business needs, resulting in a “veritable Tower of Babel, where the language between the IT organization and the business has been confounded, and they no longer understand each other.”

Despite increased migration to cloud services, it does not appear that there is an immediate impact on InfoSec-related jobs.   Among the 12 computer-related job classifications tracked by the Department of Labor’s Bureau of Labor Statistics (BLS), information security analysts, along with computer and information research scientists, were among those whose jobs did not report unemployment during the first two quarters of 2011.

John Reed, executive director at IT staffing firm Robert Half Technology, attributes the high growth to the increasing organizational awareness of the need for security and hands-on IT security teams to ensure appropriate security controls are in place to safeguard digital files and vital electronic infrastructure, as well as respond to computer security breaches and viruses.

Simply put: the facility of using cloud services does not replace the skills needed to analyze and interpret the data to protect the enterprise.   Outsourcing to a cloud may provide immediate efficiencies, but it’s   the IT security staff who deliver business value that ensure long term security.

Learning from Walmart

H. Lee Scott, Jr. is the current CEO of WalMart. On Jan 14, 2009, he reflected on his 9 year tenure as CEO as a guest on the Charlie Rose show.

Certain basic truths, that we all know but bear repeating, were once again emphasized. Here are my top takeaways from that interview:

1) Listen to your customers, listen harder to your critics/opponents, and get external points of view. WalMart gets a lot of negative press and new store locations often generate bitter opposition from some locals. However the majority (who vote with their dollars) would appear to favor the store. WalMart’s top management team who consider themselves decent and fair business people, with an offering that the majority clearly prefers, were unable to understand the opposition. Each side retreated to their trenches and dismissed the other. Scott described how members of the board, with external experience, were able to get Wal-Mart management to listen carefully to what the opposition was saying and with dialog, help mitigate the situation.

2) Focus like a laser on your core competency. Walmart excels at logistics, distribution, store management — the core business of retailing. It is, however, a low margin business. With its enormous cash reserves should Wal-Mart go into other areas e.g. product development where margins are much higher? While it’s tempting, remember “Jack of trades, Master of none”? 111th Congress?

3) Customers will educate themselves before shopping. In the Internet age, expect everybody to be better educated about their choices. This means, if you are fuzzy on your own value proposition and cannot articulate it well on your own product website, then expect to do poorly.

4) In business – get the 80% stuff done quickly. We all know that the first 80% goes quickly, it’s the remaining 20% that is hard and gets progressively harder (Zeno’s Paradox ). After all more than 80% of code consists of error handling. While that 20% is critical for product development, it’s the big 80% done quickly that counts in business (and in government/policy).

The fundamentals are always hard to ingrain – eat in moderation, exercise regularly and all that. Worth reminding ourselves in different settings on a regular basis.

Ananth