Predictive analytics encompasses a variety of statistical techniques from modelling, machine learning, data mining and that analyze current and historical facts to make predictions about future events. In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions.
Till late 2010, most of the enterprise business intelligence focus and analytics was around structured and some of semi-structured data (emails, logs and call records). The buzz words were ‘data-driven’ business decisions or actions. With era of big data and technology evolution, organizations are now venturing into unstructured data like sensor data from RFID tags, online web content and need for better analytical capabilities. The new buzz words that’s rapidly gaining popularity from meetings to conferences is ‘advanced analytics’ or ‘analytics-driven’ or ‘big insights’. With big data technology, the power of predictive analytics is getting a lot of coverage, and software vendors are touting the latest and greatest in technology and algorithms. Once seemed as boring and dull job or data geeks, the demand for statisticians and data scientists are high and rising. One of the recent articles from Harvard Business Review (HBR) – Oct 2012, it talks about Data Scientist: The Sexiest Job of the 21st Century.
As noted in HBR article, ‘Data are essential, but performance improvements and competitive advantage arise from analytics models that allow managers to predict and optimize outcomes. More important, the most effective approach to building a model rarely starts with the data; instead it originates with identifying the business opportunity and determining how the model can improve performance. According to research by Andrew McAfee and Erik Brynjolfsson, of MIT, companies that inject big data and analytics into their operations show productivity rates and profitability that are 5% to 6% higher than those of their peers. Often organizations look for help to start with their Advanced Predictive Analytics project. There are very limited production processes that leverage the power of Predictive Analytics that is embedded in BPM or decision making processes. In my recent experience from a Analytics Strategy and Assessment workshop for a client, the issue identified was not the tool or architecture but how to get insights from myriads of data that exist. The questions posed was a) Are we collecting and storing right data? b) What insights can be generated from this data? What organizations want to know is not what kind of technology to buy first or what techniques and training they need, but what kind of problem to go looking for. What kind of problem will show the greatest return on an investment in predictive analytics? Where can they apply predictive analytics and get a clear and compelling “win”?
Based on my experience, here’s the key 7 steps to get started with Advanced Predictive Analytics
1. Define the Problem or Pain or Opportunity
2. Identify the key Metrics
3. Identify the Right Data that support metrics #2 above
4. Analyze and Enrich the data
5. Build models for Advanced Predictive Analysis
6. Experiment the model with test subject/group
7. Embed and Implement the analytics as part of business process or application
Most of the organizations are lost in the process of identifying or collecting data before documenting the step# 1. As mentioned above from my recent experience, the organization was collecting and storing all possible data without defining what insights they want to generate to drive business at tactical and strategic level. Operational decisions align well with Predictive Analytics model. Most of the operational and tactical decisions are made by front line field level staff from call centers to customer service to sales reps. Predictive Analysis that can be embedded into business applications as part of workflow processes can add a tremendous value to providing an excellent service to customers as well as significantly increase results and outcome from cross-sell or up-sell opportunities. Many predictive analytic tools support access to a wide range of data sources, including those typically branded “big data,” such as unstructured text , or semi-structured Web logs and sensor data. The problem is that organizations are trying to apply these technologies to the wrong problem. With the urge to prove ROI from investments in Big Data and Analytics, organizations focus on large and wider problems than focusing on Operational level or tactical problem that can give an opportunity to implement and prove the solution.
Predictive models analyze past performance to assess how likely a customer is to exhibit a specific behavior in the future in order to improve marketing effectiveness. This category also encompasses models that seek out subtle data patterns to answer questions about customer performance, such as fraud detection models. Predictive models often perform calculations during live transactions, for example, to evaluate the risk or opportunity of a given customer or transaction, in order to guide a decision. Big data and predictive analytics can be combined together in the operational environment. By focusing on operational decisions, we can put big data to work, using it to drive predictions that improve our ability to make good decisions at the operational level. With advancement in computing speed, individual agent modeling systems can simulate human behavior or reaction to given stimuli or scenarios. The new term for animating data specifically linked to an individual in a simulated environment is Avatar Analytics.
From my recent analytics on Social media marketing for a specific brand, we found out some interesting statistics like 23% of customers repurchase on same day and most of the repurchases happens within 5 days of initial purchase. These are key insights for marketing campaign. Healthcare and Financial organizations has a huge potential to leverage predictive analytics for fraud detection, cross-sell and up-sell and interventions. In my experience at one of the leading healthcare companies where I managed the BI and Analytics team, analytics played a significant role on how members/patients were stratified by risk scores using statistical models. Organizations are gaining momentum in leveraging big data for insights and advanced analytics using statistical model and predictive modeling across many industry domains. Whether it is using analytics to predict customer behavior, set pricing strategy, optimize ad spending or manage risk, analytics is moving to the top of the management agenda.
The Big Data Institute 