Cognitive Computing: From hype and pilot to going mainstream implementation

Cognitive Computing started in late 1960’s with initial innovation around supercomputers. However the decision making capability, simulation and self learning intelligence was limited. With major advances in computing power, cognitive behavior science, computational intelligence, the researchers in Cognitive Computing made significant advancement. With advanced medical science and neuropschology, computer scientists were able to study the mechanics of human brain and that allowed the scientists to build computative models modeled after human mind. These models would allow association with various attributes and parameters from past experiences into cognitive systems. Cognitive computing and Decision Science was born with scientists developing computers that that operated at a higher rate of speed and accuracy than the human brain did.

As per IBM, Cognitive computing systems learn and interact naturally with people to extend what either humans or machine could do on their own. They help human experts make better decisions by penetrating the complexity of Big Data. Big Data growth is accelerating as more of the world’s activity is expressed digitally. Not only is it increasing in volume, but also in speed, variety and uncertainty. Most data now comes in unstructured forms such as video, images, symbols and natural language – a new computing model is needed in order for businesses to process and make sense of it, and enhance and extend the expertise of humans. Rather than being programmed to anticipate every possible answer or action needed to perform a function or set of tasks, cognitive computing systems are trained using artificial intelligence (AI) and machine learning algorithms to sense, predict, infer and, in some ways, think.

Cognitive computing reassess the nature of relationships between multiple variables and the environment factors. It can help assess the nature of the relationship between people and their increasingly pervasive today’s digital environment. They may play the role of assistant or coach for the user, or they may act virtually autonomously in many problem-solving situations. The boundaries of the processes and domains these systems will affect are still elastic and emergent. They must learn as attribures and variables changes, and as questions and decisions evolve. They must resolve ambiguity and tolerate unpredictability. They must be engineered to feed on dynamic data in real time, or near real time. They must interact easily with users so that those users can define their needs comfortably. They may also interact with other processors, devices and cloud services, as well as with people. They must aid in defining a problem by asking questions or finding additional source input if a problem statement is ambiguous or incomplete. They must “remember” previous interactions in a process and return information that is suitable for the specific application at that point in time. They must understand, identify and extract contextual elements such as meaning, syntax, time, location, appropriate domain, regulations, user’s profile, process, task and goal. They may draw on multiple sources of information, including both structured and unstructured digital information, as well as sensory inputs (visual, gestural, auditory or sensor-provided). The output from Cognitive systems may be prescriptive, suggestive, instructive or simply entertaining.

Cognitive informatics is a cutting-edge and multi-disciplinary research field that tackles the fundamental problems shared by modern informatics, computation, software engineering, AI, computational intelligence, cybernetics, cognitive science, neuropsychology, medical science, systems science, philosophy, linguistics, economics, management science, and life sciences. The development and the cross fertilization between the aforementioned science and engineering disciplines have led to a whole range of emerging research areas known as cognitive informatics.

Sample Cognitive Computing Use Cases

  1. Shopping: Cognitive computing systems’ ability to evaluate and generate hypotheses will help retail industries to find patterns, correlations and insights in mountains of unstructured and structured data. Watson’s app development platform is already moving into this physical-virtual space. The startup, Fluid, has layered Watson on top of its Expert Personal Shopper app for retail brands. Watson will be your personal shopping assistant. Store associates will also have similar intelligent tech providing them instant product information, customer loyalty data, sales histories, user reviews, blogs and magazines, so that when you do need to talk with another human, they know exactly how to help.
  2. Medical: evidence-based learning, hypothesis generation and natural-language skills could help medical professionals make key decisions in patient diagnosis and treatment. The objective is to give Doctors and Surgeons a quick way to access diagnostic and treatment options based on updated research.
  3. Banking: In fraud detection, financial institutions could have cognitive tools that enable them to go beyond analyses of cardholders’ credit transaction histories; cognitive computing might provide them with new “associational” intelligence, such as when an individual is most likely to make purchases, what he is likely to buy, and under what circumstances.
  4. Finance: Benefits from Cognitive Computing also will be seen by financial advisors, including individuals that handle their own portfolios, as the technology enables bringing together relevant, current and personalized information and remembering questions and preferences
  5. Weather Forecasting and Planning: Weather forecasting can benefit from Cognitive Computing and big data analytics. For instance, IBM’s Deep Thunder, a research project that creates precise, local weather forecasts, can predict severe storms in a specific area up to three days before the event. This early-warning system gives local authorities and residents enough time to make preparations

As per Deloitte University Press – Cognitive analytics can help address some key challenges. It can improve prediction accuracy, provide augmentation and scale to human cognition, and allow tasks to be performed more efficiently (and automatically) via context-based suggestions. For organizations that want to improve their ability to sense and respond, cognitive analytics offers a powerful way to bridge the gap between the promise of big data and the reality of practical decision making.

IBM has calculated that the market size for cognitive computing services in the private sector is in the neighborhood of $50 billion. At present, there are very few vendors in the field. While IBM has announced the creations of the Watson Group to commercialize cognitive computing and Google has acquired AI startup Deepmind, there are few companies in the space. Much of the work is still happening at a university level or within research organizations. Cognitive computing is still early from a commercialization perspective. It is likely another three to five years before an industry wide adoption and its impact on a wide range of companies. For a while at least, cognitive computing will fill a niche in industries where highly complex decision making is the norm, such as healthcare and financial markets. Eventually, it will become a normal tool in every corporate toolbox, helping to augment human decision making.


Top 10 Big Data and Analytics Predictions for 2015

Predictions for 2015

Forrester’s forecast that Hadoop will become an “enterprise priority” in the next 12 months, International Data corp. has just gazed into its own crystal ball, and sees a future where spending on Big Data analytics is set to grow three times faster in 2015.

Here’s our prediction of Top 10 Big Data and Analytics trends for 2015

  1. Rise of Chief Data Officer and Data Scientists positions
  2. Cloud Computing and Cloud Data Warehouse will grow adoption
  3. Data Visualization tools will become very popular with business users with advanced visualization capabilities
  4. Data Integration Hub at most large and mid-scale enterprise will expand to include unstructured data and incorporate advanced and predictive analytics using machine learning and advanced capabilities such as Watson
  5. Integration with Enterprise Mobile Apps and Big data technology on the rise
  6. Organizations will move from data lakes to processing data platform. Self-service Big Data with predictive analytics and advanced visualizations will go mainstream
  7. Companies and organizations will struggle to find data engineers and data scientist talent. Many legacy ETL/Integration and EDW resources will fill the gap.
  8. The Internet of Things (IoT) will be the next critical focus for data/analytics services
  9. Data Providers will evolve rapidly into new business model
  10. Data Privacy and security will gain momentum in implementation

While we can’t know for sure if each of these things will come true, we do know that the world of big data is changing. It’s no longer about just having access to data and the ability to store it, but instead the ability to achieve actionable results with data through predictive and prescriptive analytics. Only time will tell how this evolves, but if you aren’t leveraging data to compete and win, it’s time to get on board. The big data and analytics market will reach $125 billion worldwide in 2015, according to IDC and if you are not investing or planning to invest, you are already behind your competitor or losing your spot from being an A player.

Building your Data Science Team: Business Engineering Unit

Every day, we create 2.5 Quintillion bytes of data – so much that 90% of the data in the world has been created in last 2 years alone. Every organization is now accumulating Terabytes and Petabytes of data coming from various devices – machines, mobile, user, web logs and cookies, social media, application servers and transactional logs, etc. Organizations are rushing to store this wealth of information fearing missed opportunities. The challenge is not in storing this information but able to find usage of this data to its competitive advantage.

Leading organizations are rapidly reorganizing themselves and building the Data Science Team. This is very similar pattern that you may have noticed in early to mid 90’s wherein organization started with Business Intelligence team rebranding MIS or DSS teams. The key to competitive advantage is not by storing all and most of the data but by deriving value and insight from it and be able to tie it with business plan that can drive tangible business outcomes. This new function that is evolving to be a Data Science team primarily needs four tiered layer of experts: Data Science Champion, Business/Product Managers, Analytical Data Modeler and Big Data Engineer. To make the Data Science team successful, it’s key that they operate under a Data Science Champion such as Chief Data Officer or Chief Data Scientist and not under a traditional IT organization.

Let us define what each of these layers mean to an organization.

§         Data Science Champion: This is an Executive level sponsor such as Chief Data Officer or Chief Data Scientist. They lay out the vision and lead the mission for the team. This champion is a domain expert in a data science field. A large organization may have several Chief Data Scientist roles specific to LOB working for a Chief Data Officer.

§         Business/Product Managers: They are the product managers from the business that closely partners with the CDO or Chief Data Scientist

§         Analytical Data Modeler: These are the advanced mathematician and statisticians who can apply their computer skills to create complex analytical models largely contributing to the mission

§         Big Data Engineer: These are the computer science engineers who apply the sophisticated engineering skills to process large volumes of datasets using big data tools.

It’s hard to find a Data Scientist who will have all the above 4 skills – leadership, business knowledge, analytical experience, big data processing skills. A successful Data Science Team is partnership of above defined four distinct roles. When building your team, it’s important to focus on these key data science skills: analytical and curiosity mind, creativity, domain knowledge, advanced math skills including a solid background in calculus, geometry, linear algebra, and statistics and a computer science background. Leadership, Analytical and Computer Science skills are important, particularly for the first members chosen for your data science team.

A new concept of Business Engineering Unit under the leadership of CDO/Chief Data Scientist is slowly evolving that will house the Data Science team. This has been seen in some of the large retailers, telecom, and media/entertainment companies. Data Science deals with identifying the real problem or a business opportunity. Unlike IT as a support organization in many companies, the Business Engineering Unit needs to be a profit center that will contribute to the company’s topline or bottomline. To prove the value from Data Science, the Champion needs to initially focus on the hardest business problems within an organization or unique business opportunity that have the highest return for key stakeholders. Organization boundaries and internal political environment are often the biggest challenges facing a data science team.

Companies that have made investments in big-data computing will reap extraordinary near-term and long-term benefits. Data Science is perhaps the biggest innovation in computing in the last decade.  The benefits from data science have already been proven in some industry sectors; the challenge is to extend the technology and to apply it more widely and in all facets of interaction between humans and machines.

Top 10 Big Data Trends in 2014

In January 2014, IDG published their latest big data enterprise survey and predictions for 2014 finding that on average, enterprises will spend $8M on big data –related initiatives in 2014. The study also found that 70% of enterprise organizations have either deployed or are planning to deploy big data-related projects and programs.

Here’s our Top 10 Big Data Trends in 2014:

  1. Big Data as a Service and Big Data Analytics will go mainstream
  2. More companies will implement Predictive Analytics, Machine Learning
  3. Data Science and Big Data Analytics will be embedded in BI for actionable Insights into Operational Reports and Executive Dashboards
  4. Cloud computing and Big Data will be tightly integrated with BI solutions
  5. Enterprise will be using big data techniques to secure IT infrastructure
  6. Hadoop will be used for operational system and transactional application
  7. Hadoop will be implemented as extensions to part of Enterprise Information Management solutions
  8. Big Data and Data Scientists skills shortage will grow as companies start ramping up hiring for big data and data science projects
  9. Rise in M&A activity in Big Data space with legacy BI companies acquiring niche big data vendors
  10. Companies will start new roles defined as Chief Data Scientists, Chief Data Officers and Chief Analytic Officer

TechNavio’s analysts forecast the Global Big Data market to grow at a CAGR of 34.17 percent over the period 2013-2018. TechNavio’s report, the Global Big Data Market 2014-2018.

The key vendors dominating this market space are IBM Corp., Hewlett-Packard Co., Oracle Corp., and Teradata Corp.

Other vendors are 1010data Ltd., 10gen Inc., Accenture Inc., Amazon Web Services, Attivio Inc., Calpont Corp., Capgemini Inc., ClickFox Inc., Cloudera Inc., Computer Sciences Corp., Couchbase Inc., Datameer Inc., DataStax Inc., Dell Inc., Digital Reasoning Systems Inc., EMC Corp., Fractal Analytics Inc., Fujitsu Ltd., Hitachi Ltd., Hortonworks Inc., HPCC Systems Inc., Huawei Technologies Co. Ltd., Informatica Corp., Intel Corp., Karmasphere Inc., Logica plc, MapR Technologies Inc., MarkLogic Inc., Microsoft Corp., Mu Sigma Inc., NetApp Inc., Opera Solutions Inc., ParAccel Inc., Pervasive Software Inc., QlikTech Ltd., RainStor Inc., Red Hat Inc., SAP AG, SAS Institute Inc., Seagate Inc., Siemens Information Systems Ltd., Splunk Inc., Supermicro Computer Inc., Tableau Software Inc., Tata Consultancy Services Ltd., Think Big Analytics Inc., and Xerox Corp.

Top Big Data Posts for Year 2013 !!!

To close the busy 2013 year, here is the list of Top blogs for the Year 2013 by TBDI.

Total Hits: 16200+

Total Followers: 2100+

Total Basic Member: 10000+

We are growing rapidly! Thanks to all your support!

Home page / Archives 3,660
Big Data Use-Cases – Retail, Manufacturing and Auto Industry 1,515
Big Data Use-Cases in Telecom, Media and Entertainment industry 1,500
Big Data Use-cases – Banking and Financial Services 1,415
Our Favorite 40+ Big Data use-cases. What’s your? 1,129
Big Data Use-cases – Insurance, Energy and Utilities, Travel and Hi-Tech 1,080
Big Data Use-Cases in Healthcare – Provider, Payer and Care Management 896
10 Big Data Implementation Best Practices 887
Introduction to Big Data and Hadoop Ecosystem – For Beginners! 652
Will Hadoop replace or augment your Enterprise Data Warehouse? 486
7 steps to Advanced Predictive Analytics! 377
About TBDI 270
Top 10 Blogs so far this year 212
Addressing the big data security! 189
Wonder how to start your Big Data POC / Lab? 165
Hadoop meets SQL 149
Building your Launch Team! 120
Big Data Scientists, Architects, Lead and Developer Needed for IBM – Immediate Need!!! 119
The Hadoop Data Warehouse! 114
Running Hadoop in the Cloud 108
6 Steps to Start Your Big Data Journey 105
Big Data making the difference in Retail industry! 96
Big Data Analytics – Acquire, Grow and Retain Customers 89
Using Big Data Analytics in Healthcare 83
Join TBDI – The Big Data Institute! 77
Invitation to The Big Data Institute (TBDI) Partnership program 70
Who will drive your Big Data Bus? Executive leadership in a new age – By Joseph Baird 57
Analytics in Banking Services 45
Predicting Customer Behavior with Analytics! 44
Facebook and Twitter Social Media Analytics – From White House to Boardroom! 38
A new type of Blog? What Big Data might mean to the business 29
Welcome to new Partners and Alliances 27
Big Data = more diverse data 24
I wonder if exploiting Big Data will enable big companies to grow even bigger or whether it will enable smaller companies to compete with them to level the playing field? 23
Big Data…Defi… 21
Regulation – a class of Big Data apps 21
Join us this Thursday, May 2nd for our next Big Data Developer Meetup! 17
Making the most of what you have 17
Policy for Establishing New TBDI Chapters 13
TBDI Definition… 10
TBDI Big Data & Data Science Meetup in So California, USA 8
Natural Selection in Business – Does using Big Data provide a sustainable advantage? 5
The TBDI News! 4
Reserve your seat now – TBDI Big Data MeetUp: Big Data Analytics and Data Science – 2014 Trends 4
Being Bold and Big with your Big Data Pilots 4