In 2013 big-data has evolved from being a question mark to a business fact, big data truly got BIG in 2013
Thanks to NSA big-data is now big-news. The questionable access to big-data, its use in real-time mining, national security and a host of other needs have exposed main-streets capabilities to the power of big data.
Structured, semi-structured and unstructured have moved from concepts to reality, as have data architects and data scientists. What we like? Who we like to talk to? When we buy? How we buy? Where we go? are all becoming very valuable questions for companies seeking an ever larger share of your minds and spend. Proactive companies are harvesting the business value of ‘Business-NOW’ and real-time decisions at local and global levels.Big data has come a long way and has stabilized in 2013. Nascar now runs big data analytics on every car on the track with a clear aim at reducing accidents and improving the safety of cars. They track the emotions of the drivers and predict their propensity to take rash or irrational decisions. In the air real-time data comes at the rate of four to six terabytes per engine on a flight. Manufacturers are installing devices that talk to each other, the internet of things, and alert operators of service requirements, or care takers that an accident has taken place - please send an ambulance to a designated GPS location. The interesting thing is that 99.99% of all this data is redundant, i.e. all is well. It is the 0.005% of data that says something is not right that needs to be highlighted, in real-time, to an alert level for it to provide true business solutions. In 2013 we see manufacturers, retailers, airlines, military, pharmaceuticals and almost all industries start to wet their toes into big-data, and every company that has tried it has come back with raw diamonds that they will now need to polish and hone to perfection.
Big-data today is the beginning of the next age- the information ageBig data is the equivalent of the first few gallons of oil that Rockefeller refined to become the gasoline king of days gone by. His gasoline drove the ford cars and launched the transportation revolution. It took machines from clunky steam engines to our modern gasoline engines. On one side big-data is helping the world and humanity by analyzing cloud and weather patterns to predict droughts and famines, or to predict the spread of diseases. On the other side the same data is funded by major corporations to forecast where the next famine will be so they can buy grain that these countries will need in the near future. While the red-cross is using big data to plan their logistics and doctors others are using it to snoop on their citizens and their conversations. However, no matter how we look at this there are volumes of data being created and there is untold business value in that data from those that are willing to harness it in a scientific manner. According to Gartner over 64% of companies interviewed had started investing into big-data analytics in one form or another. Companies like BOFA, HSBC, AIG, Federal Communications Commission, NSA, have all hired experienced data scientists into the new position of Chief Data Officer realizing that data is the new form of currency and proactive companies are beginning to realize that data alone is not enough, for without information that data can often become a hindrance to true business value. These companies have installed ‘Chief Business Value Architects’ with only one aim- meet business expectations ( a goal set by Gartner in 2009 but coming to fruition in 2013-14)
Lessons learned in 2012-13 with HANA as the sample solution
1. HANA is not just a technology installation: Enterprises that have installed SAP HANA or big-data as a technology installation have hit various technocratic hurdles. Whereas, companies that have looked at ‘Business Value and business benefits’ with a clear question of show-me-the-business-benefit attitude have truly gained from these installations.
2. Technology is the foundation: The fact remains that though technology is not to goal it is the foundation. Big-data and HANA are possible because of the technology advancements. While technology is the starting point it is also imperative to ensure that companies partner with technology experts that understand business value.
3. What got you here will not take you there: There is a high probability that 50-70% of reports in your production BI environment are either not being used, or will never be used, by your business users. There is no benefit in following the same architecture, standards and processes that got you to this mess in the first place. In 2013 Gartner stated “This is a time of accelerating change, where your current IT architecture will be rendered obsolete. You must lead through this change, selectively destroy low impact systems, and aggressively change your IT cost structure. This is the New World of the (big-data), the next age of computing.” Mr. Sondergaard, Sr. Analyst at Gartner. Read this many times. Read between the lines. Cast it in stone and reconsider everything that has got you to this state including your methodology and partners. Don't make the mistake of migrating to this new technology with old architecture, standards, processes or methodologies.
4. The big elephant of big data is Hadoop: Companies planning to mine unstructured data into enterprise analytics and decisions will find Hadoop as the prime provider for such data. As a technology Hadoop is a very complex beast, as a business solution there are startups that will extract and reduce the required data in quite an effortless manner. So the key is a scientific approach and not technocratic
5. The price of HANA is never fixed: Even though SAP may say the price of HANA is fixed there are many ways to reduce the tactical and strategic cost of migrating to HANA. By taking the migration of HANA in a scientific manner companies have increased quality of data by 30-50% and simultaneously reduced the cost of migrating to HANA by as much as 40 to 60%. Though it seems complex it follows simple rules of professional deployment that can be repeated across time and space. Over a period of 10 years this can be many millions of dollars saved.
6. Partner with scientific principles and Business Value Attainment: The goal has to be ‘Meet Business Expectations’. If you simply take your existing BI environment, for example, and migrate it to HANA then you will have gained very little, lost a lot and wasted a lot of time and money of your business stakeholders.
The future is limited only to the capabilities of your imagination: One fact is apparent that no one can predict the true potential of big-data and the power of HANA. We all know that this has tremendous potential and business-value and architecture are the key drivers. A few companies are understanding the technology, training their business stakeholders and jointly leveraging the true potential of the power of HANA. A little more and cleaning their existing environments and leveraging the power of HANA towards better decisions and processes. A few, on the other side of the bell curve, are simply taking their existing BW, ECC, CRM and SCM environments and slapping on the power of HANA. All of them are seeing gains but the proportional business value is highest where business decisions is the focus and not technology