Data Is The New Gold In The Digital World

Thanks to the Internet we have gazillions of data-bits are being created every day. On the positive side we have millions of songs at our finger-tips that are being listened to and liked or disliked, billions of transactions that are being conducted and an equally larger number of emotional statements that signify what customer like our companies, our products or our services. On the negative side   we have millions of songs no one ever listened to, millions of fraudulent transactions, and social emotions and comments criticizing some company, product or service. All this is changing the very definition of how we select, market and take, or will take, decisions in the connected enterprise being created around us right now. In this process of the connected value-chain, our evolution to a data and information driven economy where the most valuable currency is the underlying data. This new economy, as every change before, is going to replace traditional tasks and jobs with new ones. In this economy we are seeing the first glimpse of Peter Drucker’s ’knowledge workers’ of the early 60’s being replaced by ‘Business Value Workers’ (BI Valuenomics,2010) of the 21^st century.

The best case scenario of this comes from the movie Moneyball where Billy Bean, a manager of a small baseball team in San Francisco, uses data to almost win the World Series. Here is a conversation between John the owner of Boston Sox and Billy Bean as he tries to convince Billy to join the Boston Six as their manager..

Quoting the conversation between John and Billy at Boston we find a lot of lessons, I have put my analytics reference in (brackets) :  "One of the greatest things about money (data) is that it does not care what baseball (Analytics) thinks or what it doesn’t think" he continues "You have won (Successful BI Projects) the exact number of games the Yankees won. But the Yankees paid $1.4 million per win (successful BI Project) and you paid $260 thousand. The first guy through the wall (with a new idea that actually works) always gets bloody, always. This is threatening not just the way of doing business, but in their minds it is threatening the Game (traditional experts) . But, what it’s really threatening is their livelihood, their jobs. It's threatening the way they have been doing things and every time something like that happens whether it’s the government or the way of doing business or whatever it is, the people who are holding the reigns holding the switch -they will simply go bat-shit crazy" and with a slow pause he continues, "Anybody that's not tearing down their team right now (adapting their SAP HANA methodologies) , and rebuilding it using your model (Scientific principles of a net new platform) - they're dinosaurs. They'll be sitting on their backside, on their sofas, watching the Red Sox (your company) win the World Series (deploy HANA with higher quality at lower cost and SLA than competition)”. The data principles of Billy Bean became legendary and slowly established global scientific standards and processes in baseball team management, replacing the traditional ‘gut-feel’ processes.

Data has already began to disrupt the very essence of global businesses, legacy industries and redefine how we all must conduct business today. As we move forward this acceleration on big-data driven solutions will further disrupt traditional, and technocratic, ways of doing business. While we review and try to understand the power of data, there are companies like Amazon that has already changed the  books and now retail industry, and Google and Amazon that are still in the pure data-play game now expanding to enterprise services, iTunes and Spotify that is already changing the music industry,. For example the music industry that was once dominated by conservative talent hunters, whose sole job was selection of new artists very much like the coaches in baseball, who kept hiring and working in accordance to traditional and subjective methodologies.

The digital and IoE impact on the music industry:-Today an artist like Justin Bieber, Lady Gaga, Sia and a host of new mega singers no longer depend on the personal whims and fancies of traditional talent manager.  New customers today buy music based on peer-to-peer approvals in the form of likes and dislikes. The new breed of musicians are no longer dependent on traditional recording studios or music companies. Traditional music companies that used to find one great track to sell a whole DVD have to now face the disruption caused by likes of iTune’s and the world of 99 cent single songs. The music industry is now dominated by the digital selection, success and marketing. Predictions in the music industry will be determined by identifying digital success prior to competition.

The changes being brought by the digital and big-data disruption is similarly changing how defense forces fight battles on the digital front, how farmers optimize fertilizers and watering to get every higher yields, how mining industries are heading towards the IoT managed future of mine operation without humans and IoE connected devices, where healthcare and hospitals are now entering the totally connected environment of IoE devices and 7x24 remote monitoring of critical tasks. Every industry is facing an explosive fork to their future with one path leading towards the adoption of the digital optimization with the future of big-data, IoE networked devices and the high volume, velocity and variety of data these devices are generating every minute.

At the foundation of the digital IoE, the logical evolution from IoT, solutions is data. Just above the data foundation is network and communication devices. How we generate, communicate, store, select and filter all this data is now defining the strategic future of most enterprises.

Similarly, in our BI world the question as to whether we should continue to use the skills of partners, experts and resources who remain experts in traditional BW and DW as solutions architects, modelers and designers. Experts who that have delivered global BI solutions where ‘Fewer than 30% of BI projects will meet business expectations’ (Gartner’s 2012 BI report), is a question we need to critically analyze as we step into the big-data environment. Once we enter the Big-Data arena we will realize that Big-Data is less about accumulating data and more about identifying the business drivers and using that to filter data into bite-able chunks, without this by 2016 most companies will be exploding at their seams accumulating 99% of data that they will never need, modeled into un-optimized containers that contribute to waste.

Big data is not an area where you want to start wrong and then correct the path as you go along. You need blueprint your five year strategy today and then plan on how to achieve the highest quality at the lowest cost today.