Dec 28, 2013

Big Data & HANA summary for 2013

 In 2013 big-data has evolved from being a question mark to a business fact, big data truly got BIG in 2013

Thanks to NSA big-data is now big-news. The questionable access to big-data, its use in real-time mining, national security and a host of other needs have exposed main-streets capabilities to the power of big data.
Structured, semi-structured and unstructured have moved from concepts to reality, as have data architects and data scientists.  What we like? Who we like to talk to? When we buy? How we buy? Where we go? are all becoming very valuable questions for companies seeking an ever larger share of your minds and spend. Proactive companies are harvesting the business value of ‘Business-NOW’ and real-time decisions at local and global levels.
Big data has come a long way and has stabilized in 2013. Nascar now runs big data analytics on every car on the track with a clear aim at reducing accidents and improving the safety of cars. They track the emotions of the drivers and predict their propensity to take rash or irrational decisions. In the air real-time data comes at the rate of four to six terabytes per engine on a flight. Manufacturers are installing devices that talk to each other, the internet of things, and alert operators of service requirements, or care takers that an accident has taken place - please send an ambulance to a designated GPS location. The interesting thing is that 99.99% of all this data is redundant, i.e. all is well. It is the 0.005% of data that says something is not right that needs to be highlighted, in real-time, to an alert level for it to provide true business solutions. In 2013 we see manufacturers, retailers, airlines, military, pharmaceuticals and almost all industries start to wet their toes into big-data, and every company that has tried it has come back with raw diamonds that they will now need to polish and hone to perfection.

Big-data today is the beginning of the next age- the information age
Big data is the equivalent of the first few gallons of oil that Rockefeller refined to become the gasoline king of days gone by. His gasoline drove the ford cars and launched the transportation revolution. It took machines from clunky steam engines to our modern gasoline engines.  On one side big-data is helping the world and humanity by analyzing cloud and weather patterns to predict droughts and famines, or to predict the spread of diseases. On the other side the same data is funded by major corporations to forecast where the next famine will be so they can buy grain that these countries will need in the near future. While the red-cross is using big data to plan their logistics and doctors others are using it to snoop on their citizens and their conversations. However, no matter how we look at this there are volumes of data being created and there is untold business value in that data from those that are willing to harness it in a scientific manner. According to Gartner over 64% of companies interviewed had started investing into big-data analytics in one form or another. Companies like BOFA, HSBC, AIG, Federal Communications Commission, NSA, have all hired experienced data scientists into the new position of Chief Data Officer realizing that data is the new form of currency and proactive companies are beginning to realize that data alone is not enough, for without information that data can often become a hindrance to true business value. These companies have installed ‘Chief Business Value Architects’ with only one aim- meet business expectations ( a goal set by Gartner in 2009 but coming to fruition in 2013-14)

Lessons learned in 2012-13 with HANA as the sample solution
1.       HANA is not just a technology installation: Enterprises that have installed SAP HANA or big-data as a technology installation have hit various technocratic hurdles. Whereas, companies that have looked at ‘Business Value and business benefits’ with a clear question of show-me-the-business-benefit attitude have truly gained from these installations.
2.       Technology is the foundation: The fact remains that though technology is not to goal it is the foundation. Big-data and HANA are possible because of the technology advancements. While technology is the starting point it is also imperative to ensure that companies partner with technology experts that understand business value.
3.       What got you here will not take you there: There is a high probability that 50-70% of reports in your production BI environment are either not being used, or will never be used, by your business users. There is no benefit in following the same architecture, standards and processes that got you to this mess in the first place. In 2013 Gartner stated  This is a time of accelerating change,  where your current IT architecture will be rendered obsolete. You must lead through this change, selectively destroy low impact systems, and aggressively change your IT cost structure. This is the New World of the (big-data), the next age of computing.” Mr. Sondergaard, Sr. Analyst at Gartner. Read this many times. Read between the lines. Cast it in stone and reconsider everything that has got you to this state including your methodology and partners. Don't make the mistake of  migrating to this new technology with old architecture, standards, processes or methodologies.
4.       The big elephant of big data is Hadoop: Companies planning to mine unstructured data into enterprise analytics and decisions will find Hadoop as the prime provider for such data. As a technology Hadoop is a very complex beast, as a business solution there are startups that will extract and reduce the required data in quite an effortless manner. So the key is a scientific approach and not technocratic
5.       The price of HANA is never fixed: Even though SAP may say the price of HANA is fixed there are many ways to reduce the tactical and strategic cost of migrating to HANA. By taking the migration of HANA in a scientific manner companies have increased quality of data by 30-50% and simultaneously reduced the cost of migrating to HANA by as much as 40 to 60%. Though it seems complex it follows simple rules of professional deployment that can be repeated across time and space. Over a period of 10 years this can be many millions of dollars saved.
6.       Partner with scientific principles and Business Value Attainment: The goal has to be ‘Meet Business Expectations’. If you simply take your existing BI environment, for example, and migrate it to HANA then you will have gained very little, lost a lot and wasted a lot of time and money of your business stakeholders.
The future is limited only to the capabilities of your imagination: One fact is apparent that no one can predict the true potential of big-data and the power of HANA. We all know that this has tremendous potential and business-value and architecture are the key drivers. A few companies are understanding the technology, training their business stakeholders and jointly leveraging the true potential of the power of HANA. A little more and cleaning their existing environments and leveraging the power of HANA towards better decisions and processes. A few, on the other side of the bell curve, are simply taking their existing BW, ECC, CRM and SCM environments and slapping on the power of HANA. All of them are seeing gains but the proportional business value is highest where business decisions is the focus and not technology

Dec 23, 2013

BoT, BoX or BoH (BW-on-HANA)

Question: What does Radio VOE (Voice of Experience) recommend about migrating  SAP-BW to
[1] Teradata;
[2] Exadata/Exalytics; or
[3] HANA
View  1: Attributes for customers with SAP applications as source system for 50%+ of critical data

View  2: Same information different visual

1.        BW-on-Teradata (BoT)

In the early 2000’s SAP partnered with Teradata as an alternative database for BW. To cut the story short globally there is currently only one customer with BW on Teradata and probably will always remain as the only customer to have ever taken a very short trip on a BoT (BW on Teradata). You don’t want to go there or ever take this path.. end of story
So that should lay to rest any endeavor for a BoT migration
PS the customer is currently seriously considering migrating their BoT to BoH in 2013..

2.       BoX (BW-on-Exadata) vs. BoH (BW-on-HANA):-

F     For BW-on-Exadata the foundational advice would be similar to the one for BW-on-TD. If 99% of the planet is going on one path don't try to reinvent another one on your own. There is inherent safety in numbers. Radio VOE has validated that the strategic advantage of following the herd far outweighs the adventurist assumptions & costs of being a loner, ie. Having the rare honor out being the 1:5000 implementations on a unique track does carry its own risks and disadvantages. The key decision is not only about tactical cost but about strategic Business Value.
       Keep this as your 'center-of-the-universe' decision factor of your final decision.


BoX (BW on Exadata)

BoH (BW-on-HANA)

What it really is


Business Application platform

What you get with BoH


Business Application Platform

Over 300 RDS solutions

Automated Database slimming applications

Predictive analytics platform

Prescriptive analytics capabilities

What you get with SoH


Over 900 prebuilt views for real-time operational views

Prebuilt Operational analytics that come free with SoH

Access to state-of-art analytics and informatics

Predictive analytics platform and algorithms


Will need Exadata Plus Exalytics, Plus

A license for every non PRD system including DR

Data base license, Plus Analytics engine

OLAP engine

Calculation engines and other applications

Check the total and final cost of ownership

All in one HANA platform and one price for PRD HANA. All other systems in the landscape come free

What type of db is this


In Memory, Columnar

Master Node RAM availability


1TB currently, 4 TB by Q2 of ’14 and 8-12TB by Q4 of ‘14


Exadata is an appliance

BW on HANA is an appliance

ACID compliance


Fully Compliant

Need for MOLAP, Aggregates, Caches



Very High Strength

Structured data - Oracle ERP

No seamless SAP integration

No seamless comprehension to SAP  config changes

Structured Data- integration to SAP Applications

100% seamless integration to SAP Master and transactional configurations

Seamless merging of multiple master data and source system deviances – on the fly in memory mergers

Medium Strength


Seamless Integration to semi Structured data


Unstructured pure Big Data

Unstructured pure Big Data

Partnered with Hadoop providers & Hadoop startups


Box compete with Teradata

HANA has no apparent competition for SAP customers.

No of Customers


2400+ to date

Additional Needs

Will need Exalytics to provide accelerated queries

Does not need any additional accelerators

HW selection

Only from Oracle

Monopoly in HW development, support and services. No competitive forces encouraged.

Multiple global HW partners approved by SAP . This leads to customer advantage as it leverages automatic market driven  competitiveness and development between HW partners

Who are the top 2 HW leaders

Oracle, Oracle and Oracle

HP is currently reported to be in the leadership position for HANA appliance installations followed by IBM

Strategic Impact

Does not align with SAP Suites strategy

Aligns to the HANA One-database strategy where customers get actual gains of HANA. Zero latency between OLTP and OLAP

How does it transform

Disc to server and back.

Data is passed from server to server till results are compiled. There is a need to break up responses


All the calculations are done in memory in the db itself

No need to break up responses

How does it perform

Slower than Teradata

Oracle Benchmark: 10 bill records <10 o:p="" secs="">

Fastest on the planet

My experience 45 billion records < 1sec

SAP Benchmark- 240 billion records < 1sec

Maturity of the technology

Is equivalent to BW Accelerator

Undisputed global leader

How does it work

1.  Database server to

2.  Compute server to

3.  Storage cells to Exadata cells

All in-memory

Data Loading

1.        ETL files in DBFS

2.        Oracle golden Gate

3.        Streams

4.        3rd Party

1.        Data integration tools

2.        SLT (Real-time from ECC

3.        BODS for batch loads

4.        DXC direct BW


Every system in the landscape is subject to a license

1.        DEV, Test, QA, and PROD

2.        HA, RD and Training

3.        Other, i.e. Projects landscape

Only PROD is licensed

All other systems and Free (not licensed)

What you get

Traditional DB accelerator

1.        Traditional ‘Wait & Debate’ engine

2.        Database

3.        Transaction Engine

Everything else supposedly are separate tools and licenses

Breakthrough HANA Platoform

1.        Real-time ‘Business NOW’ engine

2.        Database

3.        Transaction Engine

4.        Dimensional Engine

5.        Analytics Engine

6.        Predictive Engine

7.        Text Engine

Modeling methodology

Traditional- 10+ steps

ODS> ETL> DW> caching> Indexing> Aggregates> Query> Calculation Engine> Query Transforms> RESULTS

Breakthrough 3-steps

DSO> In-Memory Analytics> RESULTS

Impact of modeling method

·         Latency between operational and decision data

·         Infrastructure intensive

·         Expensive by system resources

·         Creates duplicates and is slow

Up to 1,000 times faster

No indexing or aggregates

Calculation / Transformation methodology


·         Most at OLAP (Query) layer – extremely inefficient

·         Rest in DW layer                    - Very Inefficient

Breakthrough calculation

·         Direct database transformations

·         Real-time db calculations

·         No mid-tier bottlenecks

Data-Center Impacts

·         Typically RAC (extra Costs)

·         Storage uses ASM failure groups

·         Box is a single point of failure

·         DR is a full license

Uses MPP automatic failover

Disaster recovery

·         License: full license for HCC, and even remote mirroring

·         2nd Exadata server/ non-Exadata server

·         Decompression to standard Oracle server possible with considerable performance loss

License: No extra Charge

Provided by HANA vendors

For SAP customers


All new functionalities only being developed on HANA

1.        BW seamless upgrade & migration in one step

2.        Over 900 views for ECC

3.        New RDS for CRM

4.        New RDS for SCM

5.        Industry specific solutions