Is IoT +BiG Data making traditional polling obsolete?

Big-Data and IoT are changing not only the definition of industries but proving that polling is now an obsolete methodology; It's Time for Customers and Marketers to Rethink ‘Digital’

Whatever side you stand on the controversy over polling results continues on a daily basis. You are likely to agree on three things.

1.     The first that all polls can be designed with specific question to derive very specific results

2.     The polls of the democrats are consistently  different from those of the republicans

3.     Both polls cannot be right

So after decades of polling and despite the confusion it creates in the minds of everyone involved with its designed creation, the polling itself and the results –polling has come a long, long way and I a primary job of some companies. However a recent book and what it reveals now proves that polling is a dying process that is rapidly being replaced by digital processes that are changing the very game of ‘Who we actually are?’ and ‘How we actually think?’

I presented this on the Nov 11th ‘HANA4IoT’ event at SAP Palo Alto and thought it prudent to share it with the community because of the large number of emails I received on these few slides.

For traditional polling and marketing companies this may be a big transition. After all digital analytics has been the domain of direct response and access, whereas with traditional polling one needed teams of pollsters in select geographic locations. There is no doubt that there will be no shortage of traditionalists who will find enough reasons to stay the course, while the digital marketing and analytics will continue to be a direct marketer’s dream come true.

However, it is critical to remember that in a poll we mostly get what the polling person wants to get and we never get the whole truth. If you stop a person on the street and ask randomly ask them ‘What sex and age group interests them the most in a dating site?’ There is a high probability that most respondents will not tell the truth.

Just in the same way when we poll business users in a post go live BI business users and the IT team a simple question, ‘Do the reports in your BI system meet your needs and expectations ?’ the results are always very different. However, both answers cannot be the empirical truth but just different points of view. Based on our twenty plus years of BI implementation and support experience where is the average you should generally expect to get. [1] 20-30% satisfaction scores from actual business users; [2] 70-90%satisfaction scores from IT resources; [3] under 30% from Gartner report based on feedback from over 2,500 CIO’s globally. If you feel you have the integrity to be truthful take this poll and tell us what you think. (Remember if you lie then you prove this blog correct, if you tell the truth then you may face the reality Gartner and I have been preaching about since 2010).

What we are finding is that the reality is not on who we say we are but in what we actually do, and even thought the industry is still latched on to the last click, the truth still will hold sway. So while today there is a lot of focus on that people say the future lies solely in what people do. So rather than the top funneling their results into the poll questions and the way they could be answered, the future is dealing with life on the EDGE where people actually do what they do, without being asked what they do nor why but simply the reality of what the reality is ‘As-Is’

Here is the crux of this blog from a book I have recently read called ’Dataclysm’ by Christian Rudder who is one of the president and co-founders of a site called OKCupid- a dating site like match.com, etc. As a Harvard graduate in math’s he delved into the data side of OKCupid and of humanity. The book reads like a thriller and the results are what prove that polling is quite redundant. Its reality that counts. To keep it simple will let the data do most of the differentiation between polls and reality. Caveat: [1] This is based on results from Dataclysm analysis only; [2] It’s my best explanation of what I saw and interpreted and may not represent the absolute truth.

Baseline: During the process of joining OkCupid you fill a form and some of the questions deal with what you are looking for. This helps internal algorithms align you with the right match. So a 35 year old male may state (poll) that they are interested in meeting a 25-30 year old female, or a 35 year old female may say they are interested in meeting a 35-40 year old male. The analysis below demonstrates the delta between who we state we are (poll results) and what we actually do (fact).

Fact 1: Poll- women: According to over 10 years of dating data the delta between what women say and do is rather narrow. As demonstrated in this graph 20 year old women reported (polls)  that they were looking for a 20-23 year old man, just as 35 year old women reported that they are looking for a 34 -36 year old man. While a 41-45 old women were mostly looking for 39-40 year old men. Fact- women: most of the women actually viewed, communicated and met men within their specified tolerances to a very large degree. Thus the delta between polls and fact for women is rather low with regards to dating data.

 

 Fact 2: Poll- Men: According to over 10 years of data the delta between what men say and

do is exceptionally large. As demonstrated in this graph  20 year old men reported (polls)  that they were looking for a 20-30 year old women, just as 35 year old men reported that they are looking for a 35 -42 year old women. While a 41-50 old men stated they were looking for 35-50 year old women. Fact- Men: Here is where it gets truly bizarre and funny all at the same time. Most of the men consistently actually viewed, communicated and tried to meet women within the 20 to 24 year range. Whether the man was 20 year old or 50 year old their primary preference is to meet and date a 20-24 year old woman.

So according to reality ,this represents the true gap between polling and the actual facts on the ground, the mind of men, on dating sites, looks something like this. It a very narrowband of 20-24 year old girls that they all first want to date. So next time someone tells you that you have a narrow mind they may be closer to reality than most of us would like to admit.

Whereas a poll is a form of admittance reality has a zero bias of id or ego, right or wrong, good or bad. It simply records what is.

 So in this interesting example we scratch the surface of what people (poll) enter when they have to answer a question and then what they actually go and do which is so strikingly different that it crashes the whole foundation of every wanting to depend on polling results. As my wife likes to say, at least where men are concerned.

That’s not to say that polling will no longer be a valid medium of opinions as it shall continue, But now mainly due to the ability of some powerful groups, influencers and marketers ability in proving the direct relationship between polls and their ability to predict changes in human behavior based on poll declarations at precise times. However, as we move forward more and more companies, politicians, political parties and most of all the media companies are already becoming interested in ‘Reality as it truly stands’ and not knee-jerk answers to planned questions that seek specific results.

My aim in this blog is to continue a dialog in further understanding the link between the human id and ego, between who we say we are and who we actually are, between who we say we will vote for and what we actually do. Of decreasing the delta with our big-data predictable analytics via using social data as against random polling. This challenge is a human psychology one and not a polling or digital one. We, as humans, treat ourselves as personal brand ambassadors and what we state in polls is more about self-branding rather than the truth about what and who we actually are.

At a recent conference with over 20,000 attendees we were able to track ‘in-real-time’ the movement of attendees by conference floor and booth, the movement of support staff by individual task, density of attendees and services and most important of all the tweets about different sessions and topics even as they were being broadcast-like keynote speaker sessions. Global branding is rapidly exploding into the digital space, and the digital arena is rapidly adopting to big-data and social network analytics. IoT devices connected by a seamless network devices providing streaming data turned into real-time decision focused analytics. 

All marketing, including digital marketing, is driven by a baseline of assumptions as to what customers like to do most frequently. With new IoT driven digital marketing applications we are now reaching a point where they are able to stop people in their tracks with the power of ‘who we actually are’ and the customers' emotional experience. In other words it is now time for customers to shift their thinking from traditional polling methods to more scientific methods and get access to far greater truth than polls can ever hope to provide.  

This is not going to be an easy shift to get accustomed to or to pull off. Matching true actions in real-time response means tracing the customer journey from a bottoms-up, inverse funnel all the way to actionable predictions, through to validation of final actions. This will probably be a massive data undertaking no matter how we look at it. However, unlike polling the process is iterative and self-correcting so we never have to send out new minions out into the field and start afresh in every new poll. It will build on itself and become more and more automated and accurate with time.

Capturing this new inverted digital polling and market emotion needs new approaches that go far beyond standard the capabilities of traditional algorithms that we typically worked with.  That’s because customers now expect end-to-end analytics that often take place over months and are accompanies with branding, mail campaigns, direct tele-campaigns, and short burst efforts as we get closer to the close of the campaigns. The more real-time data you have the better you can manage the campaign, the more accurate the analytics the better your decisions become. In essence, rather than focusing on the polls and their accuracy now decision makers can focus on long-term goals, aligned to short term facts and the rapid realignment of strategies to facts in true real-time by geo location. Its hardly the process that traditional polling and marketing companies are used to.

None of these skills are rocket science, nor are they simple to develop or acquire. Traditional polling and marketing companies will continue to trudge in processes that they are familiar with simply because it is their playing field, period. But then these are digital assets that have seen tremendous breakthroughs over the past few years and are today critical for success in our new digital world.

It is now up to customers to decide if they want to embrace the new big-data and digital world that is rapidly encompassing everything we do, or stick with direct polling and marketing as we have done for the past few decades. The price to pay is obsolescence. Because no matter what you or I think the world of branding and marketing has already gone digital. Polling is the next game in town as we have moved past sending humans to the streets and asking questions to strangers with the only responses being [1] the way my questions are designed to be answered, and [2] what the respondent thinks they want to tell us. It has nothing to do with who we actually are.