Data and Instinct – Inspiration’s Yin and Yang


With all the momentum around the easy access to customer data across the digital landscape and the variety of tools that let marketers make quick sense of that data, some might think human intuition is becoming obsolete.

We’re here to say human instinct is alive and well in the world of IoT and Big Data. Marketing Land’s article Human Intuition Vs. Marketing Data: Forging A New Alliance states,


“The human brain remains one of the most flexible and potent sources of computing on the planet…And the most effective marketing analytics solutions are those that recognize that human intuition – and perhaps most importantly, human curiosity – plays a critical role in moving from data to insight to decision to action.”

Netflix Decides with Data and Human Instinct

Netflix gets a lot of attention for its use of data to drive its powerful recommendation engine. It’s often cited as a key to the company’s continued global success, helping people discover lesser-known content that will delight them.

But along with streaming a large catalogue of movies and TV shows, Netflix has been gradually shifting its business model to emphasize more of its own original TV shows and movies. These Netflix originals (think House of Cards and Orange Is The New Black) have become cultural phenomena in their own right.

Netflix has said it uses data analytics in researching the development of new programming. But at the recent DLD Conference in Munich, Germany, CEO Reed Hastings noted that the company still leans just as hard on “gut instinct” in the realm of selecting its original content.

“We start with the data,” Hastings said. “But the final call is always gut. It’s informed intuition.”

It’s a good reminder for just about anyone working in the realm of Big Data. Yes, data can drive exciting new insights and learning. It’s a revolution. But that doesn’t mean it’s time to forget the human side of the equation.

And it’s notable that this argument is made by Hastings. Not only is he one of Silicon Valley’s most successful CEOs, he’s also a former artificial intelligence engineer. So he knows a thing or two about Big Data and how it works under the hood.

In the case of Netflix, that means the company bets big on data. But it also means, according to Hastings, that the people in position to make decisions have to be able to interpret that data, know its limits and make those hard calls on “gut instinct.”

Data won’t provide all the answers, and at some point the human in charge has to decide.

Of course, data picks right up again after those decisions have been made. For Netflix, that has provided interesting and unexpected insights into programming and viewing behavior.

For example, the company is funding development of an original French-language series, called Marseille, because data shows that French TV is popular around the world. That might come as a surprise to people in France, where the reputation for the quality of TV shows remains far below the country’s films.

Netflix also discovered through its data that many shows develop audiences in ways that might seem unpredictable. For one new show, Narcos, the plot is based in South America, created by a French studio and a massive hit in Germany of all places.

Discovering what drives those connections, Hastings said, will be a key tool in Netflix’s long-range goal of revolutionizing the way we make, watch and discover content.

Marketing Land’s article hits the nail on the head: “Creative interpretation of the data is always going to be essential.”

The inspiration companies find to solve their biggest challenges hinges on people knowing what questions to ask and what priorities to set. Data is essential in helping answer those questions and achieving those goals. When human instinct and the right data work in concert, the yin and yang of inspiration comes to life in the best ways possible.

3 Reasons to Be Thankful for Data

give thanks - Thanksgiving concept


What better time to think about all that we’re thankful for than Thanksgiving? For businesses, there has never been a better time to be grateful for data and all it can do to help deliver new opportunities. Like a moist, juicy roast turkey, data is the one thing everyone seems to be eating up. Let’s look at three reasons why data is so delectable. We promise you won’t feel sleepy after consuming this list.


Bountiful Amounts of Data

The Pilgrims arrived in the New World during a particularly harsh winter, making it very difficult for them to find food and shelter. Of course we know they persevered despite the shortage of resources after they learned to harvest crops that produced an abundance of food for them to not only survive, but thrive.  Fast-forward to today, and we gorge ourselves on turkey and various complex carbohydrates in celebration of the early settlers’ efforts.

Like the sustenance which was once so hard to come by, data too has gone from scarce to abundant, and there’s plenty of it to feast on. Just as the Pilgrims struggled to live off the land, data scientists once had to work especially hard to cultivate meaningful bits of data. When data was scarce, they had to extract, track and survey to get at it. The objective was always to find out how and where to get more of it. Today, data is everywhere and the new goal is understanding how to make sense of it.

Technology has been one of the biggest catalysts of the data explosion. With the digitization of business, the amount of information we are collecting is growing so large that there are debates on just how big it is – it probably has grown even more since you read that sentence . Both structured and unstructured in nature, this wealth of information has made it possible to produce insights and achieve outcomes that were previously inconceivable. Businesses can better identify trends and risk, organizations can tackle health and wellness issues, and governments can solve economic and social challenges.

While we should certainly be grateful for the abundance of data, we must be careful how we use the information. It is important not to overindulge or horde it. Instead we must recognize the type of data that will sustain us and avoid the empty calories that may lead us astray. Just like the Pilgrims planted the right seeds that would bring them sustenance, we must choose the kernels of data that will drive meaningful value and insights.


Data Freedom

The Pilgrims came to America from England in search of religious freedom.  They yearned for an open society characterized by a flexible stricture, freedom of belief and dissemination of information. We are witnessing a similar evolution in the way data is accessed and shared. The concept of data sharing is officially defined as making a certain piece of data free to use, reuse and redistribute –subject only, at most, to the requirement to attribute and/or share-alike. In other words, we’re at a point in history where some information can be freely used to solve key problems or make progress towards specific goals.

There are many examples of data that can be openly shared. However, it’s not just about the numbers, but the insights that come along with it that pose the most benefits when freely distributed. This concept offers benefits across both the private and public sector. Businesses can gain a new level of transparency into new opportunities for services/goods, make better decisions based on more accurate information, and reduce costs for data conversion. But perhaps the biggest advantage of an open data ecosystem is for individual citizens. That’s because the sharing of information between governments can help everything from increase economic activity, address national disasters in a swifter manner, and even reduce health issues.

There are several types of data that can be shared among governmental functions. There is the sharing of data among governmental agencies within a single country. Second is the sharing of data across borders between International governments. And lastly, there is the sharing of data between businesses and government; this refers to voluntary sharing of data, beyond the legal reporting obligations of governments.

So what exactly should governments be sharing? Observations are crucial – such as the National Weather Service issuing a Hurricane watch. It’s about sharing conclusions that can help different agencies better prepare for that projected weather event. Ultimately, this is the value of open data: multiple organizations, with mutual interests, sharing their independent observations in a manner that lets each organization draw more accurate, informed and insightful conclusions.

Unfortunately, in many cases the decisions made by government officials are not always based on all of the information that might be pertinent to the situation. The same goes for businesses, which are somewhat reluctant to let go of their first-party data. With the freedom and ease to share information at our hands, we have the opportunity to achieve maximum economic and social benefits.

At the end of the day, data is nothing without analytics to help make sense of it.  We should always be cognizant about ways in which specific pieces shared data are used to address specific questions and help establish new ideas. The Pilgrims likely did not use all of their food sources to cook a single meal.  We shouldn’t use all the data available to solve a problem just because it’s in front of our face.


A Brave New World of Data

As hard as I tried, I could not come up with a very clever parallel between the Pilgrims and the Internet of Things (IoT). But, this new buzz word represents such a major data opportunity to be thankful for, and, well, the Pilgrims had, umm, things, so here we go.

The IoT refers to the growing network of interconnected objects – devices and machines which are not connected but aware of, and discoverable to, other devices and machines around them. It’s mobile, virtual and instantaneous. Data can be gathered and shared from anything like a car to a refrigerator, which means we’ll be witnessing a huge increase in the amount of data being generated – huge streams of data that will enable new types of insight. We now have an opportunity to organize this information in compelling ways to reach new conclusions.

The IoT has the opportunity to fundamentally change industries. From the automotive industry, where new data signals from cars may help improve safety conditions, to the supply chain, where the real-time passing of information can avoid disruptions in manufacturing. Organizations will quickly realize transformational change of their business models given the rate at which digital technologies are connected and constantly evolving.

As beneficial as the IoT will be, it will not flourish without the right data management and analytics capabilities. Organizations will need to invest time and resources in order to mine valuable insights from the data generated by the interactions that occur between machines.


In Summary

These are just three examples of how data is changing the face of business and frankly, society. There are certainly countless other reasons to be thankful for data, depending on what your business goals are and what you want to achieve. As I’ve noted, within each of these instances, while there is much to be thankful for, it is vital we be cautious and smart when taking advantage of new data opportunities. Just like the Pilgrims, we are using this new frontier to create a new world of endless possibilities.

Thinking Big in Michigan: One University’s Commitment to the Future of Data

COd3YxaWUAA1jo_This past Saturday at the University of Michigan, the hometown Wolverines beat Rutgers 49-16 in front of 109,879 fans at Michigan Stadium, a cathedral of college football.

They call the stadium the Big House, but for Michigan, this fall has also been about Big Data.

The well-respected higher-ed institution recently announced that it would spend $100 million over five years on its new Data Science Initiative. The ambition here is immense. The DSI will tie together three existing institutes that already have 40 faculty members. Over the next four years, the program will hire another 35 faculty. It will also provide new programs to train graduate and undergraduate students in fields associated with data science, a much-needed boost to provide talent and opportunity in an industry that is booming.

So why is UofM placing such a large bet? The answer is at once simple and illustrative: The university believes Big Data is the key to solving the key societal challenges and business opportunities of our time.

 “Big data can provide dramatic insights into the nature of disease, climate change, social behavior, business and economics, engineering, and the basic biological and physical sciences,” said President Mark Schlissel in a statement announcing the program. “U-M is in a unique position to leverage this investment in data science for the good of society.” 

In announcing the program, Michigan officials offered several examples of programs already underway that they hope will serve as models and inspiration for the kind of work that can be leveraged by the DSI:

  • University of Michigan’s Mobility Transformation Center: In an Internet of Things-focused application, researchers are gathering data from 3,000 connected cars and trucks driving the streets of its hometown, Ann Arbor. The DSI will help the center further its transportation studies by assisting with the collection, storage and analysis of the data so it can be dramatically expanded in scope.
  • Medicine and public health: To accelerate the transition of discoveries made by basic research to practical patient care, UofM researchers will use Big Data to develop new models for biomedical and health research. The hope is that a data-driven approach on a large scale will hasten the development of more precise diagnosis and personalized medicine.
  • Teaching and learning: The DSI will turn the microscope on its own university to explore ways Big Data can provide insight into how effective teaching and learning occur. The idea is to find ways to provide more personalized instruction to students on a giant scale.

Health care. Finance. Transportation. Environment. Education. Pharmaceuticals. The scope is so broad that the university has said that almost every academic department will be involved with Big Data to some degree.

It will be thrilling to watch this project come together. If Big Data really is going to change the world, the University of Michigan aims to claim its spot on the front line of the revolution.

Get Ready for Disruption

Ines Hegedus-GarciaOver the past decade, we’ve seen a wave of disruption hit industry after industry. It started with publishing: Books, newspapers and magazines got hammered by new digital publishers. Established media outlets got scooped by bloggers, and ad spending was suddenly split between online banner ads and traditional print.

The disruption was even more profound for the music industry. Illegal file-sharing, enabled by websites like Napster, generated copyright battles and outcry from artists like Metallica. The industry got turned upside down as more and more consumers turned from physical CDs to digital downloads and streaming – new business models that needed to be tested and refined before announcing success.

Where was your business during the digital content upheaval? Chances are you were watching safely from the sidelines. Well, your turn may be coming – along with a lot of business leaders – to see your industry engulfed in chaos.

Welcome to the Internet Era

News, stories, music and images were in the first wave of disruption because their content is easy to digitize and distribute. The next wave will be a melding of physical and digital processes. Think Airbnb and Uber. Both created logistics layers that enable private parties to exchange cash for services in real time. The Internet of Things will allow any industry with physical units to track, optimize and reallocate them at algorithm speeds. Gartner predicts that by 2017 digital businesses will be in the thick of it, grappling with fresh capabilities to digitize physical processes in way that lets businesses use embedded data for competitive advantage.

Why so soon? Sensor technology is getting more powerful and less expensive. Memory is cheap, and more intelligent chips have the potential to reshape analog processes and products in the real world.

It’s Not About Technology

Most established businesses are not wired internally to deal with sweeping changes that could torpedo their business models. So the first step to prepare for disruption is not really about the technology. It’s more important to evaluate and update the culture and the internal management of a business.

Gartner analyst Carol Rozwell talks about “five digital freedoms” – new rules that can guide organizations on how to build a culture that will be resilient in the face of change. In her post, Storm-Proofing Your Organization, Rozwell talks about how the freedom to try and fail is antithetical to traditional management practices – and can bring much-needed flexibility and resiliency into corporate cultures.

The five digital freedoms are:

  • Employees must be free to connect with other employees
  • They must be free to contribute their ideas
  • They must be free to create and experiment
  • They must be free to choose their work environment
  • They must be free to challenge leadership

This is a massive reimagining of how things usually go in the business world. And the challenge should not be underestimated. At the core, what Rozwell is saying is that traditional top-down management may not work in this new era. Leaders must learn to let go of the decision-making process, to listen and to include more voices. That can be scary.

But to get started, Gartner offers a few tips. Rather than trying to do everything at once, try a few basics to get started.

  • Examine and identify the types of disruption coming at your business. Ask: Which of the five freedoms might be best to help adapt to this change?
  • Examine employee performance metrics. Ask: Are you judging employees by criteria that encourage them to cling to old habits rather than taking risks and trying new things?
  • Examine training programs. Ask: If we implement management and cultural changes, will employees understand and believe in them enough to really embrace them and use their new power effectively?

Getting ready means granting people a measure of autonomy and control. It means preparing for new technology challenges and new ways of accessing the human ingenuity within an organization’s brain trust, no matter where people sit on the org chart. Like earthquake-proof buildings that flex with seismic unrest, companies must be rearchitected with flexibility and resiliency in mind. Because when it hits, disruption will ripple across industries, breaking structures too brittle to survive.

Image credit: Ines Hegedus-Garcia

The Great Build-Out: Infrastructure Needed for the Internet of Things

Dennis van Zuijlekom_green

I love the first line of this Gigaom article: “Infrastructure is something that people are used to not thinking about.” But ignoring infrastructure is not an option when it comes to supporting the Internet of Things (IoT).

Consider the numbers. Currently, humans generate about 2.8 zettabytes of data a year. That’s 2.8 trillion gigabytes of emails, tweets, search queries, Facebook posts and online purchases. That number is expected to double every two or three years.

Add to that devices, sensors and other sources of machine data. Pundits project there will be more devices added to the Internet than people in the coming decade. By 2020, we’re expecting to have anywhere from 26 billion devices connected to the Internet, on up to 50 billion or even 200 billion.

What’s it going to take, to build an infrastructure robust enough to handle that volume of data?

That’s the main topic at the Gigaom Structure conference  coming to San Francisco on June 18. Really smart folks are looking at how to build a back end to support hundreds of billions of sensors and devices – along with several billion people.

Some industry experts believe that building out an infrastructure for IoT will be much the same as building out a typical IT infrastructure: servers, storage and network connections hooking into the cloud, delivering scalability and availability. But what, some ask, if they could make it better than that? What if we could have a memory of each device, its history and data, so a device failure doesn’t result in a loss of data?

There’s opportunity here, says Gigaom research analyst Dave Ohara. He writesWe need infrastructure that provides a sort of institutional memory of what you’ve done with your devices. Where do you think the money is in the infrastructure of IoT? A low-cost infrastructure that quickly gets commoditized or a value added service for the Infrastructure of IoT users will stick with?”

He’s talking about making an infrastructure that’s stronger, more scalable, and smarter than what we have today. And maybe, just maybe, it’ll usher us into a new age of computing where data is always available, device-independent and easily integrated between platforms. It’s a goal worth striving for.

Image credit: Dennis van Zuijlekom


The Case for Better BI

Internet of Things image with spindles

There’s been a lot of buzz about the Internet of Things (IoT) the last few years – how it’s going to wire up the world, from coffee makers to cargo ships, and provide visibility, control and insight to consumers and businesses alike. The question is: How will people get and use all this data?

Makers of Business Intelligence (BI) software are already under pressure to make their tools more intuitive, so business people can use them without help from their IT departments. What about the rest of us? Who’s going to build the interfaces that let me start the coffee maker from bed, warm up the car, monitor my power usage and check my daughter’s Internet chat history? Google is on board to help, but it’s going to take a whole new business ecosystem to manage, integrate and serve the data, much less make it accessible and actionable.

We need heavy-duty technologies to help us organize and derive meaning from raw data, starting with real-time data platforms and identity resolution capabilities. And we need more people who can analyze the data. Today, only .5% of the world’s data goes through any analysis at all, according to IDC’s Digital Universe study.

Why does it matter so much, that we all have access to the tools we need to get, analyze and mine our data? Because the value of the IoT is in the data, says this Wired article. Nearly $2 trillion dollars in economic value, if Gartner’s estimate is correct. That’s a lot of potential revenue for the companies that figure out how to manage and monetize the data tsunami that’s headed our way.

The next generation of BI will be user-friendly, mobile, flexible and self-service, meeting the needs of the masses within their respective lines of business. With these tools in place, there’s no telling what insights savvy business people – and maybe regular Joes like you and me – will be able to do to change the world we live in.