Soft vs. Hard Data

by | Oct 22, 2018 | Data Analysis

A line from a Harvard Business Review article by Thomas Redman recently caught my eye:

Redman’s article resonated with me because it focuses on something that I focus on with my clients: context.

In other words, how did the data come to be, and what is the larger story behind the data? As Redman writes:

[Data scientists] recognize that there are nuances and quality issues in the data that they can’t understand while sitting at their desks. They recognize that the world is filled with “soft data,” relevant sights, sounds, smells, tastes, and textures that are yet to be digitized — and hence are unavailable to those working at their computers. (Think of things like the electricity in the air at a political rally and the fear in the eyes of an executive faced with an unexpected threat.) They know they must understand the larger context, the real problems and opportunities, how decision makers decide, and how their predictions will be used.

It’s this concept of “soft data” that piqued my interest.

Behind the Intent

The internet as we know it contains information on everything that we ever wanted to know. That information just sits there until we encounter a need to learn more about it, such as when we want to make a purchase or solve a specific problem. As it relates to the buying process, Google calls this the zero moment of truth, which it defines as “the moment in the buying process when the consumer researches a product prior to purchase.”

As the folks at Yoast see it:

Search intent has to do with the reason why people conduct a specific search.

When you work with data in a business context, being deeply curious about the data also means understanding the persuasion principles behind the content (the hard data) in order to understand the soft data. In other words, the number of clicks in a given marketing campaign are just meaningless numbers — unless you understand what is going on in the consumer’s mind.

Soft Drives Hard

Most people don’t know that all that behavior can be (ethically) controlled to achieve a certain outcome. (For more on the subject of customer behavior, check out Hooked: How to Build Habit-Forming Products by Nir Eyal, Predictably Irrational: The Hidden Forces That Shape Our Decisions by Dan Ariely, and Influence: The Psychology of Persuasion by Robert Cialdini.)

And that’s not a new development. Long before Don Draper pitched his first campaign, copywriters and marketers were using all sorts of advertising tricks and strategies based on soft data to drive sales (hard data). So if you’re working with any type of marketing-related data, the following five books will help you uncover the soft data hiding behind all of the clicks, conversion rates, and — most importantly — credit card transactions that are taking place online.

The Week’s Top Five

Tested Advertising Methods by John Caples1. Tested Advertising Methods by John Caples

In digital marketing, everyone talks about the A/B, or split, test on a specific ad, headline, or even price point. Caples focused his career on making advertising successful through scientific testing and, if you take anything away from this book, it should be his three-step method to testing: 1. Accept nothing as true until it is tested objectively. 2. Build upon everything you learn from reading to build a stronger system. 3. Treat every ad as an ongoing test of what has been learned before.

The Robert Collier Letter Book by Robert Collier2. The Robert Collier Letter Book by Robert Collier

Collier was an expert at selling ice to eskimos. In countless examples in this book, it’s clear that he knew that conversions occur when the copy is able to affect someone emotionally. This book makes the list because of the countless examples of how Collier is able to use words to create vivid pictures in your mind to create desire behind a given product.


My Life in Advertising and Scientific Advertising by Claude Hopkins3. My Life in Advertising and Scientific Advertising by Claude Hopkins

Hopkins (1866–1932) was obsessed with data even before data was cool, yet everything his book talks about is relevant today. My Life in Advertising makes this list for its numerous case studies and insights, such as how Hopkins conducted research to get in the heads of customers in order to position Palmolive as the country’s top shaving cream or how he moved Schlitz beer from No. 5 to No. 1 in a few months simply by telling a story about how the beer is made. 

Ogilvy on Advertising by David Ogilvy4. Ogilvy on Advertising by David Ogilvy

While there are many lessons we can learn from the person once called the “most sought-after wizard in today’s advertising industry,” the one that stands out the most, within the context of soft data, is this: “You don’t start a tinker’s chance of producing successful advertising unless you start by doing your homework. In other words, you really need to invest the time and study the product or topic that you’re working with, regardless if it’s a Rolls-Royce or your customer base, in order to be effective.”

Reality in Advertising by Rosser Reeves5. Reality in Advertising by Rosser Reeves

It’s widely believed that Mad Men’s Don Draper was based on Reeves, whose book made the list because of the numerous revenue-shattering case studies he worked on during his time at the prestigious Ted Bates advertising firm. Also notable is a discussion of the origin of the unique selling proposition, a concept that originated at Ted Bates and is widely misunderstood in the marketing world.

Like What You See?

If this issue of Missing Data has brought you any value, it would mean the world to me if you shared it with one friend using the buttons below. Thanks!