The Trap of Behavioral Data
A client's AI had six months of browsing data proving a customer segment loved premium kitchen gear. Clicks, dwell time, repeat visits, the works. The algorithm was confident. We were confident. Then someone actually called a few of them. They were browsing for gifts. They cooked maybe twice a month. The data was right. We were completely wrong. Welcome to behavioral data, where the numbers are honest and the analysts are the problem.
A client asked me to look at their AI-powered recommendation engine last year. It was impressive, genuinely. Trained on millions of customer interactions, and it had flagged a growing segment of users who kept browsing high-end kitchen products. Expensive knives, countertop mixers, artisan cookware. The behavioral data was beautiful. Long dwell times, repeat visits, clicks deep into product specs. The AI had tagged them as "premium culinary enthusiasts" and was serving them more of the same.
The conversion rate was terrible and nobody could figure out why.
Then someone on the team did something radical. They picked up the phone and called a dozen of these people.
Turns out most of them were shopping for gifts. Wedding registries, birthdays, holidays. They browsed carefully because they were spending real money on something they didn't personally understand. Half of them barely cooked. One guy told us he'd eaten takeout four nights that week.
The behavior was real. The AI's conclusion about what it meant was completely wrong.
The data told us what was happening. We filled in the why ourselves, or more accurately, we let the algorithm fill it in and didn't bother to check, because the data felt so clean and certain that checking seemed almost ungrateful. This is the professional equivalent of confidently giving someone directions to a place you've never actually been.
I've been wrong about behavioral data in ways that cost real money. Which is, honestly, the only reason I feel entitled to write this.
Recently I wrote why reported numbers demand triangularization from the ground up. Bad questions, wrong baselines, findings distorted on the way to the audience. Behavioral data's different. It's often collected beautifully. The click happened. The purchase was real. Nobody lied. The problem comes later, in what we decide it means. And that's a much sneakier problem because the data feels so trustworthy that questioning it feels almost ungrateful.
The genuine advantage of behavioral data is that it bypasses the lies people tell. Not malicious ones, mostly. Just the gap between what someone claims to value and what they actually choose when money's on the line. I call this the salad problem. Everyone orders the salad in theory. Nobody orders the salad. When a client shows me a customer satisfaction score of 87 and a renewal rate of 54, I know exactly which number to believe, and it isn't the one that made everyone feel good at the all-hands meeting. I've argued for behavioral signals over self-reported ones for most of my career and I'm not changing that position.
But here are four ways behavioral data misleads us, none of which show up in the data itself.
The first is that behavior without context is just action. Someone searches for information about a medical condition. The platform concludes they want more medical content and keeps sending it. But maybe they got a diagnosis, wanted to understand it once, and never wanted to think about it again. The behavior was recorded correctly. The conclusion was wrong.
Behavioral data tells you what happened. It can't tell you why. And in most decisions that matter, the why's the whole question. Platforms that don't understand this are the reason your search history makes you look like a completely different person than you actually are.
The second is that big behavioral datasets are great for understanding what populations do and surprisingly poor for predicting what any one person will do next. The volume creates an illusion of precision. A recommendation engine trained on fifty million people isn't predicting you. It's predicting someone who looks like you from a distance, squinting.
That gap matters a lot when the decision involves a real person's insurance premium, job application, or loan approval, and it almost never gets acknowledged by the people making those decisions.
The third is that behavior reflects the available options more than it reflects genuine preference. People do what the environment makes easy. A website designed to steer you toward one choice will produce data showing that choice winning overwhelmingly. This is not evidence that it's preferred. It's evidence the design worked. There's an entire industry built on this. They call it UX. Sometimes they mean it kindly.
The fourth is the one that troubles me most and honestly keeps me up at night, which at my age is saying something. When you use past behavior to decide what to show someone next, you change their future behavior. You're no longer measuring a person. You're measuring a person as shaped by your previous decisions about what to put in front of them.
Every major platform operates this way, and AI's made it faster, cheaper, and more invisible than it's ever been. Voter classification. Insurance pricing. Job listings. The version of the news that reaches you this morning. All of it looks objective because it's based on actions rather than opinions. The data looks clean at every step. The circularity almost never comes up in the presentation, because if it did, someone might ask an inconvenient question.
All four of these traps share one thing. The data looks fine. The problem is invisible until someone asks a question the data can't answer.
With behavioral data the questions aren't about collection. The collection is usually fine. They're about interpretation. What else do you know that the behavior alone doesn't show? What were the options available when the behavior occurred? Is there anything qualitative, something someone actually said in their own words, that complicates what the behavior appears to mean?
They're not hard questions. They're just inconvenient ones because clean data creates confidence and confidence discourages doubt. The store managers knew exactly why that product was selling. It took one conversation to find out. We almost didn't have it. I'm genuinely embarrassed by how close we came to publishing a very confident, very wrong report. We would have charged full price for it too.
Behavioral data is a genuine gift when you use it honestly and carefully. For forty years it's outperformed every other signal I've worked with at predicting what people will actually do next. I'm not walking that back.
But there's a real difference between data that feels trustworthy and data you've actually earned the right to trust. The most expensive mistakes I've seen in this business came from people who knew that distinction and forgot it the moment the data looked clean.
Ask what shaped the behavior before you decide what it means.
The Great Zandini Sees:
Behavioral data tells you what people did. It has no idea why. Neither do most of the people charging you to analyze it.
