Data story fails
Over the last year, I have uncovered a lot of interesting data about how work works. For instance, who knew that a third of job candidates are straight-up ghosted after completing interview loops? (Everyone who has ever interviewed for a job, that's who.) I also discovered exactly how AI has absolutely destroyed my inbox, and that foundation models have statistical bias baked in. And lots more.
Sometimes the data turns out to be exactly what I expected. Yay? I guess it's nice to feel validated even when the data itself is depressing.
But what about all the other times? This week, I want to talk about three cases where my starting assumptions were totally wrong.
Fail #1: I was wrong about remote work
At different times during my leadership of Textio, we were both an entirely co-located company and a fully distributed one. While I've always believed in the unique magic of working in person, I saw how we were able to reach a wider array of talent after going remote. I had also read all the qualitative research claiming that, on average, remote environments have higher employee satisfaction and retention.
So when I set out to look at collaboration patterns in remote vs. in-person meetings, I thought that remote settings would shine in a couple of aspects. In particular, I expected that the greater diversity of remote teams would result in greater diversity of perspective in remote conversations.
Uh, nope!
Instead, I found that many more people participate in conversations that are in person rather than remote. Even further, I found that people are significantly more likely to disagree with each other in person rather than remotely.
Most strikingly, I found that the dropoff in participation rate in remote settings was especially striking for women. This is exactly the opposite of what I first hypothesized.
So yeah. I was wrong about remote.
Fail #2: I was wrong about who says "I told you so"
I have done a lot of research about how people talk to each other at work. I've found many times over how much more often men interrupt conversations than women, except for how women in exec roles are bigger interrupters than anyone else. So when I set out to discover who is most likely to say "I told you so" in workplace conversations, I expected men and senior women to be at the top of the list. Nope!
Instead, I found that men and women say "I told you so" at similar rates. They are both more likely to say "I told you so" to people of their same gender.
When men say "I told you so," 73% of the time, they're saying it to other men. Given that men and women are about evenly represented as participants in the meeting corpus, this is striking.
The women's data is even more lopsided. 87% of the women's "I told you so"s are said to other women.
In other words: I was wrong again, Batman!
Fail #3: I was wrong about small talk
Full disclosure: I am pretty good at making small talk, but it exhausts me. For many years, I assumed that I was alone in this, as evidenced by the overwhelming prevalence of networking happy hours and the many people attending them.
So when I set out to find out how small talk impacts meetings at work, I expected to find that a little chit-chat upfront helped the rest of the meeting go more smoothly. I figured team members would appreciate the feeling of connection before getting down to work.
You guessed it: Nope!
It turns out that the more time you spend on small talk, the less people say in the work portion of the conversation later on. In other words, small talk has a chilling effect on professional discussion rather than an inviting one. And taken in aggregate, the managers who spend the most time on small talk are rated the lowest overall by their teams.
So maybe it's not just me. Maybe no one loves those conference happy hours after all?
The bottom line: The heart of any successful data story is the hypothesis you start with. A good hypothesis gives you a starting lens on the data. A great hypothesis is one where the story will be interesting whether your initial assumptions are right or wrong.
It's fun to be right! But it's much more interesting when you build a great story from a hypothesis that turns out to be wrong.
Kieran
From now until August 31, my Viral Data Stories 101 course is half-price for nerd processor subscribers. Learn to tell your own data stories clearly and effectively!
My latest data stories | Build like a founder | nerdprocessor.com