Video killed the volleyball star
Over the last year, I have recorded approximately eleventy zillion hours of my daughter playing volleyball. Do I know anything about volleyball? Not really! But my kid fell in love with the sport last year and has limitless enthusiasm for analyzing her game to see how she can improve. Cue the iPhone.
You know who else has limitless enthusiasm for analyzing her game? ChatGPT.
Unlike, say, me at age 15, my daughter loves getting feedback. She recently uploaded six hours of her footage to ChatGPT. Then she compared ChatGPT's notes to the skills assessment she got from coaches after the clinic.
What she got back was shockingly good. Skill for skill, ChatGPT saw exactly what the coaches saw. But where she got just a paragraph of high-level feedback from the coaches, she received pages of actionable, detailed analysis from ChatGPT.
This got me thinking: What would AI say about me on video?
You may not play volleyball, but you're still a star
You may not play volleyball at work, but you're still on camera plenty. You're having meetings and giving presentations and making your little TikToks. There's tons of footage of you working on video.
For the sake of experimentation, I wanted to see what AI said about my own public speaking clips. I included clips from three different stages of my career: 1) when I worked at Microsoft, 2) the early days of Textio, and 3) stuff from the last five years.
I asked these questions:
- Overall, how strong am I as a public speaker?
- What are my strengths and weaknesses as a speaker?
- How have my public speaking patterns evolved over time?
I was interested in whether the feedback would be insightful. I was also specifically curious about how AI would assess me differently in the different eras of my career.
Overall, AI is accurate, trending sycophantic
On the whole, the summary of my strengths and weaknesses is on point. The feedback is quite close to what I've heard from human critics.
As usual, though, AI is kind of a kiss-up; I'm a good speaker, but am I really in the top 1%? And gotta love the gender-trope reminder that I should smile more.
Overall, though, this is solid. It is directionally fair and the feedback is actionable.
Getting old is hazardous to your health
I wanted to see how the system characterized changes in my public speaking skills and behaviors over time. Look how many comments about my age show up in the era-by-era assessment! And we still get that reminder to smile more.
Not even 200 words, and we still manage to pack in:
- "highly dynamic and energetic, with a young professional's intensity"
- "less outwardly dynamic" as I get older
- "could have smiled more to increase warmth"
- "less physical energy" in the clips where I am oldest
- "high, youthful dynamism"
Don't get old, people. ChatGPT will call out your wrinkles faster than you can say ageism!
But what if I do need to smile more?
These patterns got me curious, and because I am a glutton for punishment, I did another mini-experiment. I selected the first public video clips I found from 25 men and 25 women, all well-respected, all speaking about future of work topics. I asked AI to rate each speaker on a 1-5 scale for clarity, energy, and warmth:
In every dimension, AI scores the women worse. The delta is biggest for warmth.
I've looked at a lot of AI data sets for bias, but sometimes, it isn't even subtle.
That wasn't masochistic enough for me, so I also asked for a binary classification of each speaker: Would this person be a better speaker if they smiled more?
Surprise! Just like many human audiences, AI says that 92% of the women should smile more (compared to only 16% of the men).
The bottom line: From volleyball to workplace public speaking, AI was able to analyze video input to provide high-quality, detailed feedback. This is honestly amazingly cool.
And yet, just as in every other work and life scenario since the dawn of time, women are going to be asked to smile more.
Kieran
From now until August 31, my Viral Data Stories 101 course is half-price for nerd processor subscribers. Learn to tell your own data stories clearly and effectively!
My latest data stories | Build like a founder | nerdprocessor.com