The prompt industrial complex
These days, AI can produce most common workplace artifacts. From emails to slide decks to financial models to working code, a workable first draft is often just one prompt away.
Ok, maybe not one prompt away. It usually takes a few tries to describe exactly what you want. If you care about quality, getting a credible artifact typically involves several prompt <> production cycles. AI refines its output based on how you evolve your prompt, and you refine your prompt based on how AI evolves its output.
Case in point: The other day I was working on a slide deck that included several graphs and visuals. I spent an off-and-on afternoon iterating to update my prompts. Three hours later, I was happy with only three of the 18 slides that AI had produced for my deck. I spent 30 minutes making the remaining 15 slides myself and called it good.
At what point is writing a prompt more work than just rolling up your sleeves and building the thing yourself? I had long since passed that point.
My prompts are getting longer
I have used AI for many purposes in the last month, and most of them are not artifact creation. Of the last 100 prompts I have made for personal projects, 22 are direct artifact creation, mostly code.
Of the last 100 prompts I have made at work, slightly more are direct artifact creation, but the majority are still for other purposes. I use AI more often for workflow creation than for artifact creation.
Some kinds of prompts are longer than others. In particular, my prompts for summarization and brainstorming are much shorter than my prompts for creating workflows or artifacts.
Why do some prompts take so much more typing?
The specification problem
With most open-ended uses of AI, I am just looking to create directionally interesting output rather than something precise, so my prompts are shorter. But when I'm creating an artifact, like code or a spreadsheet or a slide deck, I care a lot about the details. My prompts need to be specific enough for the output to be correct.
Here's the problem: If my prompts for artifact creation are too short, AI hallucinates a lot of dumb stuff and builds something unusable. But writing prompts specific enough to create high-quality artifacts can take just as long as building the artifact myself, if not even longer. My initial prompt length to build that 18-slide deck came in well over 1,000 words. If you add in all the words I added across the 15+ prompt iterations I made, I topped 3,000 words in total. At the end, I had three good slides. And I'm an experienced prompt writer!
Of course, not all artifacts are created equal. I am an excellent writer of words, but only a so-so writer of code. In addition, I generally share documents and slide decks directly with my audience, whereas with a piece of code I am much more likely to share its productized output rather than the code itself.
Taken together, this means that I care way more about the shape of AI-generated text than AI-generated code. If a software project is complex, I produce better software by writing long iterative prompts than I do by writing it the old-fashioned way. After all, I'm a PM from way back; I rule at specification.
Artifact rehydration is the new hotness
The other day my friend Sam Schillace shared some ideas he wanted me to read. But he didn't hand me a doc or a slide deck. Instead, he sent me a markdown file that I asked AI to rehydrate to the exact level of depth I wanted. I was able to use his markdown to generate both a detailed 6-page read and the 2-paragraph summary.
I found both formats useful for different reasons. Because I had already discussed the ideas with Sam several times, I was confident that the rehydrated docs represented his POV accurately. He put the most important elements into his markdown, and then individual readers could produce the artifact they preferred on demand.
In other words, his markdown specified context, my prompt specified format, and the two combined to build me customized final artifacts. He shared the markdown with several other people too, and they were able to create custom artifacts of their own using his same markdown foundation. Pretty cool.
Generalizing from this example: When prompts fail to generate the artifacts we want, it is usually because we have provided incomplete or poisoned context. Long prompts are an attempt to control this.
What do you think? Are your prompts getting longer?
Kieran
If you liked this story, why not subscribe to nerd processor and get the back issues? Also, why not learn to tell data stories of your own?
My latest data stories | Build like a founder | nerdprocessor.com