I built a Lenny chatbot using GPT-3. Here’s how to build your own.

Lenny Rachitsky and Dan Shipper walk through how a Lenny-specific chatbot was built on GPT-3 using his newsletter archive. The piece explains the difference between GPT-3 and ChatGPT, why raw language models tend to hallucinate and lack current or paywalled data, and how stuffing relevant context into prompts helps. It then shows a practical pipeline: indexing article chunks as embeddings, retrieving the most relevant chunks for a question, and passing them to GPT-3. The post matters to product people because it demonstrates that content-based chatbots can be built with little code and points to new possibilities for creators, personal knowledge management, and enterprise knowledge.

01Key takeaways

• Use GPT-3 rather than ChatGPT when you need direct programmatic control over prompts and outputs.
• Expect hallucinations and confident wrong answers; ground responses by supplying the source text in the prompt.
• Split your content into chunks and index them as embeddings so the most relevant passages can be retrieved per question.
• Keep prompts within token limits by selecting only the most relevant context rather than the whole archive.
• Repackaging evergreen content as a chatbot can reduce repetitive questions and open new monetization routes for creators.
• Hands-on experimentation is the best way to prepare for AI-driven product work, even without a coding background.

02Key sections

Why build a content chatbot

Newsletters are one-directional, so the authors wanted readers to be able to ask questions of the archive. Dan Shipper built a working bot in a short time to show it was feasible.

GPT-3 versus ChatGPT

GPT-3 is a general-purpose completion model that can be accessed directly, while ChatGPT is a conversation-tuned version only reachable through its own app. That makes GPT-3 the practical base for a custom bot.

Prompting, hallucination and missing data

GPT-3 predicts likely word sequences, so it can answer confidently and wrongly, and its training data has a cutoff and excludes paywalled content. Benchmarks and specific facts from the archive are often missed without help.

Stuffing context and embedding the archive

Relevant passages can be placed into the prompt like an open-book note card, which improves accuracy. Because prompts have a token limit, the archive is split into chunks, converted to embeddings, and indexed for similarity search.

Implications for creators, notes and enterprises

Any evergreen content can be repackaged as a chatbot, personal notes can be queried instead of filed, and companies can use similar tools as an automated knowledge librarian.

03From the post

“1. Five steps to starting your product-led growth motion, part 2 2. How Coda builds product 3. Discussion: What’s one change you’ve made to your product development process in the past year that’s had the most impact on your team’s success? Subscribe to get access to these posts, and every post. Last month, Dan Shipper (co-founder and CEO of Every) launched a chatbot trained on the Huberman Lab podcast, and I was blown away. I pinged Dan to see what it would take to build something like this for my newsletter content (as I was super-curious about the process, plus readers have been asking for it over the past couple of months, including just yesterday!), and by the next morning, Dan had a functioning chatbot 🤯 Not only did Dan offer to build and launch the chatbot, he suggested that we use this opportunity to help people learn how to build their own. And that’s exactly what you’ll find below. Dan walks us through the basics of AI and GPT-3, how to set up an environment to play with the…”

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.