As models become more capable and reliable, it’s more important than ever to design tools and context thoughtfully. When models had short context windows and low agency, basic document retrieval into the model’s context window made a dramatic difference; but as frontier models boast millions of tokens of context and run for minutes and hours through tool calls and cycles of compaction, there is an ever growing list of concerns all vying for our models’ scarce attention. Extending context windows is an expensive and incomplete workaround.
In this talk, we will share some of the principles and techniques we found useful in navigating these problems ourselves as we worked on the goal of improving our assistant, Puck, from a simple retrieval-based chatbot to a deeply knowledgeable general assistant. In particular, we will touch on two prevailing challenges. First, we’ll share how we’ve sought to raise the signal-to-noise ratio in Puck’s context by thinking of context engineering itself as a search problem, involving every step of the pipeline from indexing to subagents. Second, we’ll share how Puck approaches blending faithful, up-to-date structured data queries with the richness, breadth, incompleteness, and frequent conflicts latent in unstructured data in production. We’ll walk through a few concrete tactics in detail within both of these pillars, demonstrating that creative and useful approaches often come when we stop thinking of databases, search indexes, tools, and subagents as separate components, but as different solutions to the same underlying, age-old problem: searching for signal in a confusing and noisy world.
