Can Your AI Analyst Keep a Secret? - Aaron Linsky, Bridgewater & Nikhil Benesch, turbopuffer

Highlights

• Bridgewater Associates relies on an unstructured database called The Vault, which contains approximately 100 million documents sourced from 135 different external providers alongside internal research and meeting recordings.

• Traditional vector search architectures collapse at scale when applied to fine-grained permission models, leading to multi-second or minute-long query latencies.

• TurboPuffer operates directly on cloud object storage like S3 and GCS, keeping data inert and cold until a query activates it, which cuts storage costs significantly.

• Combining pre-filtering and post-filtering into a native filtering strategy clusters vectors and checks permission boundaries concurrently to maintain high recall and speed.

• Across 450 million vectors derived from 100 million documents, TurboPuffer achieves a P90 search latency of 120 milliseconds.

Timeline

1. 00:00 - 02:07

   The Permission Challenge in AI Search

   Building an effective AI analyst requires grounding responses in internal organizational data. While public web data is universally accessible, internal documents possess granular permissions tied to specific user groups or individual assignments. Managing these security constraints during information retrieval represents a notoriously difficult systems engineering challenge.

   • AI analysts process questions by retrieving facts from both public web sources and private internal databases.
   • Public documents present no permission barriers because all organizational users share equal access.
   • Internal organizational documents carry complex, individualized permission rules across emails, meeting notes, and reports.
2. 02:07 - 06:24

   Bridgewater's Research Process and The Vault

   For 50 years, Bridgewater has pursued a systematic understanding of global markets by placing human researchers at the center of a five-step learning loop. Transitioning this research methodology toward machine-led AI agents requires continuous access to high-quality data repositories. The Vault aggregates massive volumes of structured time-series data and unstructured research while enforcing strict document-level permissions.

   • Bridgewater systematically encodes macroeconomic principles into algorithmic models, evolving over decades from paper legal pads and Excel to Scala and Python.
   • The internal learning framework follows a five-step research cycle consisting of perception, hypothesis, investigation, systemization, and industrialization.
   • The Vault indexes approximately 100 million documents derived from 135 external research providers as well as internal writing and meeting transcripts.
3. 06:24 - 09:43

   TurboPuffer Architecture and Object Storage Economics

   Traditional disk-based search engines incur massive infrastructure costs when scaling to terabytes of enterprise data. TurboPuffer addresses this by leveraging the low cost of cloud object storage, only paying for expensive memory and SSD caching when queries actually hit specific data regions. At 100 million document scale, naive linear scans of memory become computationally unviable for real-time applications.

   • TurboPuffer builds search infrastructure directly on top of hyperscaler object storage systems like S3, GCS, and Azure Blob Storage.
   • Cold data remains inert on object storage at minimal cost until active queries pull specific namespaces into high-performance SSD or memory caches.
   • Naive sequential scanning across 100 million documents requires several seconds even on modern hardware, failing standard 100-millisecond latency requirements.
4. 09:43 - 17:48

   Native Filtering for Scalable Vector Search

   Standard vector indices speed up similarity searches by zoning in on nearby coordinate clusters, but permission filters break this optimization when authorized documents scatter across the vector space. Post-filtering strategies often fail to find enough matching documents before timing out. By combining pre-filtering and post-filtering into native filtering, TurboPuffer evaluates cluster-level permissions first, achieving 120-millisecond P90 latencies across 450 million vectors.

   • Vector embeddings translate conceptual relationships into multi-dimensional numeric coordinates where proximity signifies semantic similarity.
   • Pure post-filtering discards non-permitted search results after vector retrieval, frequently resulting in poor recall and truncated result sets.
   • Native filtering groups vectors into clusters, checks user permissions against those clusters to discard unauthorized data, and performs localized sequential scans.