# RAG with TypeScript

Canonical: https://gaboesquivel.com/blog/2025-05-typescript-rag

**Published:** 2025-06-02
**Summary:** How retrieval-augmented generation works, and how to implement it with TypeScript SDKs, embeddings, and vector stores.
**Categories:** Engineering, Artificial Intelligence, Backend, Fullstack
**Technologies:** NodeJS, TypeScript, JavaScript, AI SDK, OpenAI

A language model can use information supplied with a request, but it doesn't automatically have access to a team's internal documents or recently changed facts. [Retrieval-augmented generation (RAG)](https://gaboesquivel.com/ai) connects the model to an external knowledge source. The application retrieves relevant information, passes it into the model's context, and uses the result to produce an answer that can reference those sources.

## How RAG works

A retrieval system has two parts: preparing information for search and finding the relevant material when a question arrives.

1. **Indexing.** Extract text from documents, divide it into useful passages, attach source metadata, and build a searchable index. Embeddings can capture semantic similarity, while keyword indexes preserve precise names and terms. The index can change without retraining the language model.
2. **Retrieval and generation.** When a question arrives, search the index for relevant passages. Depending on the application, combine semantic and keyword retrieval, filter results by the user's permissions, and rerank candidates before passing selected evidence to the model. The model generates an answer from the supplied context.

![Separate RAG indexing and query pipelines connected through a search index](https://gaboesquivel.com/images/diagrams/2025-05-typescript-rag.svg)

RAG prepares searchable knowledge ahead of time, then retrieves relevant authorized context when answering a question.

RAG can improve factual grounding and expose current or private information, but retrieval alone doesn't guarantee a correct answer. The application still needs to consider source quality, access control, conflicting documents, and how the answer is evaluated.

## Similarity search

Semantic search is the core of retrieval. Instead of matching exact words, embeddings place text in a high-dimensional space where proximity reflects meaning, so "attorney" and "lawyer" land close together even when a document uses only one of them.

Similarity is measured with a distance metric such as cosine similarity, and vector databases use approximate nearest neighbor (ANN) algorithms to find the closest chunks without comparing every vector. The result is retrieval that matches a query to documents by concept rather than by syntax.

## Where RAG fits

RAG is the right architecture for:

* Domain-specific assistants that answer from a custom knowledge base.
* Chatbots that have to stay accurate as their sources change.
* Document Q&A across PDFs, websites, or internal wikis.

## Node.js in the RAG stack

[Node.js](https://gaboesquivel.com/tech/nodejs) is a practical choice for RAG when a team already works in [JavaScript](https://gaboesquivel.com/tech/javascript) and [TypeScript](https://gaboesquivel.com/tech/typescript), because the embedding APIs and integrations are mature enough to cover the whole pipeline:

* Generate embeddings with OpenAI or the Hugging Face Inference API.
* Store and query vectors in `pgvector`, Pinecone, Weaviate, or Qdrant.
* Run [end-to-end workflows](https://gaboesquivel.com/fullstack) in Next.js server functions, Vercel edge functions, or Node.js servers.

A growing set of open-source SDKs built for LLM apps covers the rest, with patterns for streaming responses, function calling, and vector search.

## Open-source TypeScript AI SDKs

### Vercel AI SDK

Vercel's TypeScript-first toolkit for AI apps, with streaming and tool use across React, Next.js, and Node.js:

* A unified API for OpenAI, Anthropic, and other providers.
* Streaming UI and edge-runtime compatibility.
* Tool calling with typed context.
* [https://github.com/vercel/ai](https://github.com/vercel/ai)

### LangChain.js

The JavaScript and TypeScript version of LangChain, for LLM chains, agents, and RAG pipelines:

* Chains, tools, agents, retrievers, and memory.
* Integrations with vector stores like Pinecone, Supabase, and Weaviate.
* Structured output parsing, evals, and streaming.
* [https://github.com/langchain-ai/langchainjs](https://github.com/langchain-ai/langchainjs)

### LlamaIndex.TS

The TypeScript version of LlamaIndex, for ingesting, indexing, and querying data in RAG systems:

* Indexing for documents and metadata.
* LLM-driven query planning.
* Retriever composition and routing.
* [https://github.com/jerryjliu/llamaindex-ts](https://github.com/jerryjliu/llamaindex-ts)

### Mastra

A TypeScript agent framework with persistent memory, threads, and embedded knowledge graphs:

* Agents with tool use and long-term memory.
* JSON knowledge graphs and fact-based reasoning.
* Context-aware multi-step workflows.
* [https://github.com/mastra-ai/mastra](https://github.com/mastra-ai/mastra)

### Agentic.so

A standard library of TypeScript AI tools that exposes functions to LLMs cleanly:

* Typed agent workflows.
* AI-accessible function definitions.
* An SDK for tool abstraction.
* [https://github.com/agentic-dev/agentic](https://github.com/agentic-dev/agentic)

## Vector storage options

Where the vectors live depends on the stack you already run:

* `pgvector` is a PostgreSQL extension for vector similarity search, and it fits full-stack apps that already use Postgres.
* Dedicated vector databases like Pinecone, Weaviate, and Qdrant add scalable nearest-neighbor search and metadata filtering.
* OpenAI's managed vector store, released in 2024, is integrated with their embeddings and models and suits small-to-medium workloads where you want little operational overhead.

## RAG in projects I've shipped

On [LegalAgent](https://gaboesquivel.com/project/legal-agent), RAG supplies case context, document summaries, and procedural guidance to a legal assistant, and on [Bitlauncher](https://gaboesquivel.com/project/bitlauncher), an AI and crypto launchpad, the chatbot uses RAG for longer answers about projects. Python still leads the AI tooling ecosystem, but the TypeScript stack has been enough for both, and it lets web engineers ship AI features without switching languages.