Gemini's New File Search Just Leveled Up RAG Agents (10x Cheaper)

Gemini File Search API Overview [00:00]

• Gemini's new file search API simplifies the process of interacting with documents by allowing direct uploads, automatic embedding generation, and immediate chat capabilities.
• This functionality is being offered at a low price, contributing to its recent visibility.
• It enables the creation of RAG agents in Naden without the need for extensive data pipeline construction.

"So Gemini's new file search API is awesome because it allows you to drop in a document. It generates those embeddings and then you're right away able to just chat with it and ask it questions and get answers back."

How Gemini File Search Works [00:33]

• Users upload files to Gemini, which then automatically chunks and embeds them, making them accessible for Niten agents as knowledge.
• This process is similar to using vector databases like Pinecone or Superbase but eliminates the complex data pipeline typically involved in understanding file types, adding metadata, splitting content, and processing embeddings.
• Gemini handles all storage and searching, removing the need for users to build their own search systems or databases.

"Basically you upload your files to Gemini. It automatically chunks and embeds them and then you can have your nitni agents using this as knowledge."

Gemini File Search Pricing [01:26]

• The pricing model charges only for uploading, indexing, or embedding, at a rate of 15 cents per 1 million tokens processed.
• A 121-page PDF (approximately 95,000 tokens) costs less than 15 cents for processing.
• Storage is currently free for all uploaded data.
• Querying costs are primarily for chat model usage, with a table estimating costs for 100 GB storage and 1 million queries per month at $0 for storage, $12 for indexing, and an estimated $35 base fee for high query volume.
• Compared to Superbase and Pine Cone, Gemini's File Search is presented as significantly cheaper and easier to set up, especially for direct document chat functionality.

"They only charge you for uploading or for your indexing or your embedding. So when you upload a file, you pay 15 cents for every 1 million tokens processed."

Naden Implementation: Four HTTP Requests [03:27]

• The implementation in Naden requires four core HTTP requests.
• The first request is to create a file store (referred to as a folder for easier contextualization).
• The second is to upload a file to the Google Cloud environment.
• The third request moves the uploaded file into the previously created folder.
• The fourth request sets up the query to connect an AI agent to the tool, enabling chat interaction with the file's content.

"There's going to be four HTTP requests that you guys will be looking at, and I'm going to walk through exactly how I set them up."

Setting Up the 'Create Store' Request [05:40]

• The process begins by referring to the API documentation, specifically the "importing files" section.
• The first operation is to create a file search store using a POST request.
• The endpoint URL is copied, and a Gemini API key is required, which can be obtained from Google AI Studio.
• It's recommended to set up an authentication query parameter for the API key instead of pasting it directly in the URL for better security and convenience.
• A file store name, such as "YouTube-test", is chosen and the request is executed, returning a success message with the store's name, display name, and creation time.

"So the first thing that you can see what we have to do is we have to create a file search store."

Uploading and Importing Files [07:40]

• After creating the store, the next step is to upload a file to Google and then import it into the created store.
• This involves using an "initiate resumable upload" POST request from the API documentation.
• The API endpoint ends in "files," and the previously set up query off for the API key is used.
• The Naden binary file is then sent in the body of the request.
• The upload process temporarily holds the file in Google's environment with an expiration time, providing metadata like name, MIME type, size, URI, state, and source.

"The next thing we need to do is we need to upload the file to Google and then we have to move it into that file store."

Importing Files into the Store [09:18]

• To move the file from Google's temporary storage into the designated file store, an "import files into the right store" POST request is used.
• This request requires the store name to be part of the endpoint URL, not a query parameter.
• The name of the file store created in the first step is copied and pasted into the URL.
• The JSON body of the request includes the name of the file that was just uploaded.
• Executing this step provides a success message, confirming the file has been successfully placed into the folder.

"What we have to do is actually move that file that's up in Google's cloud into our folder."

Querying the Knowledge-Based Tool [10:41]

• The final step is to query the agent using a "generate content" POST request, which retrieves an answer.
• The request uses Gemini 2.5 Flash as the model and requires the store name to be specified to know where to search.
• The "from AI function" is used for the query, allowing the agent to formulate the question based on user input.
• The prompt instructs the agent to be a helpful RAG agent, answer questions using the knowledge-based tool, ground answers in truth, and cite sources. It also specifies that queries to the tool should only contain text, without punctuation, question marks, or new lines.
• A demo shows the agent correctly answering a question about club breakage during a golf round, citing the source.

"The way that this knowledge based tool works is it is another HTTP request."

Testing with Multiple Documents and Evaluation [12:37]

• The system is tested by uploading a second document (about Nvidia) and asking a question regarding its Q1 fiscal summary, which the agent successfully answers, citing the Nvidia press release.
• The output from the knowledge-based tool shows a candidates array with the answer text and grounding metadata, including the specific chunks and their sources.
• A more comprehensive evaluation involves uploading three PDFs: Rules of Golf (22 pages), Nvidia announcement (9 pages), and Apple's 10-K (121 pages), totaling nearly 200 pages.
• Ten questions are posed across these documents, and the agent achieves a correctness score of 4.2 out of 5, with individual scores of 5, 5, 4, and 5 for most questions, despite the documents having no correlation and being in the same store.

"We have three total PDFs in our store. We have the rules of golf, which is 22page PDF. We've got the Nvidia announcement, which was nine pages. And then we have Apple's 10K, which was 121 pages."

Final Considerations for RAG and Gemini File Search [15:49]

• Data Management: The API doesn't automatically handle updates or deletions, leading to potential duplicate data if files are re-uploaded, which can degrade response quality.
• Document Quality: The effectiveness of responses depends on the quality of the input documents; "garbage in, garbage out" applies, and pre-processing may be necessary for messy or poorly scanned files. Gemini's OCR capabilities are mentioned but not a cure-all.
• Chunk-Based Retrieval Limitations: For needing the context of an entire document, transcript, or video, chunk-based retrieval might not be optimal unless combined with granular metadata tagging. An example is given where asking for the total number of rules in a PDF returned an incorrect answer because the system had to chunk it.
• Security and Privacy: Documents are stored on Google's servers, so sensitive information like PII should not be uploaded, as it will be processed and indexed, compromising privacy. Compliance with regulations like GDPR, HIPAA, or CCPA should be considered.

"It's great that you can drop in these files, but what happens if you need to update a file and throw it back in? Now you have duplicate data because Google isn't really keeping a record of all of this data and making sure that things aren't duplicated or that old ones need to be updated and deleted, things like that."