Multi-LLM RAG Benchmark: Document Q&A with Groundtruth Comparison

Pixeltable is a declarative interface for working with text, images, embeddings, and even video, enabling you to store, transform, index, and iterate on data.

Disclaimer: This app is running on OpenAI, Mistral, and Fireworks accounts with my own API keys 😞. This Hugging Face Space uses the free tier (2vCPU, 16GB RAM), which may result in slower processing times, especially for embedding generation and large document processing. Embeddings are generated using the sentence-transformer library with the 'intfloat/e5-large-v2' model. If you wish to use this app with your own hardware or API keys for improved performance, you can: duplicate this Hugging Face Space, run it locally, or use Google Colab with the Free limited GPU support.

  1. Ingests Documents: Uploads your PDF documents and a ground truth file (CSV or XLSX).
  2. Process and Retrieve Data: Store, chunk, index, orchestrate, and retrieve all data.
  3. Generates Answers: Leverages OpenAI to produce accurate answers based on the retrieved context.
  4. Compares Results: Displays the generated answers alongside the ground truth for easy evaluation.
  1. Upload your ground truth file (CSV or XLSX) with the following two columns: question and correct_answer.
  2. Upload one or more PDF documents that contain the information to answer these questions.
  3. Click "Process Files and Generate Output" to start the RAG process.
  4. View the results in the table below, comparing AI-generated answers to the ground truth.
100 500
Chunk Separator

Pixeltable Table