Skip to main content

LLM Providers

Ravioli supports local, private, and cloud-hosted Large Language Models (LLMs) to drive query synthesis, schema inference, SQL code generation, and interactive AI notebook execution.

Privacy & Cost Efficiency

By default, Ravioli prioritizes local execution via Ollama (Local) to ensure absolute data privacy and zero inference costs. However, in many corporate environments where local hardware limitations or strict context size needs override these concerns, Ravioli offers native integrations with cloud-hosted alternatives like Ollama Cloud and upcoming integrations with Google Gemini.

Please be aware of the trade-off: by adding integration with Ollama Cloud or Google Gemini, it essentially means the information and queried data will be sent over to the providers' servers for processing.

Ollama (Default Local & Cloud)​

Ravioli supports local execution and cloud-hosted setups for Ollama.

  • Setup & Configuration: See the detailed Ollama Setup Guide for connection details, routing configurations, and RAM/VRAM resource saving mechanisms.

Google Gemini (Upcoming)​

Google AI Studio integration is planned to support Gemini models.

  • Capabilities: See the Gemini Setup Guide for features like massive context windows and database schema/code analysis.

Configuration Schema​

The settings database stores LLM configurations under the ollama system setting key.

JSON Payload Schema​

{
  "mode": "custom", 
  "base_url": "http://host.docker.internal:11434",
  "default_model": "gemma3:8b",
  "api_key": "••••••••"
}

Connection Testing​

You can verify the connection status directly in the UI using the Test Connection button, which calls the endpoint: GET /api/v1/settings/ollama/test

This endpoint runs a connection handshake with the target Ollama node to verify accessibility and list all loaded/available models on that Ollama service.