GopherGate

A unified, high-performance LLM proxy gateway built in Go. It provides OpenAI-compatible /v1/chat/completions, /v1/images/generations, /v1/responses, and /v1/models endpoints to access multiple providers (OpenAI, Gemini, DeepSeek, Moonshot, Grok, Ollama) with built-in token tracking, real-time cost calculation, multi-user authentication, and a management dashboard.

Features

• Unified API: OpenAI-compatible /v1/chat/completions, /v1/images/generations, /v1/responses, and /v1/models endpoints.
  • The /v1/responses endpoint (OpenAI Responses API) is currently supported for OpenAI models only. Non-OpenAI providers (Gemini, DeepSeek, Moonshot, Grok, Ollama) return a "not supported" response.
• Multi-Provider Support:
  • OpenAI: GPT-4o, GPT-4o Mini, GPT-5, GPT-5.4, o1/o3/o4 reasoning models, DALL-E 2/3 image generation.
  • Google Gemini: Gemini 2.5 Flash/Pro, Gemini 3 Flash/Pro previews, Imagen 3 image generation.
  • DeepSeek: DeepSeek Chat, Reasoner, V4 Flash, V4 Pro.
  • Moonshot: Kimi K2.5, K2.6 reasoning models.
  • xAI Grok: Grok-3, Grok-4, Grok-4.3 reasoning models.
  • Ollama: Local LLMs running on your network.
• Observability & Tracking:
  • Asynchronous Logging: Non-blocking request logging to SQLite using background workers.
  • Token Counting: Precise estimation and tracking of prompt, completion, and reasoning tokens.
  • Database Persistence: Every request logged to SQLite for historical analysis and dashboard analytics.
  • Streaming Support: Full SSE (Server-Sent Events) support for all providers.
• Multimodal (Vision): Image processing (Base64 and remote URLs) across compatible providers.
• Image Generation: DALL-E 2/3 (OpenAI) and Imagen 3 (Gemini) via OpenAI-compatible /v1/images/generations endpoint.
• Automatic Model Routing:
  • Hierarchical Routing: Groups can target other groups, cascading through multiple levels until a concrete model is reached. Cycle detection and depth limiting (max 10) prevent infinite loops.
  • Heuristic strategy: Free, zero-latency keyword matching (e.g. "debug" or "step by step" routes to the reasoning model).
  • Classifier strategy: Uses a cheap LLM to rate task complexity on a configurable scale (1-10), then selects the appropriate model. Bucket mapping distributes ratings proportionally across targets.
  • Two-Level Dispatch: A dispatcher group (classifier, threshold=10) auto-routes to tier groups by complexity score, which then apply their own internal strategies.
  • Metadata: Groups support logic_level (1-10 complexity scale) and primary_use (description) fields for organizational clarity.
  • Pre-seeded with provider groups, tier groups (heavy-logic / standard-pro / fast-flow), and a dispatcher. Model groups are exposed in /v1/models so clients can discover them.
• Multi-User Access Control:
  • Admin Role: Full access to all dashboard features, user management, and system configuration.
  • Viewer Role: Read-only access to usage analytics, costs, and monitoring.
  • Client API Keys: Create and manage multiple client tokens for external integrations.
• Reliability:
  • Circuit Breaking: Protects providers when they are down, auto-recovers after timeout.
  • Provider-Aware Classification: Classifier selector models are routed to the correct provider automatically.

DeepSeek Language Note

DeepSeek models default to Chinese for some prompts. GopherGate automatically injects an English system prompt ("Always respond in English.") when no system message is present. If the client provides its own system prompt, it is left untouched.

Security

GopherGate is designed with security in mind:

• Signed Session Tokens: Management dashboard sessions are secured using HMAC-SHA256 signed tokens.
• Encrypted Storage: Support for encrypted provider API keys in the database.
• Auth Middleware: Secure client authentication via database-backed API keys.

Note: You must define an LLM_PROXY__ENCRYPTION_KEY in your .env file for secure session signing and encryption.

Tech Stack

• Runtime: Go 1.22+
• Web Framework: Gin Gonic
• Database: sqlx with SQLite (CGO-free via modernc.org/sqlite)
• Frontend: Vanilla JS/CSS with Chart.js for visualizations

Getting Started

Prerequisites

• Go (1.22+)
• SQLite3 (optional, driver is built-in)
• Docker (optional, for containerized deployment)

Quick Start

1. Clone and build:

   git clone <repository-url>
   cd gophergate
   go build -o gophergate ./cmd/gophergate
   

2. Configure environment:

   cp .env.example .env
   # Edit .env and add your configuration:
   # LLM_PROXY__ENCRYPTION_KEY=... (32-byte hex or base64 string)
   # OPENAI_API_KEY=sk-...
   # GEMINI_API_KEY=AIza...
   # DEEPSEEK_API_KEY=sk-...
   # MOONSHOT_API_KEY=...
   # GROK_API_KEY=xai-...
   # For Ollama (optional): Set base URL and enable
   # LLM_PROXY__PROVIDERS__OLLAMA__BASE_URL=http://localhost:11434/v1
   # LLM_PROXY__PROVIDERS__OLLAMA__ENABLED=true
   # LLM_PROXY__PROVIDERS__OLLAMA__MODELS=llama3,gemma2,mistral
   

3. Run the proxy:

   ./gophergate
   

The server starts on http://0.0.0.0:8080 by default. Configure LLM_PROXY__SERVER__PORT in .env to change it.

Quick Deploy Script

A deploy.sh script is included for production restarts:

./deploy.sh
# git pull -> go build -> stop old process -> start new process

Deployment (Docker)

# Build the container
docker build -t gophergate .

# Run the container
docker run -p 8080:8080 \
  -e LLM_PROXY__ENCRYPTION_KEY=your-secure-key \
  -v ./data:/app/data \
  gophergate

Management Dashboard

Access the dashboard at http://localhost:8080.

• Auth: Login, session management, and status tracking.
• Usage: Summary stats, time-series analytics, and provider breakdown.
• Clients: API key management and per-client usage tracking.
• Providers: Provider configuration and status monitoring.
• Model Groups: Define auto-routing groups with heuristic or classifier strategies. Supports logic level and primary use metadata.
• Models: Model enable/disable and cost configuration.
• Users: Admin-only user management for dashboard access.
• Monitoring: Live request stream via WebSocket.

Default Credentials

• Username: admin
• Password: admin123 (You will be prompted to change this on first login)

Forgot Password? You can reset the admin password to default by running:

./gophergate -reset-admin

API Usage

The proxy is a drop-in replacement for OpenAI. Configure your client:

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="YOUR_CLIENT_API_KEY"  # Create in dashboard
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

Responses API

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="YOUR_CLIENT_API_KEY"
)

# OpenAI Responses API (supported for OpenAI models only)
response = client.responses.create(
    model="gpt-4o",
    input="Explain quantum computing in one paragraph.",
    instructions="You are a helpful assistant.",
    temperature=0.7,
    max_output_tokens=500
)
print(response.output_text)

Note: The /v1/responses endpoint is currently supported for OpenAI models only.

Automatic Model Routing

Use a model group name to let gophergate pick the best model automatically:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="YOUR_CLIENT_API_KEY"
)

# Simple query -- routes to the cheap/fast model
response = client.chat.completions.create(
    model="fast-flow",
    messages=[{"role": "user", "content": "What is 2+2?"}]
)

# Complex query -- routes to the reasoning model automatically
response = client.chat.completions.create(
    model="heavy-logic",
    messages=[{"role": "user", "content": "Write a Python red-black tree implementation."}]
)

Two-Level Dispatch

The dispatcher group uses a classifier to score prompts 1-10, then routes to the appropriate tier group:

# Automatically routed based on complexity:
# 1-3  -> fast-flow    (classification, basic Q&A)
# 4-7  -> standard-pro (general assistant, long docs)
# 8-10 -> heavy-logic  (complex coding, logic, agents)
response = client.chat.completions.create(
    model="dispatcher",
    messages=[{"role": "user", "content": "Debug this race condition in my Go code."}]
)
# This goes: dispatcher -> heavy-logic -> deepseek-v4-pro

Pre-seeded groups:

Group	Level	Strategy	Targets	Primary Use
fast-flow	2	heuristic	deepseek-v4-flash, gpt-5.4-nano	Classification, JSON, Basic Q&A
standard-pro	5	heuristic	gpt-5.4-mini, gemini-3-flash-preview	General Assistant, Long Docs
heavy-logic	9	heuristic	grok-4.3, kimi-k2.6, deepseek-v4-pro	Complex Coding, Logic, Agents
dispatcher	-	classifier	fast-flow, standard-pro, heavy-logic	Auto-dispatches by complexity
deepseek-auto	-	heuristic	deepseek-chat, deepseek-reasoner	Legacy provider group
openai-auto	-	heuristic	gpt-4o-mini, gpt-4o	Legacy provider group
gemini-auto	-	heuristic	gemini-2.0-flash, gemini-2.5-pro	Legacy provider group

Image Generation (DALL-E / Imagen)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="YOUR_CLIENT_API_KEY"
)

# DALL-E 3 (OpenAI)
resp = client.images.generate(
    model="dall-e-3",
    prompt="A cute gopher wearing a top hat",
    n=1,
    size="1024x1024"
)
print(resp.data[0].url)

# Imagen 3 (Gemini) -- uses same endpoint
resp = client.images.generate(
    model="imagen-3.0-generate-001",
    prompt="A gopher coding in Go",
    n=1,
    size="1024x1024"
)
print(resp.data[0].url)  # Returns data URI (Gemini returns base64)

License

MIT