diff --git a/BACKEND_ARCHITECTURE.md b/BACKEND_ARCHITECTURE.md index 6a459a00..cf432628 100644 --- a/BACKEND_ARCHITECTURE.md +++ b/BACKEND_ARCHITECTURE.md @@ -30,7 +30,7 @@ The GopherGate backend is implemented in Go, focusing on high performance, clear ## Key Components ### 1. Provider Interface (`internal/providers/provider.go`) -Standardized interface for all LLM backends. Implementations handle mapping between the unified format and provider-specific APIs (OpenAI, Gemini, DeepSeek, Grok). +Standardized interface for all LLM backends. Implementations handle mapping between the unified format and provider-specific APIs (OpenAI, Gemini, DeepSeek, Grok, Moonshot, Ollama). ### 2. Model Registry & Pricing (`internal/utils/registry.go`) Integrates with `models.dev/api.json` to provide real-time model metadata and pricing. diff --git a/README.md b/README.md index c93b932d..64294a29 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,10 @@ GopherGate is designed with security in mind: # OPENAI_API_KEY=sk-... # GEMINI_API_KEY=AIza... # MOONSHOT_API_KEY=... + # For Ollama (optional): Set base URL and enable + # LLM_PROXY__PROVIDERS__OLLAMA__BASE_URL=http://localhost:11434/v1 + # LLM_PROXY__PROVIDERS__OLLAMA__ENABLED=true + # LLM_PROXY__PROVIDERS__OLLAMA__MODELS=llama3,gemma2,mistral ``` 3. Run the proxy: @@ -120,6 +124,10 @@ Moonshot models are available through the same OpenAI-compatible endpoint. For example, use `kimi-k2.5` as the model name after setting `MOONSHOT_API_KEY` in your environment. +Ollama models (like `llama3`, `gemma2`, `mistral`) are also available through the same +endpoint after enabling Ollama in configuration and setting the base URL to your +Ollama server (default: `http://localhost:11434/v1`). + ### Python ```python from openai import OpenAI