LobotomyLabs/GopherGate
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update README with Xiaomi MiMo support, HTTP connection pooling, token caching, and thread safety
CI / Lint (push) Has been cancelled
Details
CI / Test (push) Has been cancelled
Details
CI / Build (push) Has been cancelled
Details

Browse Source
This commit is contained in:
newkirk
2026-07-21 13:43:57 -04:00
parent 654f6ab6d1
commit d23da551ab
1 changed files with 7 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -2
View File
@@ -5,19 +5,24 @@ A unified, high-performance LLM proxy gateway built in Go. It provides OpenAI-co
## Features
- **Unified API:** OpenAI-compatible `/v1/chat/completions`, `/v1/images/generations`, `/v1/responses`, and `/v1/models` endpoints.
- The `/v1/responses` endpoint (OpenAI Responses API) is currently supported for OpenAI models only. Non-OpenAI providers (Gemini, DeepSeek, Moonshot, Grok, Ollama) return a "not supported" response.
- The `/v1/responses` endpoint (OpenAI Responses API) is currently supported for OpenAI models only. Non-OpenAI providers (Gemini, DeepSeek, Moonshot, Grok, Ollama, Xiaomi) return a "not supported" response.
- **Multi-Provider Support:**
- **OpenAI:** GPT-4o, GPT-4o Mini, GPT-5, GPT-5.4, o1/o3/o4 reasoning models, DALL-E 2/3 image generation.
- **Google Gemini:** Gemini 2.5 Flash/Pro, Gemini 3 Flash/Pro previews, Imagen 3 image generation.
- **DeepSeek:** DeepSeek Chat, Reasoner, V4 Flash, V4 Pro.
- **Moonshot:** Kimi K2.5, K2.6 reasoning models.
- **xAI Grok:** Grok-3, Grok-4, Grok-4.3 reasoning models.
- **Xiaomi MiMo:** MiMo v2.5 models.
- **Ollama:** Local LLMs running on your network.
- **Observability & Tracking:**
- **Asynchronous Logging:** Non-blocking request logging to SQLite using background workers.
- **Token Counting:** Precise estimation and tracking of prompt, completion, and reasoning tokens.
- **Database Persistence:** Every request logged to SQLite for historical analysis and dashboard analytics.
- **Streaming Support:** Full SSE (Server-Sent Events) support for all providers.
- **Streaming Support:** Full SSE (Server-Sent Events) support with `X-Accel-Buffering: no` for unbuffered, low-latency streaming.
- **High Performance & Thread Safety:**
- **Connection Pooling:** Shared HTTP transport with connection pooling (`MaxIdleConns: 200`), TCP keep-alives, and HTTP/2 multiplexing across all provider drivers.
- **In-Memory Token Caching:** In-memory `sync.Map` TTL caching (10s valid, 2s negative cache) for client token authentication to eliminate SQLite bottlenecking.
- **Thread-Safe Core:** Full RWMutex locking across provider maps, model registry lookups, and router reloads.
- **Multimodal (Vision):** Image processing (Base64 and remote URLs) across compatible providers.
- **Image Generation:** DALL-E 2/3 (OpenAI) and Imagen 3 (Gemini) via OpenAI-compatible `/v1/images/generations` endpoint.
- **Automatic Model Routing:**