docs: sync documentation with current implementation and archive stale plan

RUST_BACKEND_REVIEW.md

@@ -0,0 +1,58 @@
+# LLM Proxy Rust Backend Code Review Report
+
+## Executive Summary
+This code review examines the `llm-proxy` Rust backend, focusing on architectural soundness, performance characteristics, and adherence to Rust idioms. The codebase demonstrates solid engineering with well-structured modular design, comprehensive error handling, and thoughtful async patterns. However, several areas require attention for production readiness, particularly around thread safety, memory efficiency, and error recovery.
+
+## 1. Core Proxy Logic Review
+### Strengths
+- Clean provider abstraction (`Provider` trait).
+- Streaming support with `AggregatingStream` for token counting.
+- Model mapping and caching system.
+
+### Issues Found
+- **Provider Manager Thread Safety Risk:** O(n) lookups using `Vec` with `RwLock`. Use `DashMap` instead.
+- **Streaming Memory Inefficiency:** Accumulates complete response content in memory.
+- **Model Registry Cache Invalidation:** No strategy when config changes via dashboard.
+
+## 2. State Management Review
+- **Token Bucket Algorithm Flaw:** Custom implementation lacks thread-safe refill sync. Use `governor` crate.
+- **Broadcast Channel Unbounded Growth Risk:** Fixed-size (100) may drop messages.
+- **Database Connection Pool Contention:** SQLite connections shared without differentiation.
+
+## 3. Error Handling Review
+- **Error Recovery Missing:** No circuit breaker or retry logic for provider calls.
+- **Stream Error Logging Gap:** Stream errors swallowed without logging partial usage.
+
+## 4. Rust Idioms and Performance
+- **Unnecessary String Cloning:** Frequent cloning in authentication hot paths.
+- **JSON Parsing Inefficiency:** Multiple passes with `serde_json::Value`. Use typed structs.
+- **Missing `#[derive(Copy)]`:** For small enums like `CircuitState`.
+
+## 5. Async Performance
+- **Blocking Calls:** Token estimation may block async runtime.
+- **Missing Connection Timeouts:** Only overall timeout, no separate read/write timeouts.
+- **Unbounded Task Spawn:** For client usage updates under load.
+
+## 6. Security Considerations
+- **Token Leakage:** Redact tokens in `Debug` and `Display` impls.
+- **No Request Size Limits:** Vulnerable to memory exhaustion.
+
+## 7. Testability
+- **Mocking Difficulty:** Tight coupling to concrete provider implementations.
+- **Missing Integration Tests:** No E2E tests for streaming.
+
+## 8. Summary of Critical Actions
+### High Priority
+1. Replace custom token bucket with `governor` crate.
+2. Fix provider lookup O(n) scaling issue.
+3. Implement proper error recovery with retries.
+4. Add request size limits and timeout configurations.
+
+### Medium Priority
+1. Reduce string cloning in hot paths.
+2. Implement cache invalidation for model configs.
+3. Add connection pooling separation.
+4. Improve streaming memory efficiency.
+
+---
+*Review conducted by: Senior Principal Engineer (Code Reviewer)*