100 lines
4.6 KiB
Markdown
100 lines
4.6 KiB
Markdown
# Project Plan: LLM Proxy Enhancements & Security Upgrade
|
|
|
|
This document outlines the roadmap for standardizing frontend security, cleaning up the codebase, upgrading session management to HMAC-signed tokens, and extending integration testing.
|
|
|
|
## Phase 1: Frontend Security Standardization
|
|
**Primary Agent:** `frontend-developer`
|
|
|
|
- [x] Audit `static/js/pages/users.js` for manual HTML string concatenation.
|
|
- [x] Replace custom escaping or unescaped injections with `window.api.escapeHtml`.
|
|
- [x] Verify user list and user detail rendering for XSS vulnerabilities.
|
|
|
|
## Phase 2: Codebase Cleanup
|
|
**Primary Agent:** `backend-developer`
|
|
|
|
- [x] Identify and remove unused imports in `src/config/mod.rs`.
|
|
- [x] Identify and remove unused imports in `src/providers/mod.rs`.
|
|
- [x] Run `cargo clippy` and `cargo fmt` to ensure adherence to standards.
|
|
|
|
## Phase 3: HMAC Architectural Upgrade
|
|
**Primary Agents:** `fullstack-developer`, `security-auditor`, `backend-developer`
|
|
|
|
### 3.1 Design (Security Auditor)
|
|
- [x] Define Token Structure: `base64(payload).signature`.
|
|
- Payload: `{ "session_id": "...", "username": "...", "role": "...", "exp": ... }`
|
|
- [x] Select HMAC algorithm (HMAC-SHA256).
|
|
- [x] Define environment variable for secret key: `SESSION_SECRET`.
|
|
|
|
### 3.2 Implementation (Backend Developer)
|
|
- [x] Refactor `src/dashboard/sessions.rs`:
|
|
- Integrate `hmac` and `sha2` crates (or similar).
|
|
- Update `create_session` to return signed tokens.
|
|
- Update `validate_session` to verify signature before checking store.
|
|
- [x] Implement activity-based session refresh:
|
|
- If session is valid and >50% through its TTL, extend `expires_at` and issue new signed token.
|
|
|
|
### 3.3 Integration (Fullstack Developer)
|
|
- [x] Update dashboard API handlers to handle new token format.
|
|
- [x] Update frontend session storage/retrieval if necessary.
|
|
|
|
## Phase 4: Extended Integration Testing
|
|
**Primary Agent:** `qa-automation`
|
|
|
|
- [ ] Setup test environment with encrypted key storage enabled.
|
|
- [ ] Implement end-to-end flow:
|
|
1. Store encrypted provider key via API.
|
|
2. Authenticate through Proxy.
|
|
3. Make proxied LLM request (verifying decryption and usage).
|
|
- [ ] Validate HMAC token expiration and refresh logic in automated tests.
|
|
|
|
## Phase 5: Code Quality & Refactoring
|
|
**Primary Agent:** `fullstack-developer`
|
|
|
|
- [x] Refactor dashboard monolith into modular sub-modules (`auth.rs`, `usage.rs`, etc.).
|
|
- [x] Standardize error handling and remove `unwrap()` in production paths.
|
|
- [x] Implement system health metrics and backup functionality.
|
|
|
|
---
|
|
|
|
# Phase 6: Cache Cost & Provider Audit (ACTIVE)
|
|
**Primary Agents:** `frontend-developer`, `backend-developer`, `database-optimizer`, `lab-assistant`
|
|
|
|
## 6.1 Dashboard UI Updates (@frontend-developer)
|
|
- [ ] **Update Models Page Modal:** Add input fields for `Cache Read Cost` and `Cache Write Cost` in `static/js/pages/models.js`.
|
|
- [ ] **API Integration:** Ensure `window.api.put` includes these new cost fields in the request body.
|
|
- [ ] **Verify Costs Page:** Confirm `static/js/pages/costs.js` displays these rates correctly in the pricing table.
|
|
|
|
## 6.2 Provider Audit & Stream Fixes (@backend-developer)
|
|
- [ ] **Standard DeepSeek Fix:** Modify `src/providers/deepseek.rs` to stop stripping `stream_options` for `deepseek-chat`.
|
|
- [ ] **Grok Audit:** Verify if Grok correctly returns usage in streaming; it uses `build_openai_body` and doesn't seem to strip it.
|
|
- [ ] **Gemini Audit:** Confirm Gemini returns `usage_metadata` reliably in the final chunk.
|
|
- [ ] **Anthropic Audit:** Check if Anthropic streaming requires `include_usage` or similar flags.
|
|
|
|
## 6.3 Database & Migration Validation (@database-optimizer)
|
|
- [ ] **Test Migrations:** Run the server to ensure `ALTER TABLE` logic in `src/database/mod.rs` applies the new columns correctly.
|
|
- [ ] **Schema Verification:** Verify `model_configs` has `cache_read_cost_per_m` and `cache_write_cost_per_m` columns.
|
|
|
|
## 6.4 Token Estimation Refinement (@lab-assistant)
|
|
- [ ] **Analyze Heuristic:** Review `chars / 4` in `src/utils/tokens.rs`.
|
|
- [ ] **Background Precise Recount:** Propose a mechanism for a precise token count (using Tiktoken) after the response is finalized.
|
|
|
|
## Critical Path
|
|
Migration Validation → UI Fields → Provider Stream Usage Reporting.
|
|
|
|
```mermaid
|
|
gantt
|
|
title Phase 6 Timeline
|
|
dateFormat YYYY-MM-DD
|
|
section Frontend
|
|
Models Page UI :2026-03-06, 1d
|
|
Costs Table Update:after Models Page UI, 1d
|
|
section Backend
|
|
DeepSeek Fix :2026-03-06, 1d
|
|
Provider Audit (Grok/Gemini):after DeepSeek Fix, 2d
|
|
section Database
|
|
Migration Test :2026-03-06, 1d
|
|
section Optimization
|
|
Token Heuristic Review :2026-03-06, 1d
|
|
```
|
|
|