hobokenchicken/GopherGate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add cache token tracking and cache-aware cost calculation
Some checks failed
CI / Check (push) Has been cancelled
Details
CI / Clippy (push) Has been cancelled
Details
CI / Formatting (push) Has been cancelled
Details
CI / Test (push) Has been cancelled
Details
CI / Release Build (push) Has been cancelled
Details

Browse Source 
Track cache_read_tokens and cache_write_tokens end-to-end: parse from
provider responses (OpenAI, DeepSeek, Grok, Gemini), persist to SQLite,
apply cache-aware pricing from the model registry, and surface in API
responses and the dashboard.

- Add cache fields to ProviderResponse, StreamUsage, RequestLog structs
- Parse cached_tokens (OpenAI/Grok), prompt_cache_hit/miss (DeepSeek),
  cachedContentTokenCount (Gemini) from provider responses
- Send stream_options.include_usage for streaming; capture real usage
  from final SSE chunk in AggregatingStream
- ALTER TABLE migration for cache_read_tokens/cache_write_tokens columns
- Cache-aware cost formula using registry cache_read/cache_write rates
- Update Provider trait calculate_cost signature across all providers
- Add cache_read_tokens/cache_write_tokens to Usage API response
- Dashboard: cache hit rate card, cache columns in pricing and usage
  tables, cache token aggregation in SQL queries
- Remove API debug panel and verbose console logging from api.js
- Bump static asset cache-bust to v5
This commit is contained in:
Dustin Newkirk
2026-03-02 14:45:21 -05:00
parent 232f092f27
commit db5824f0fb
19 changed files with 352 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
static/index.html
View File

@@ -4,7 +4,7 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>LLM Proxy Gateway - Admin Dashboard</title>
<link rel="stylesheet" href="/css/dashboard.css?v=4">
<link rel="stylesheet" href="/css/dashboard.css?v=5">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css">
<link rel="icon" href="img/logo-icon.png" type="image/png" sizes="any">
<link rel="apple-touch-icon" href="img/logo-icon.png">
@@ -166,19 +166,19 @@
</div>
<!-- Scripts (cache-busted with version query params) -->
<script src="/js/api.js?v=4"></script>
<script src="/js/auth.js?v=4"></script>
<script src="/js/dashboard.js?v=4"></script>
<script src="/js/websocket.js?v=4"></script>
<script src="/js/charts.js?v=4"></script>
<script src="/js/pages/overview.js?v=4"></script>
<script src="/js/pages/analytics.js?v=4"></script>
<script src="/js/pages/costs.js?v=4"></script>
<script src="/js/pages/clients.js?v=4"></script>
<script src="/js/pages/providers.js?v=4"></script>
<script src="/js/pages/models.js?v=4"></script>
<script src="/js/pages/monitoring.js?v=4"></script>
<script src="/js/pages/settings.js?v=4"></script>
<script src="/js/pages/logs.js?v=4"></script>
<script src="/js/api.js?v=5"></script>
<script src="/js/auth.js?v=5"></script>
<script src="/js/dashboard.js?v=5"></script>
<script src="/js/websocket.js?v=5"></script>
<script src="/js/charts.js?v=5"></script>
<script src="/js/pages/overview.js?v=5"></script>
<script src="/js/pages/analytics.js?v=5"></script>
<script src="/js/pages/costs.js?v=5"></script>
<script src="/js/pages/clients.js?v=5"></script>
<script src="/js/pages/providers.js?v=5"></script>
<script src="/js/pages/models.js?v=5"></script>
<script src="/js/pages/monitoring.js?v=5"></script>
<script src="/js/pages/settings.js?v=5"></script>
<script src="/js/pages/logs.js?v=5"></script>
</body>
</html>