Hermes CLI + litellm でカスタムエンドポイントを設定する方法を紹介しました
Hermes CLI 自体は Google AI Studio に対応しているので使えます
設定しているカスタムエンドポイント側の gemini が function calling や web_search に対応していなかったりリクエストやレスポンスの形式が少し異なる場合にはそのままでは使えません
services:litellm:image: docker.litellm.ai/berriai/litellm:latest
container_name: litellm
environment:- AI_SERVICE_API_KEY=${AI_SERVICE_API_KEY}ports:-"4000:4000"volumes:- ./litellm_config.yaml:/app/config.yaml
restart: unless-stopped
command:["--config","/app/config.yaml"]gemini-proxy:# Python proxy for gemini CLI (native Gemini API format).# Replaces nginx because request body JSON transformation is required:# strips unsupported 'id' field from function_call/function_response,# rewrites :streamGenerateContent?alt=sse → :streamGenerateContentSse,# and injects AI_SERVICE_API_KEY as x-goog-api-key header.# Configure gemini CLI to use port 4001 instead of 4000.build:context: .
dockerfile: Dockerfile.proxy
container_name: gemini-proxy
environment:- AI_SERVICE_API_KEY=${AI_SERVICE_API_KEY}ports:-"4001:4001"restart: unless-stopped
proxy.py
API を互換させるためのプロキシです
先ほども紹介しましたが custom endpoint が完全互換の場合は不要です
#!/usr/bin/env python3"""
Proxy for gemini CLI → Your custom Gemini endpoint.
Handles:
- URL mapping: :streamGenerateContent?alt=sse → :streamGenerateContentSse
- Request body: strip unsupported 'id' field from function_call / function_response
- Auth: replace incoming key with AI_SERVICE_API_KEY as x-goog-api-key header
"""import json
import os
import httpx
from fastapi import FastAPI, Request
from fastapi.responses import Response, StreamingResponse
CUSTOM_BASE = os.environ.get("CUSTOM_BASE","https://your-gemini-custom-endpoint",)
API_KEY = os.environ["AI_SERVICE_API_KEY"]
app = FastAPI()defstrip_unsupported_fields(body:dict)->dict:"""Strip 'id' from function_call/function_response — field not supported by this endpoint."""for content in body.get("contents",[]):for part in content.get("parts",[]):for key in("functionCall","function_call"):ifisinstance(part.get(key),dict):
part[key].pop("id",None)for key in("functionResponse","function_response"):ifisinstance(part.get(key),dict):
part[key].pop("id",None)return body
@app.post("/v1beta/models/{model}:{method}")asyncdefproxy(request: Request, model:str, method:str):
is_streaming = method =="streamGenerateContent"# Custom endpoint uses a dedicated SSE endpoint instead of ?alt=sse query param
target_method ="streamGenerateContentSse"if is_streaming else method
target_url = f"{CUSTOM_BASE}/{model}:{target_method}"
body_bytes =await request.body()try:
body = strip_unsupported_fields(json.loads(body_bytes))
body_bytes = json.dumps(body, ensure_ascii=False).encode()except(json.JSONDecodeError, AttributeError):pass
upstream_headers ={"Content-Type":"application/json","x-goog-api-key": API_KEY}if is_streaming:asyncdefgenerate():asyncwith httpx.AsyncClient(timeout=120.0)as client:asyncwith client.stream("POST", target_url, content=body_bytes, headers=upstream_headers
)as resp:asyncfor chunk in resp.aiter_bytes():yield chunk
return StreamingResponse(
generate(),
media_type="text/event-stream",
headers={"Cache-Control":"no-cache","X-Accel-Buffering":"no"},)else:asyncwith httpx.AsyncClient(timeout=120.0)as client:
resp =await client.post(
target_url, content=body_bytes, headers=upstream_headers
)return Response(
content=resp.content,
status_code=resp.status_code,
media_type=resp.headers.get("content-type","application/json"),)
Dockerfile.proxy
先程の proxy.py をビルドしてイメージを作成する dockerfile です
FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir fastapi httpx uvicorn
COPY proxy.py .
CMD["uvicorn","proxy:app","--host","0.0.0.0","--port","4001"]
model_list:-model_name: gemini-3.5-flash
litellm_params:# gemini/ prefix is required to use API key auth instead of Vertex AI ADC.# Model name must match model_name above to avoid LiteLLM response model override error# that corrupts streaming responses. drop_params: true handles unsupported fields.model: gemini/gemini-3.5-flash
api_base: https://your-gemini-custom-endpoint:generateContent
api_key: os.environ/AI_SERVICE_API_KEY
max_tokens:4096temperature:0.2rpm:30tpm:60000-model_name: gemini-3.1-flash-lite
litellm_params:# gemini/ prefix is required to use API key auth instead of Vertex AI ADC.model: gemini/gemini-3.5-flash
api_base: https://your-gemini-custom-endpoint:generateContent
api_key: os.environ/AI_SERVICE_API_KEY
litellm_settings:drop_params:truegeneral_settings:global_max_parallel_requests:1max_parallel_requests:1# Pass-through: forward native Gemini API requests (/v1beta/models/...) directly# to the upstream endpoint, bypassing LiteLLM's model routing and SDK layer.# This avoids the "cannot override response model" bug caused by GenerateContentResponse# not having a 'model' attribute in LiteLLM's google_genai provider path.pass_through_endpoints:-path:"/v1beta/models/{path:path}"target:"https://your-gemini-custom-endpoint/gemini/{path}"headers:x-goog-api-key: os.environ/AI_SERVICE_API_KEY
router_settings:num_retries:0retry_after:20allowed_fails:1cooldown_time:180timeout:120stream_timeout:120server_settings:port:4000
Unfortunately, I **cannot execute any commands** in this environment — the sandbox is rejecting all shell execution with:
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
This means I cannot:
- Read the actual file contents
- Apply patches
- Run tests
Could you check if there's a sandbox configuration issue? Once commands can run, I can implement the fix directly. Alternatively, if you can share the contents of the two files, I can produce the exact patch needed.
Codex's Linux sandbox uses bubblewrap and needs access to create user namespaces.
func(c *BaseController)CurrentUsername()string{if name := c.GetSession("user"); name !=nil{if username, ok := name.(string); ok {return username
}}return"anonymous"}
