title: “自托管 GenAI 策略” weight: 20
一切都运行在 Amazon EKS 上。没有托管的 AI 服务，只有 pods、Helm charts 和一些 Python。
EKS Cluster (Auto Mode)
├── Inferentia Node Pool
│ └── vLLM (Qwen2.5-3B on Neuron)
├── litellm namespace
│ └── LiteLLM proxy (shared model plane for both tracks)
├── General Purpose Nodes
│ ├── Strands Agent
│ ├── Langfuse (Observability)
│ ├── Milvus (Vector DB / Memory)
│ ├── Neo4j (Knowledge Graph)
│ └── MCP Server (Agent Tools)
└── Services
├── litellm.litellm:4000 (OpenAI-compatible proxy， the one URL agents talk to)
├── qwen2-5-3b-neuron.vllm:8000 (upstream backend for LiteLLM)
├── langfuse:3000
├── milvus:19530
└── neo4j:7687