Infrastructure as Code · AWS READY TO APPLY
LANGFUSE × EKS
FARGATE 部署方案
使用 Terraform 在 AWS 一键部署 Langfuse 可观测性平台。全 Serverless —— EKS 控制面托管 + Fargate 无节点计算,零 EC2 实例,Helm Chart 交付。
13
Terraform 模块文件
87+
AWS 资源对象
26
可调参数变量
0
EC2 节点
3
可用区部署
~35m
首次 Apply 耗时
01
方案概述Overview
| 项目 | 说明 |
|---|---|
| 编排引擎 | Amazon EKS (Kubernetes 1.32) |
| 计算模式 | Fargate (Serverless Pod,无 EC2) |
| 部署方式 | Helm Chart langfuse-k8s v1.5.14 |
| 数据库 | Aurora PostgreSQL Serverless v2 |
| 缓存 | ElastiCache Redis 7.1 (TLS) |
| 分析引擎 | ClickHouse 集群内 StatefulSet + Keeper |
| 对象存储 | S3 (IRSA 免 AK/SK) |
| 持久化 | EFS (ClickHouse 数据卷,CSI 驱动) |
| 入口 | ALB Ingress (可选 HTTPS + Route53 域名) |
| 预计耗时 | 首次 apply 约 25–35 分钟 |
| 月费估算 | ~$175–240 (最小测试配置) |
02
整体架构System Topology
FIG.1 — TOPOLOGY
graph TB
User["👤 用户浏览器"]
subgraph VPC["VPC 10.0.0.0/16"]
subgraph PUB["Public Subnets · 3 AZ"]
ALB["Application Load Balancer
HTTP:80 / HTTPS:443"] NAT["NAT Gateway ×1"] end subgraph PRIV["Private Subnets · 3 AZ"] subgraph EKS["EKS Cluster · Fargate"] WEB["langfuse-web Pod ×1"] WORKER["langfuse-worker Pod ×1"] CH["ClickHouse Shard ×2
+ Keeper ×2"] LBCTL["ALB Controller / EFS CSI"] end RDS[("Aurora PostgreSQL
Serverless v2")] REDIS[("ElastiCache
Redis 7.1")] EFS[/"EFS 文件系统
ClickHouse 数据"/] end end S3[("S3 Bucket
events / exports / media")] User -->|HTTPS| ALB ALB -->|target-type ip| WEB WEB --> RDS WEB --> REDIS WEB --> CH WORKER --> RDS WORKER --> S3 CH --> EFS NAT -->|出站| User style ALB fill:#fff3dd,stroke:#d97e00,stroke-width:2px style EKS fill:#e8f0f8,stroke:#10345c,stroke-width:2px style RDS fill:#e8f0f8,stroke:#10345c style REDIS fill:#e8f0f8,stroke:#10345c style S3 fill:#fff3dd,stroke:#d97e00
HTTP:80 / HTTPS:443"] NAT["NAT Gateway ×1"] end subgraph PRIV["Private Subnets · 3 AZ"] subgraph EKS["EKS Cluster · Fargate"] WEB["langfuse-web Pod ×1"] WORKER["langfuse-worker Pod ×1"] CH["ClickHouse Shard ×2
+ Keeper ×2"] LBCTL["ALB Controller / EFS CSI"] end RDS[("Aurora PostgreSQL
Serverless v2")] REDIS[("ElastiCache
Redis 7.1")] EFS[/"EFS 文件系统
ClickHouse 数据"/] end end S3[("S3 Bucket
events / exports / media")] User -->|HTTPS| ALB ALB -->|target-type ip| WEB WEB --> RDS WEB --> REDIS WEB --> CH WORKER --> RDS WORKER --> S3 CH --> EFS NAT -->|出站| User style ALB fill:#fff3dd,stroke:#d97e00,stroke-width:2px style EKS fill:#e8f0f8,stroke:#10345c,stroke-width:2px style RDS fill:#e8f0f8,stroke:#10345c style REDIS fill:#e8f0f8,stroke:#10345c style S3 fill:#fff3dd,stroke:#d97e00
03
网络架构Network Layout
FIG.2 — NETWORK
graph LR
IGW["Internet Gateway"]
subgraph VPC["VPC 10.0.0.0/16"]
subgraph A["AZ-a"]
PUB1["Public
10.0.48.0/24"] PRIV1["Private
10.0.0.0/20"] end subgraph B["AZ-b"] PUB2["Public
10.0.49.0/24"] PRIV2["Private
10.0.16.0/20"] end subgraph C["AZ-c"] PUB3["Public
10.0.50.0/24"] PRIV3["Private
10.0.32.0/20"] end NAT["NAT Gateway ×1
(省钱模式)"] VPCE["VPC Endpoints
S3 Gateway + STS Interface"] end IGW --- PUB1 IGW --- PUB2 IGW --- PUB3 NAT --- PRIV1 NAT --- PRIV2 NAT --- PRIV3 VPCE -.-> PRIV1 style NAT fill:#fff3dd,stroke:#d97e00,stroke-width:2px style VPCE fill:#e8f0f8,stroke:#10345c,stroke-dasharray:4 3
10.0.48.0/24"] PRIV1["Private
10.0.0.0/20"] end subgraph B["AZ-b"] PUB2["Public
10.0.49.0/24"] PRIV2["Private
10.0.16.0/20"] end subgraph C["AZ-c"] PUB3["Public
10.0.50.0/24"] PRIV3["Private
10.0.32.0/20"] end NAT["NAT Gateway ×1
(省钱模式)"] VPCE["VPC Endpoints
S3 Gateway + STS Interface"] end IGW --- PUB1 IGW --- PUB2 IGW --- PUB3 NAT --- PRIV1 NAT --- PRIV2 NAT --- PRIV3 VPCE -.-> PRIV1 style NAT fill:#fff3dd,stroke:#d97e00,stroke-width:2px style VPCE fill:#e8f0f8,stroke:#10345c,stroke-dasharray:4 3
私有子网承载全部有状态服务(EKS Pod / RDS / Redis / EFS),经 NAT 出公网拉取镜像;S3 走 Gateway Endpoint 不出公网,STS 走 Interface Endpoint 支撑 IRSA 令牌交换。
04
数据流架构Data Flow
FIG.3 — SEQUENCE
sequenceDiagram
participant SDK as Langfuse SDK
participant ALB as ALB Ingress
participant Web as langfuse-web
participant Worker as langfuse-worker
participant PG as Aurora PG
participant CH as ClickHouse
participant Redis as ElastiCache
participant S3 as S3 Bucket
SDK->>ALB: POST /api/public/ingestion
ALB->>Web: Forward (ip mode)
Web->>Redis: 写入事件队列
Web->>PG: trace 元数据
Web-->>SDK: 207 Multi-Status
loop 异步消费
Worker->>Redis: 拉取事件
Worker->>CH: 批量写入分析数据
Worker->>S3: media / exports
end
Note over Web,CH: 查询链路
Web->>CH: 分析查询 traces/scores
Web->>PG: 元数据 projects/users
05
IAM 权限模型IRSA Federation
FIG.4 — IRSA
graph TB
subgraph SA["EKS ServiceAccounts"]
SA1["langfuse:langfuse"]
SA2["kube-system:aws-load-balancer-controller"]
SA3["kube-system:efs-csi-controller-sa"]
end
subgraph ROLES["IAM Roles"]
R1["langfuse
path /kubernetes/"] R2["aws-load-balancer-controller"] R3["langfuse-efs-csi"] R4["langfuse-eks (Cluster)"] R5["langfuse-fargate (Pod Exec)"] end subgraph POL["Policies"] P1["S3 读写 · 限定 bucket"] P2["ELB/EC2 完整权限"] P3["EFS Access Point 管理"] P4["AmazonEKSClusterPolicy
+ ServicePolicy"] P5["FargatePodExecutionRolePolicy"] end SA1 -->|AssumeRoleWithWebIdentity| R1 SA2 -->|AssumeRoleWithWebIdentity| R2 SA3 -->|AssumeRoleWithWebIdentity| R3 R1 --- P1 R2 --- P2 R3 --- P3 R4 --- P4 R5 --- P5 style SA fill:#e8f0f8,stroke:#10345c style ROLES fill:#fff9ef,stroke:#d97e00 style POL fill:#f2f5f8,stroke:#9db0c4
path /kubernetes/"] R2["aws-load-balancer-controller"] R3["langfuse-efs-csi"] R4["langfuse-eks (Cluster)"] R5["langfuse-fargate (Pod Exec)"] end subgraph POL["Policies"] P1["S3 读写 · 限定 bucket"] P2["ELB/EC2 完整权限"] P3["EFS Access Point 管理"] P4["AmazonEKSClusterPolicy
+ ServicePolicy"] P5["FargatePodExecutionRolePolicy"] end SA1 -->|AssumeRoleWithWebIdentity| R1 SA2 -->|AssumeRoleWithWebIdentity| R2 SA3 -->|AssumeRoleWithWebIdentity| R3 R1 --- P1 R2 --- P2 R3 --- P3 R4 --- P4 R5 --- P5 style SA fill:#e8f0f8,stroke:#10345c style ROLES fill:#fff9ef,stroke:#d97e00 style POL fill:#f2f5f8,stroke:#9db0c4
三个业务 ServiceAccount 通过 OIDC Web Identity 联合绑定各自 IAM Role,Pod 内零静态凭据。集群角色与 Fargate 执行角色为标准 AWS 托管策略。
06
文件结构说明File Map
terraform-aws-langfuse-no-domain-test/
├── main.tf # 入口: provider 配置 + module 调用 + 参数覆盖
├── setup.sh # 部署后脚本: 自动修复 NEXTAUTH_URL
├── .terraform.lock.hcl # Provider 版本锁定
└── module/
├── versions.tf # Terraform / Provider 版本约束
├── variables.tf # 所有可调参数 (26 个)
├── locals.tf # 局部变量 (tag 命名)
├── outputs.tf # 输出: cluster 信息 / ALB DNS / bucket
├── vpc.tf # VPC + Subnets + NAT + VPC Endpoints
├── eks.tf # EKS Cluster + Fargate Profile + OIDC
├── ingress.tf # ALB Controller (IAM + Helm)
├── langfuse.tf # Langfuse Helm values + K8s Secret
├── postgresql.tf # Aurora PostgreSQL Serverless v2
├── redis.tf # ElastiCache Redis 7.1
├── clickhouse.tf # ClickHouse EFS PV + Access Points
├── s3.tf # S3 Bucket + IRSA Role + Lifecycle
├── efs.tf # EFS + CSI Driver + StorageClass
└── tls-certificate.tf # ACM + Route53 (enable_dns=true 时)07
部署前置条件Prerequisites
7.1 工具要求
| 工具 | 最低版本 | 用途 |
|---|---|---|
| Terraform | >= 1.0 | IaC 编排 |
| AWS CLI | v2 | Provider 认证 + EKS token |
| kubectl | >= 1.28 | K8s 操作 (setup.sh) |
| Helm | >= 3.x | Helm provider 依赖 |
7.2 AWS 权限面
执行 apply 的身份需要覆盖:EC2 · EKS · IAM · RDS · ElastiCache · S3 · EFS · ELB · CloudWatch Logs · STS,启用域名时另需 ACM + Route53。
7.3 认证配置
# 方式一: 环境变量 export AWS_ACCESS_KEY_ID=AKIA... export AWS_SECRET_ACCESS_KEY=... export AWS_DEFAULT_REGION=us-east-1 # 方式二: profile aws configure --profile langfuse-deploy export AWS_PROFILE=langfuse-deploy
08
部署步骤Deploy Procedure
8.1 参数调整 — main.tf
provider "aws" {
region = "us-east-1" # ← 改为目标 Region
}
module "langfuse" {
source = "./module"
# ====== 域名 (二选一) ======
domain = "langfuse-test.local" # 无域名模式 (测试推荐)
enable_dns = false
# domain = "langfuse.yourcompany.com" # 有域名模式 (生产)
# enable_dns = true
# ====== 规模配置 ======
name = "langfuse"
use_single_nat_gateway = true # true=省$32/月
postgres_instance_count = 1 # 2=生产 HA
postgres_min_capacity = 0.5 # ACU
postgres_max_capacity = 2.0
cache_instance_count = 1 # 2=生产
clickhouse_replicas = 2 # 最少 2
langfuse_web_replicas = 1
langfuse_worker_replicas = 1
use_encryption_key = false # 生产建议 true
# ====== 资源限制 ======
langfuse_cpu = "0.5"
langfuse_memory = "1Gi"
clickhouse_cpu = "0.5"
clickhouse_memory = "2Gi"
clickhouse_keeper_cpu = "0.25"
clickhouse_keeper_memory = "0.5Gi"
}8.2 执行部署
cd terraform-aws-langfuse-no-domain-test terraform init # 初始化 provider terraform plan -out=plan.tfplan # 预览变更 terraform apply plan.tfplan # 执行 (约 25-35 分钟)
8.3 部署后配置(无域名模式)
bash setup.sh # 1. aws eks update-kubeconfig 配置 kubectl # 2. 等待 ALB DNS 就绪 (最长 5 分钟) # 3. helm upgrade 设置正确的 NEXTAUTH_URL # 4. 等待 web Pod Ready
8.4 验证
terraform output alb_dns_url # http://langfuse-xxxx.us-east-1.elb.amazonaws.com kubectl get pods -n langfuse # langfuse-web-xxx 1/1 Running # langfuse-worker-xxx 1/1 Running # langfuse-clickhouse-shard0-0-0 1/1 Running # langfuse-clickhouse-shard0-1-0 1/1 Running # langfuse-zookeeper-0 1/1 Running # langfuse-zookeeper-1 1/1 Running kubectl get ingress -n langfuse # 浏览器打开 ALB URL → 注册管理员账号
09
生产环境加固建议Hardening
FIG.5 — TEST vs PROD
graph LR
subgraph T["测试配置"]
T1["单 NAT"]
T2["PG ×1"]
T3["Redis ×1"]
T4["无加密 Key"]
T5["HTTP Only"]
T6["0.0.0.0/0"]
end
subgraph P["生产配置"]
P1["NAT per-AZ"]
P2["PG ×2 Multi-AZ"]
P3["Redis ×2 + Multi-AZ + 静态加密"]
P4["Encryption Key 开启"]
P5["HTTPS + ACM + Route53"]
P6["限定来源 CIDR"]
P7["CH ×3 · Web/Worker ×2+"]
end
T1 -.升级.-> P1
T2 -.升级.-> P2
T3 -.升级.-> P3
T4 -.升级.-> P4
T5 -.升级.-> P5
T6 -.升级.-> P6
style T fill:#fdeeee,stroke:#d64545
style P fill:#e9f7f0,stroke:#0e9f6e
生产 main.tf 参考
module "langfuse" {
source = "./module"
domain = "langfuse.yourcompany.com"
enable_dns = true
name = "langfuse"
use_single_nat_gateway = false # 每 AZ 一个 NAT
postgres_instance_count = 2 # Multi-AZ
postgres_min_capacity = 1.0
postgres_max_capacity = 8.0
cache_instance_count = 2
redis_multi_az = true
redis_at_rest_encryption = true
clickhouse_replicas = 3
langfuse_web_replicas = 2
langfuse_worker_replicas = 2
use_encryption_key = true
langfuse_cpu = "2"
langfuse_memory = "4Gi"
clickhouse_cpu = "2"
clickhouse_memory = "8Gi"
clickhouse_keeper_cpu = "1"
clickhouse_keeper_memory = "2Gi"
ingress_inbound_cidrs = ["10.0.0.0/8", "172.16.0.0/12"]
}10
新环境适配 ChecklistAdaptation
- REGION — 修改 main.tf 中 provider "aws" 的 region
- IAM 命名冲突 — 若目标账号已有 aws-load-balancer-controller Role,给 module/ingress.tf 中的 Role/Policy 名加前缀
- STATE 后端 — 生产配置 S3 backend + DynamoDB 状态锁
- VPC 复用 — 使用已有 VPC 时传入 vpc_id + private/public_subnet_ids
- 域名 — 生产开启 enable_dns=true,确保 Route53 NS 已委托
- CHART 版本 — 确认 langfuse_helm_chart_version 兼容目标 Langfuse 版本
- K8S 版本 — kubernetes_version 默认 1.32,确认目标 region 支持
- S3 命名 — bucket 名 = {domain去点}-{name},确保全局唯一
11
销毁Teardown
terraform destroy # 全量销毁 (约 15-20 分钟) # ⚠ EFS PV 为 Retain 策略,需手动清理 # ⚠ S3 开启了版本控制,先清空对象: aws s3 rm s3://langfuse-test-local-langfuse --recursive terraform destroy
⚠️EFS 数据卷
persistent_volume_reclaim_policy = Retain,destroy 不会删除实际数据;S3 bucket 启用 Versioning,必须先 --recursive 清空所有版本,否则 destroy 报 BucketNotEmpty。12
费用明细Cost Breakdown
| 资源 | 规格 | 月费 (USD) |
|---|---|---|
| EKS Cluster | 控制面托管 | $73 |
| Fargate (web + worker) | 0.5C / 1G ×2 | ~$15 |
| Fargate (ClickHouse) | 0.5C/2G ×2 + Keeper ×2 | ~$25 |
| NAT Gateway | ×1 | $32 |
| Aurora Serverless v2 | 0.5 ACU 常驻 | ~$43 |
| ElastiCache Redis | t4g.small ×1 | ~$25 |
| EFS | elastic throughput | ~$5–10 |
| ALB | ×1 | ~$20 |
| S3 | 少量数据 | <$1 |
| 合计 | ≈ $240 |
💡EKS 控制面 $73/月为固定费用。短期测试建议用完即
terraform destroy;长期运行可评估换 EC2 节点组降本。13
故障排查Troubleshooting
# Pod 无法调度 (Fargate Profile 未就绪) kubectl describe pod -n langfuse <pod-name> aws eks list-fargate-profiles --cluster-name langfuse # 状态应为 ACTIVE # ALB 未创建 kubectl get ingress -n langfuse -o yaml kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller # ClickHouse CrashLoop kubectl logs -n langfuse langfuse-clickhouse-shard0-0-0 kubectl describe pv clickhouse-data-0 # 检查 EFS 挂载 # 数据库连接失败 kubectl logs -n langfuse -l app.kubernetes.io/component=web | grep -iE "postgres|redis" # 检查安全组是否放行 VPC CIDR # NEXTAUTH 错误 (OAuth 回调失败) helm get values langfuse -n langfuse | grep nextauth # 须与实际访问 URL 一致