Infrastructure as Code · AWS READY TO APPLY

LANGFUSE × EKS
FARGATE 部署方案

使用 Terraform 在 AWS 一键部署 Langfuse 可观测性平台。全 Serverless —— EKS 控制面托管 + Fargate 无节点计算,零 EC2 实例,Helm Chart 交付。

geoffrey@us8g: ~/terraform-aws-langfuse
13
Terraform 模块文件
87+
AWS 资源对象
26
可调参数变量
0
EC2 节点
3
可用区部署
~35m
首次 Apply 耗时
01

方案概述Overview

项目说明
编排引擎Amazon EKS (Kubernetes 1.32)
计算模式Fargate (Serverless Pod,无 EC2)
部署方式Helm Chart langfuse-k8s v1.5.14
数据库Aurora PostgreSQL Serverless v2
缓存ElastiCache Redis 7.1 (TLS)
分析引擎ClickHouse 集群内 StatefulSet + Keeper
对象存储S3 (IRSA 免 AK/SK)
持久化EFS (ClickHouse 数据卷,CSI 驱动)
入口ALB Ingress (可选 HTTPS + Route53 域名)
预计耗时首次 apply 约 25–35 分钟
月费估算~$175–240 (最小测试配置)
02

整体架构System Topology

FIG.1 — TOPOLOGY
graph TB User["👤 用户浏览器"] subgraph VPC["VPC 10.0.0.0/16"] subgraph PUB["Public Subnets · 3 AZ"] ALB["Application Load Balancer
HTTP:80 / HTTPS:443"] NAT["NAT Gateway ×1"] end subgraph PRIV["Private Subnets · 3 AZ"] subgraph EKS["EKS Cluster · Fargate"] WEB["langfuse-web Pod ×1"] WORKER["langfuse-worker Pod ×1"] CH["ClickHouse Shard ×2
+ Keeper ×2"] LBCTL["ALB Controller / EFS CSI"] end RDS[("Aurora PostgreSQL
Serverless v2")] REDIS[("ElastiCache
Redis 7.1")] EFS[/"EFS 文件系统
ClickHouse 数据"/] end end S3[("S3 Bucket
events / exports / media")] User -->|HTTPS| ALB ALB -->|target-type ip| WEB WEB --> RDS WEB --> REDIS WEB --> CH WORKER --> RDS WORKER --> S3 CH --> EFS NAT -->|出站| User style ALB fill:#fff3dd,stroke:#d97e00,stroke-width:2px style EKS fill:#e8f0f8,stroke:#10345c,stroke-width:2px style RDS fill:#e8f0f8,stroke:#10345c style REDIS fill:#e8f0f8,stroke:#10345c style S3 fill:#fff3dd,stroke:#d97e00
03

网络架构Network Layout

FIG.2 — NETWORK
graph LR IGW["Internet Gateway"] subgraph VPC["VPC 10.0.0.0/16"] subgraph A["AZ-a"] PUB1["Public
10.0.48.0/24"] PRIV1["Private
10.0.0.0/20"] end subgraph B["AZ-b"] PUB2["Public
10.0.49.0/24"] PRIV2["Private
10.0.16.0/20"] end subgraph C["AZ-c"] PUB3["Public
10.0.50.0/24"] PRIV3["Private
10.0.32.0/20"] end NAT["NAT Gateway ×1
(省钱模式)"] VPCE["VPC Endpoints
S3 Gateway + STS Interface"] end IGW --- PUB1 IGW --- PUB2 IGW --- PUB3 NAT --- PRIV1 NAT --- PRIV2 NAT --- PRIV3 VPCE -.-> PRIV1 style NAT fill:#fff3dd,stroke:#d97e00,stroke-width:2px style VPCE fill:#e8f0f8,stroke:#10345c,stroke-dasharray:4 3

私有子网承载全部有状态服务(EKS Pod / RDS / Redis / EFS),经 NAT 出公网拉取镜像;S3 走 Gateway Endpoint 不出公网,STS 走 Interface Endpoint 支撑 IRSA 令牌交换。

04

数据流架构Data Flow

FIG.3 — SEQUENCE
sequenceDiagram participant SDK as Langfuse SDK participant ALB as ALB Ingress participant Web as langfuse-web participant Worker as langfuse-worker participant PG as Aurora PG participant CH as ClickHouse participant Redis as ElastiCache participant S3 as S3 Bucket SDK->>ALB: POST /api/public/ingestion ALB->>Web: Forward (ip mode) Web->>Redis: 写入事件队列 Web->>PG: trace 元数据 Web-->>SDK: 207 Multi-Status loop 异步消费 Worker->>Redis: 拉取事件 Worker->>CH: 批量写入分析数据 Worker->>S3: media / exports end Note over Web,CH: 查询链路 Web->>CH: 分析查询 traces/scores Web->>PG: 元数据 projects/users
05

IAM 权限模型IRSA Federation

FIG.4 — IRSA
graph TB subgraph SA["EKS ServiceAccounts"] SA1["langfuse:langfuse"] SA2["kube-system:aws-load-balancer-controller"] SA3["kube-system:efs-csi-controller-sa"] end subgraph ROLES["IAM Roles"] R1["langfuse
path /kubernetes/"] R2["aws-load-balancer-controller"] R3["langfuse-efs-csi"] R4["langfuse-eks (Cluster)"] R5["langfuse-fargate (Pod Exec)"] end subgraph POL["Policies"] P1["S3 读写 · 限定 bucket"] P2["ELB/EC2 完整权限"] P3["EFS Access Point 管理"] P4["AmazonEKSClusterPolicy
+ ServicePolicy"] P5["FargatePodExecutionRolePolicy"] end SA1 -->|AssumeRoleWithWebIdentity| R1 SA2 -->|AssumeRoleWithWebIdentity| R2 SA3 -->|AssumeRoleWithWebIdentity| R3 R1 --- P1 R2 --- P2 R3 --- P3 R4 --- P4 R5 --- P5 style SA fill:#e8f0f8,stroke:#10345c style ROLES fill:#fff9ef,stroke:#d97e00 style POL fill:#f2f5f8,stroke:#9db0c4

三个业务 ServiceAccount 通过 OIDC Web Identity 联合绑定各自 IAM Role,Pod 内零静态凭据。集群角色与 Fargate 执行角色为标准 AWS 托管策略。

06

文件结构说明File Map

tree
terraform-aws-langfuse-no-domain-test/
├── main.tf                     # 入口: provider 配置 + module 调用 + 参数覆盖
├── setup.sh                    # 部署后脚本: 自动修复 NEXTAUTH_URL
├── .terraform.lock.hcl         # Provider 版本锁定
└── module/
    ├── versions.tf             # Terraform / Provider 版本约束
    ├── variables.tf            # 所有可调参数 (26 个)
    ├── locals.tf               # 局部变量 (tag 命名)
    ├── outputs.tf              # 输出: cluster 信息 / ALB DNS / bucket
    ├── vpc.tf                  # VPC + Subnets + NAT + VPC Endpoints
    ├── eks.tf                  # EKS Cluster + Fargate Profile + OIDC
    ├── ingress.tf              # ALB Controller (IAM + Helm)
    ├── langfuse.tf             # Langfuse Helm values + K8s Secret
    ├── postgresql.tf           # Aurora PostgreSQL Serverless v2
    ├── redis.tf                # ElastiCache Redis 7.1
    ├── clickhouse.tf           # ClickHouse EFS PV + Access Points
    ├── s3.tf                   # S3 Bucket + IRSA Role + Lifecycle
    ├── efs.tf                  # EFS + CSI Driver + StorageClass
    └── tls-certificate.tf      # ACM + Route53 (enable_dns=true 时)
07

部署前置条件Prerequisites

7.1 工具要求

工具最低版本用途
Terraform>= 1.0IaC 编排
AWS CLIv2Provider 认证 + EKS token
kubectl>= 1.28K8s 操作 (setup.sh)
Helm>= 3.xHelm provider 依赖

7.2 AWS 权限面

执行 apply 的身份需要覆盖:EC2 · EKS · IAM · RDS · ElastiCache · S3 · EFS · ELB · CloudWatch Logs · STS,启用域名时另需 ACM + Route53

7.3 认证配置

bash
# 方式一: 环境变量
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_DEFAULT_REGION=us-east-1

# 方式二: profile
aws configure --profile langfuse-deploy
export AWS_PROFILE=langfuse-deploy
08

部署步骤Deploy Procedure

8.1 参数调整 — main.tf

hcl
provider "aws" {
  region = "us-east-1"   # ← 改为目标 Region
}

module "langfuse" {
  source = "./module"

  # ====== 域名 (二选一) ======
  domain     = "langfuse-test.local"   # 无域名模式 (测试推荐)
  enable_dns = false
  # domain   = "langfuse.yourcompany.com"  # 有域名模式 (生产)
  # enable_dns = true

  # ====== 规模配置 ======
  name                      = "langfuse"
  use_single_nat_gateway    = true      # true=省$32/月
  postgres_instance_count   = 1         # 2=生产 HA
  postgres_min_capacity     = 0.5       # ACU
  postgres_max_capacity     = 2.0
  cache_instance_count      = 1         # 2=生产
  clickhouse_replicas       = 2         # 最少 2
  langfuse_web_replicas     = 1
  langfuse_worker_replicas  = 1
  use_encryption_key        = false     # 生产建议 true

  # ====== 资源限制 ======
  langfuse_cpu              = "0.5"
  langfuse_memory           = "1Gi"
  clickhouse_cpu            = "0.5"
  clickhouse_memory         = "2Gi"
  clickhouse_keeper_cpu     = "0.25"
  clickhouse_keeper_memory  = "0.5Gi"
}

8.2 执行部署

bash
cd terraform-aws-langfuse-no-domain-test

terraform init                     # 初始化 provider
terraform plan -out=plan.tfplan    # 预览变更
terraform apply plan.tfplan        # 执行 (约 25-35 分钟)

8.3 部署后配置(无域名模式)

bash
bash setup.sh
# 1. aws eks update-kubeconfig 配置 kubectl
# 2. 等待 ALB DNS 就绪 (最长 5 分钟)
# 3. helm upgrade 设置正确的 NEXTAUTH_URL
# 4. 等待 web Pod Ready

8.4 验证

bash
terraform output alb_dns_url
# http://langfuse-xxxx.us-east-1.elb.amazonaws.com

kubectl get pods -n langfuse
# langfuse-web-xxx                1/1   Running
# langfuse-worker-xxx             1/1   Running
# langfuse-clickhouse-shard0-0-0  1/1   Running
# langfuse-clickhouse-shard0-1-0  1/1   Running
# langfuse-zookeeper-0            1/1   Running
# langfuse-zookeeper-1            1/1   Running

kubectl get ingress -n langfuse
# 浏览器打开 ALB URL → 注册管理员账号
09

生产环境加固建议Hardening

FIG.5 — TEST vs PROD
graph LR subgraph T["测试配置"] T1["单 NAT"] T2["PG ×1"] T3["Redis ×1"] T4["无加密 Key"] T5["HTTP Only"] T6["0.0.0.0/0"] end subgraph P["生产配置"] P1["NAT per-AZ"] P2["PG ×2 Multi-AZ"] P3["Redis ×2 + Multi-AZ + 静态加密"] P4["Encryption Key 开启"] P5["HTTPS + ACM + Route53"] P6["限定来源 CIDR"] P7["CH ×3 · Web/Worker ×2+"] end T1 -.升级.-> P1 T2 -.升级.-> P2 T3 -.升级.-> P3 T4 -.升级.-> P4 T5 -.升级.-> P5 T6 -.升级.-> P6 style T fill:#fdeeee,stroke:#d64545 style P fill:#e9f7f0,stroke:#0e9f6e

生产 main.tf 参考

hcl
module "langfuse" {
  source = "./module"

  domain     = "langfuse.yourcompany.com"
  enable_dns = true

  name                      = "langfuse"
  use_single_nat_gateway    = false     # 每 AZ 一个 NAT
  postgres_instance_count   = 2         # Multi-AZ
  postgres_min_capacity     = 1.0
  postgres_max_capacity     = 8.0
  cache_instance_count      = 2
  redis_multi_az            = true
  redis_at_rest_encryption  = true
  clickhouse_replicas       = 3
  langfuse_web_replicas     = 2
  langfuse_worker_replicas  = 2
  use_encryption_key        = true

  langfuse_cpu              = "2"
  langfuse_memory           = "4Gi"
  clickhouse_cpu            = "2"
  clickhouse_memory         = "8Gi"
  clickhouse_keeper_cpu     = "1"
  clickhouse_keeper_memory  = "2Gi"

  ingress_inbound_cidrs     = ["10.0.0.0/8", "172.16.0.0/12"]
}
10

新环境适配 ChecklistAdaptation

  • REGION — 修改 main.tf 中 provider "aws" 的 region
  • IAM 命名冲突 — 若目标账号已有 aws-load-balancer-controller Role,给 module/ingress.tf 中的 Role/Policy 名加前缀
  • STATE 后端 — 生产配置 S3 backend + DynamoDB 状态锁
  • VPC 复用 — 使用已有 VPC 时传入 vpc_id + private/public_subnet_ids
  • 域名 — 生产开启 enable_dns=true,确保 Route53 NS 已委托
  • CHART 版本 — 确认 langfuse_helm_chart_version 兼容目标 Langfuse 版本
  • K8S 版本 — kubernetes_version 默认 1.32,确认目标 region 支持
  • S3 命名 — bucket 名 = {domain去点}-{name},确保全局唯一
11

销毁Teardown

bash
terraform destroy                  # 全量销毁 (约 15-20 分钟)

# ⚠ EFS PV 为 Retain 策略,需手动清理
# ⚠ S3 开启了版本控制,先清空对象:
aws s3 rm s3://langfuse-test-local-langfuse --recursive
terraform destroy
⚠️EFS 数据卷 persistent_volume_reclaim_policy = Retain,destroy 不会删除实际数据;S3 bucket 启用 Versioning,必须先 --recursive 清空所有版本,否则 destroy 报 BucketNotEmpty。
12

费用明细Cost Breakdown

资源规格月费 (USD)
EKS Cluster控制面托管$73
Fargate (web + worker)0.5C / 1G ×2~$15
Fargate (ClickHouse)0.5C/2G ×2 + Keeper ×2~$25
NAT Gateway×1$32
Aurora Serverless v20.5 ACU 常驻~$43
ElastiCache Redist4g.small ×1~$25
EFSelastic throughput~$5–10
ALB×1~$20
S3少量数据<$1
合计≈ $240
💡EKS 控制面 $73/月为固定费用。短期测试建议用完即 terraform destroy;长期运行可评估换 EC2 节点组降本。
13

故障排查Troubleshooting

bash
# Pod 无法调度 (Fargate Profile 未就绪)
kubectl describe pod -n langfuse <pod-name>
aws eks list-fargate-profiles --cluster-name langfuse   # 状态应为 ACTIVE

# ALB 未创建
kubectl get ingress -n langfuse -o yaml
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller

# ClickHouse CrashLoop
kubectl logs -n langfuse langfuse-clickhouse-shard0-0-0
kubectl describe pv clickhouse-data-0                   # 检查 EFS 挂载

# 数据库连接失败
kubectl logs -n langfuse -l app.kubernetes.io/component=web | grep -iE "postgres|redis"
# 检查安全组是否放行 VPC CIDR

# NEXTAUTH 错误 (OAuth 回调失败)
helm get values langfuse -n langfuse | grep nextauth    # 须与实际访问 URL 一致