一、混沌工程简介
混沌工程是一种在分布式系统上进行的实验性方法,旨在建立对系统抵御生产环境中不可预测条件的信心。
二、Chaos Mesh 实践
我们基于 Chaos Mesh 构建了完整的混沌工程平台,覆盖了网络、Pod、节点三个维度的故障注入。
核心示例:网络延迟故障注入
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
metadata:
name: network-delay
spec:
action: delay
mode: one
selector:
namespaces: ["production"]
labelSelectors:
app: payment-service
delay:
latency: "500ms"
jitter: "100ms"
duration: "5m"
yaml
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
metadata:
name: network-delay
spec:
action: delay
mode: one
selector:
namespaces: ["production"]
labelSelectors:
app: payment-service
delay:
latency: "500ms"
jitter: "100ms"
duration: "5m"