K8S Gateway AB测试、蓝绿发布、金丝雀(灰度)发布
假设有如下三个节点的 K8S 集群：
k8s31master 是控制节点
k8s31node1、k8s31node2 是工作节点
容器运行时是 containerd
一、场景分析
阅读本文，默认您已经安装了 K8S Gateway。
关于 AB 测试、金丝雀发布，可以看这篇文章。
二、实验准备
-
镜像下载
# 在各个工作节点下载
[root@k8s31node1 ~]# ctr -n=k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest
[root@k8s31node1 ~]# ctr -n=k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest docker.io/openresty/openresty:latest[root@k8s31node2 ~]# ctr -n=k8s.io images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest
[root@k8s31node2 ~]# ctr -n=k8s.io images tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/openresty/openresty:latest docker.io/openresty/openresty:latest
-
部署 v1
apiVersion: apps/v1
kind: Deployment
metadata:name: nginx-v1
spec:replicas: 1selector:matchLabels:app: nginxversion: v1template:metadata:labels:app: nginxversion: v1spec:containers:- name: nginximage: "openresty/openresty:latest"imagePullPolicy: IfNotPresentports:- name: httpprotocol: TCPcontainerPort: 80volumeMounts:- mountPath: /usr/local/openresty/nginx/conf/nginx.confname: configsubPath: nginx.confvolumes:- name: configconfigMap:name: nginx-v1
---
apiVersion: v1
kind: ConfigMap
metadata:labels:app: nginxversion: v1name: nginx-v1
data:nginx.conf: |-worker_processes 1;events {accept_mutex on;multi_accept on;use epoll;worker_connections 1024;}http {ignore_invalid_headers off;server {listen 80;location / {access_by_lua 'local header_str = ngx.say("nginx-v1")';}}}
---
apiVersion: v1
kind: Service
metadata:name: nginx-v1
spec:type: ClusterIPports:- port: 80protocol: TCPname: httpselector:app: nginxversion: v1该 yml 定义了三个资源 ConfigMap、Deployment、Service。
- ConfigMap 定义了一个 nginx.conf 配置文件，使用 lua 脚本输出 nginx-v1。
- Deployment 定义了一个 Pod，里面运行 openresty 它是一个封装了 nginx+lua 的 web 服务器。Pod 有两个标签 app: nginx、version: v1。
- Service 代理了 Deployment 运行的 Pod。
部署 v2
apiVersion: apps/v1
kind: Deployment
metadata:name: nginx-v2
spec:replicas: 1selector:matchLabels:app: nginxversion: v2template:metadata:labels:app: nginxversion: v2spec:containers:- name: nginximage: "openresty/openresty:latest"imagePullPolicy: IfNotPresentports:- name: httpprotocol: TCPcontainerPort: 80volumeMounts:- mountPath: /usr/local/openresty/nginx/conf/nginx.confname: configsubPath: nginx.confvolumes:- name: configconfigMap:name: nginx-v2
---
apiVersion: v1
kind: ConfigMap
metadata:labels:app: nginxversion: v2name: nginx-v2
data:nginx.conf: |-worker_processes 1;events {accept_mutex on;multi_accept on;use epoll;worker_connections 1024;}http {ignore_invalid_headers off;server {listen 80;location / {access_by_lua 'local header_str = ngx.say("nginx-v2")';}}}
---
apiVersion: v1
kind: Service
metadata:name: nginx-v2
spec:type: ClusterIPports:- port: 80protocol: TCPname: httpselector:app: nginxversion: v2
三、AB 测试
1）创建 gateway
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:name: nginx-gatewaynamespace: default
spec:gatewayClassName: nginxlisteners:- name: httpprotocol: HTTPport: 802）创建 httproute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:name: abtest-routenamespace: default
spec:parentRefs:- name: nginx-gatewayhostnames:- "abtest.example.com"rules:- matches:- path:type: PathPrefixvalue: / headers:- name: "version"value: "v1"backendRefs:- name: nginx-v1kind: Serviceport: 80- matches:- path:type: PathPrefixvalue: /headers:- name: "version"value: "v2" backendRefs:- name: nginx-v2kind: Serviceport: 80
- parentRefs：绑定我们新建的 gateway。
- hostnames：定义访问的主机名。
- rules.matches：定义路由规则，PathPrefix 表示路径前缀匹配。headers 对请求头进行匹配。
- backendRefs：定义后端服务以及服务端口。
3）测试
curl -H "Host: abtest.example.com" -H "version: v1" http://192.168.40.20:30185/
curl -H "Host: abtest.example.com" -H "version: v2" http://192.168.40.20:30185/
30185 为 nginx-gateway-controller 80 映射端口。
四、金丝雀发布
1）创建 httproute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:name: canary-routenamespace: default
spec:parentRefs:- name: nginx-gatewayhostnames:- "canary.example.com"rules:- matches:- path:type: PathPrefixvalue: / backendRefs:- name: nginx-v1kind: Serviceport: 80weight: 10- name: nginx-v2kind: Serviceport: 80weight: 90
backendRefs.weight 定义流量分发的权重。
2）测试
for i in {1..20}; do curl -H "Host: canary.example.com" http://192.168.40.20:30185/; done;