当前位置：首页 > news >正文

Cilium动手实验室: 精通之旅---22.Cilium Traffic Optimization

news 2025/6/13 11:26:50

Cilium动手实验室: 精通之旅---22.Cilium Traffic Optimization

1. Lab环境
2. 业务流量分配
- 2.1 搜索 Droid
- 2.2 业务流量分配
- 2.3 思考
- 2.4 小测验
3. 本地重定向策略
- 3.1 机器人藏在哪里？
- 3.2 创建 Cilium 本地重定向策略
4. 本地节点DNS 缓存
- 4.1 本地节点DNS缓存
- 4.2 用于节点本地 DNS 缓存的 LRP
- 4.3 小测验
5. 最终实验
- 5.1 考题
- 5.2 解题

1. Lab环境

Lab环境地址

https://isovalent.com/labs/cilium-traffic-optimization/

我们将使用 Kind 来设置我们的 Kubernetes 集群，并在该 Cilium 之上。

看看它的配置：

root@server:~# yq cluster.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:ServiceTrafficDistribution: true
nodes:- role: control-plane- role: workerlabels:topology.kubernetes.io/zone: port- role: workerlabels:topology.kubernetes.io/zone: starboard
networking:disableDefaultCNI: truekubeProxyMode: none

请注意：

设置为 featureGates.ServiceTrafficDistribution true 以启用此功能，该功能将在下一个挑战中探索。
networking.disableDefaultCNI 标志从 Kind 集群中删除默认 CNI，我们安装了 Cilium。
networking.kubeProxyMode 标志在 Kind 集群上禁用了 KubeProxy。Cilium 将扮演这个角色。

在 nodes 部分中，您可以看到集群由 3 个节点组成：

1 个运行 Kubernetes 控制平面和 etcd 的control-plane节点
2 个 worker 节点用于部署应用程序

root@server:~# k get nodes
NAME                 STATUS   ROLES           AGE     VERSION
kind-control-plane   Ready    control-plane   3m46s   v1.31.0
kind-worker          Ready    <none>          3m31s   v1.31.0
kind-worker2         Ready    <none>          3m30s   v1.31.0

在实验室启动期间使用以下命令预安装了 Cilium：

cilium install \--version 1.17.1 \--namespace kube-system \--set kubeProxyReplacement=true \--set localRedirectPolicy=true \--set loadBalancer.serviceTopology=true

以下命令将等待 Cilium 启动并运行并报告其状态：

root@server:~# cilium status --wait/¯¯\/¯¯\__/¯¯\    Cilium:             OK\__/¯¯\__/    Operator:           OK/¯¯\__/¯¯\    Envoy DaemonSet:    OK\__/¯¯\__/    Hubble Relay:       disabled\__/       ClusterMesh:        disabledDaemonSet              cilium                   Desired: 3, Ready: 3/3, Available: 3/3
DaemonSet              cilium-envoy             Desired: 3, Ready: 3/3, Available: 3/3
Deployment             cilium-operator          Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium                   Running: 3cilium-envoy             Running: 3cilium-operator          Running: 1clustermesh-apiserver    hubble-relay             
Cluster Pods:          3/3 managed by Cilium
Helm chart version:    1.17.1
Image versions         cilium             quay.io/cilium/cilium:v1.17.1@sha256:8969bfd9c87cbea91e40665f8ebe327268c99d844ca26d7d12165de07f702866: 3cilium-envoy       quay.io/cilium/cilium-envoy:v1.31.5-1739264036-958bef243c6c66fcfd73ca319f2eb49fff1eb2ae@sha256:fc708bd36973d306412b2e50c924cd8333de67e0167802c9b48506f9d772f521: 3cilium-operator    quay.io/cilium/operator-generic:v1.17.1@sha256:628becaeb3e4742a1c36c4897721092375891b58bae2bfcae48bbf4420aaee97: 1

在本实验中，我们将在 tatooine 命名空间中部署多个 pod：

c3-po 和 r2-d2，代表两个 droid，分别调度在 kind-worker 和 kind-worker2 节点上
obi-wan 和 luke，代表两个 Jedis
一个 stormtroopers DaemonSet 部署每个节点上一个 stormtrooper pod。

确认配置文件内容:

root@server:~# yq tatooine.yaml
apiVersion: v1
kind: Pod
metadata:name: c3-ponamespace: landspeederlabels:name: c3-porole: droid
spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: topology.kubernetes.io/zoneoperator: Invalues: [starboard]containers:- name: nginximage: nginx:latestvolumeMounts:- name: c3-po-config-volumemountPath: /etc/nginx/conf.dvolumes:- name: c3-po-config-volumeconfigMap:name: c3-po-config
---
apiVersion: v1
kind: Pod
metadata:name: r2-d2namespace: landspeederlabels:name: r2-d2role: droid
spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: topology.kubernetes.io/zoneoperator: Invalues: [port]containers:- name: nginximage: nginx:latestvolumeMounts:- name: r2-d2-config-volumemountPath: /etc/nginx/conf.dvolumes:- name: r2-d2-config-volumeconfigMap:name: r2-d2-config
---
apiVersion: v1
kind: Pod
metadata:name: obi-wannamespace: landspeederlabels:name: obi-wanrole: jedi
spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: topology.kubernetes.io/zoneoperator: Invalues: [port]containers:- name: nginximage: nginx:latestvolumeMounts:- name: obi-wan-config-volumemountPath: /etc/nginx/conf.dvolumes:- name: obi-wan-config-volumeconfigMap:name: obi-wan-config
---
apiVersion: v1
kind: Pod
metadata:name: lukenamespace: landspeederlabels:name: lukerole: jedi
spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: topology.kubernetes.io/zoneoperator: Invalues: [starboard]containers:- name: nginximage: nginx:latestvolumeMounts:- name: luke-config-volumemountPath: /etc/nginx/conf.dvolumes:- name: luke-config-volumeconfigMap:name: luke-config
---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: stormtroopersnamespace: spaceport
spec:selector:matchLabels:name: stormtroopertemplate:metadata:labels:name: stormtrooperrole: soldierspec:containers:- name: nginximage: nginx:latestports:- containerPort: 80
---
apiVersion: v1
kind: ConfigMap
metadata:name: c3-po-confignamespace: landspeeder
data:default.conf: |server {listen 80;location / {add_header X-Identification "C3-PO";return 200 'Greetings, Human! I am C-3PO, human-cyborg relations. Fluent in over six million forms of communication, and at your service!\n';}}---
apiVersion: v1
kind: ConfigMap
metadata:name: r2-d2-confignamespace: landspeeder
data:default.conf: |server {listen 80;location / {add_header X-Identification "R2-D2";return 200 'Beep beep boop boop! *whistles*\n';}}---
apiVersion: v1
kind: ConfigMap
metadata:name: obi-wan-confignamespace: landspeeder
data:default.conf: "server {\n  listen 80;\n  add_header X-Identification \"Obi-Wan\" always;\n  add_header X-Saber-Color \"blue \\" always; \n\n  location / {\n    return 404 'These are not the droids you are looking for.\\n';\n  }\n}\n"
---
apiVersion: v1
kind: ConfigMap
metadata:name: luke-confignamespace: landspeeder
data:default.conf: "server {\n  listen 80;\n  add_header X-Identification \"Luke\" always;\n  add_header X-Saber-Color \"green \\" always; \n\n  location / {\n    return 404 'Like Obi-Wan said \\\n';\n  }\n}\n"

部署配置

root@server:~# kubectl apply -f tatooine.yaml
pod/c3-po created
pod/r2-d2 created
pod/obi-wan created
pod/luke created
daemonset.apps/stormtroopers created
configmap/c3-po-config created
configmap/r2-d2-config created
configmap/obi-wan-config created
configmap/luke-config created

确认部署正确

root@server:~# kubectl get -f tatooine.yaml -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP             NODE           NOMINATED NODE   READINESS GATES
pod/c3-po     1/1     Running   0          70s   10.244.2.38    kind-worker2   <none>           <none>
pod/r2-d2     1/1     Running   0          70s   10.244.1.230   kind-worker    <none>           <none>
pod/obi-wan   1/1     Running   0          70s   10.244.1.196   kind-worker    <none>           <none>
pod/luke      1/1     Running   0          70s   10.244.2.116   kind-worker2   <none>           <none>NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE   CONTAINERS   IMAGES         SELECTOR
daemonset.apps/stormtroopers   2         2         2       2            2           <none>          70s   nginx        nginx:latest   name=stormtrooperNAME                       DATA   AGE
configmap/c3-po-config     1      70s
configmap/r2-d2-config     1      70s
configmap/obi-wan-config   1      70s
configmap/luke-config      1      70s

2. 业务流量分配

2.1 搜索 Droid

让我们在 landspeeder 命名空间中部署一个 droids 服务：

root@server:~# kubectl -n landspeeder apply -f droids.yaml -o yaml
apiVersion: v1
kind: Service
metadata:annotations:kubectl.kubernetes.io/last-applied-configuration: |{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"droids","namespace":"landspeeder"},"spec":{"ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"role":"jedi"},"type":"ClusterIP"}}creationTimestamp: "2025-06-05T02:23:27Z"name: droidsnamespace: landspeederresourceVersion: "1780"uid: 60bb78b3-b4ba-45b8-b4a8-ebb8a4c67551
spec:clusterIP: 10.96.56.89clusterIPs:- 10.96.56.89internalTrafficPolicy: ClusteripFamilies:- IPv4ipFamilyPolicy: SingleStackports:- port: 80protocol: TCPtargetPort: 80selector:role: jedisessionAffinity: Nonetype: ClusterIP
status:loadBalancer: {}

该服务充当 L4 负载均衡器（由 Cilium 的 kube-proxy 替换实现），将流量发送到标记为 role=jedi 的 pod。

请添加图片描述

分别找到在 kind-worker 和 kind-worker2 上运行的 stormtrooper pod：

root@server:~# TROOPER1=$(kubectl -n spaceport get po -l role=soldier --field-selector spec.nodeName=kind-worker -o name)
echo $TROOPER1
TROOPER2=$(kubectl -n spaceport get po -l role=soldier --field-selector spec.nodeName=kind-worker2 -o name)
echo $TROOPER2
pod/stormtroopers-gb9lr
pod/stormtroopers-bt7xs

通过运行几次来测试服务：

root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:24:38 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:24:47 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:24:49 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:24:52 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.

您将看到来自 obi-wan 和 luke pod 的 HTTP 404 响应（检查响应标头的值以识别后端）。

也可以使用第二个 worker 节点进行尝试：

root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:25:30 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:25:33 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:25:36 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷

它始终是Luke进行响应

2.2 业务流量分配

目前，唯一支持的值是 PreferClose，它指示将流量路由到拓扑上靠近客户端的终端节点的首选项。

通过将流量保持在本地扩展区内，用户可以优化性能、成本和可靠性。

在 Kubernetes 1.30 中引入，可以直接在服务规范中启用服务流量分配（而不是像拓扑感知提示和路由那样使用注释）。

我们集群中的 worker 节点已经用众所周知的 topology.kubernetes.io/zone 标签进行了标记，以便指定它们的拓扑。

检查节点标签：

root@server:~# kubectl get no --show-labels | \GREP_COLORS='mt=1;32' grep --color=always -E '[^=]+$' | \grep --color=always topology.kubernetes.io/zone=
kind-worker          Ready    <none>          13m   v1.31.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-worker,kubernetes.io/os=linux,topology.kubernetes.io/zone=port
kind-worker2         Ready    <none>          13m   v1.31.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-worker2,kubernetes.io/os=linux,topology.kubernetes.io/zone=starboard

kind-worker 节点标记为 port （即飞机的左侧），而 kind-worker2 设置为 rightboard （飞机的右侧）。

landspeeder （飞机）命名空间中的 Pod 已指定为与 topology.kubernetes.io/zone 标签的 port 或 starboard 值的节点关联性，模拟它们在多个可用区中的部署。

检查 r2-d2 pod 的 affinity：

root@server:~# kubectl -n landspeeder get pod r2-d2 -o yaml | yq .spec.affinity
nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: topology.kubernetes.io/zoneoperator: Invalues:- port

修改配置文件

root@server:~# yq droids.yaml 
---
apiVersion: v1
kind: Service
metadata:name: droidsnamespace: landspeeder
spec:trafficDistribution: PreferCloseselector:role: jediports:- protocol: TCPport: 80targetPort: 80type: ClusterIP

应用配置

kubectl -n landspeeder apply -f droids.yaml

现在，从第一个 worker 上的 Pod 对 droids 服务进行查询：

root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:31:26 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:31:28 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:31:31 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.

多次重复该命令。所有回复都应来自 Obi-Wan（在飞机左侧）。

尝试使用第二个 worker：

root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:33:16 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:33:18 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:33:21 GMT
Content-Type: application/octet-stream
Content-Length: 23
Connection: keep-alive
X-Identification: Luke
X-Saber-Color: green 🟢Like Obi-Wan said 🤷

现在，所有响应都应该来自 Luke（在飞机的右侧），因为 Cilium 正在应用拓扑亲和性来根据调用方选择服务后端。

请添加图片描述

删除 luke pod：

kubectl -n landspeeder delete pod luke

然后再次尝试从第二个 worker 访问 droids 服务：

root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:34:36 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:34:38 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 404 Not Found
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:34:41 GMT
Content-Type: application/octet-stream
Content-Length: 46
Connection: keep-alive
X-Identification: Obi-Wan
X-Saber-Color: blue 🔵These are not the droids you are looking for.

流量现在流向 Obi-Wan，因为在第二个 trooper pod 的拓扑附近没有可用的后端来实现 droids 服务。

2.3 思考

借助服务流量分配，您现在可以避免这些令人讨厌的可用区间数据费用，并减少应用程序的延迟。

但是，服务流量分配并不是我们在本实验中要强调的唯一流量工程功能。

正如服务流量分配有助于将流量保持在区域内一样，本地重定向策略（LRP）通过将服务的流量边界定向到本地后端来阻止流量离开节点。

2.4 小测验

√	Traffic Distribution requires Cilium's eBPF-based Kube Proxy Replacement.
√	Traffic Distribution requires Kubernetes 1.30
×	Traffic Distribution supports values `PreferClose` and `PreferRemote`
×	Pods need to be annotated with `networking.kubernetes.io/traffic-distribution` in order to enlist.

3. 本地重定向策略

本地重定向策略最初是在 Cilium 1.9 中引入的，通常被希望优化网络性能和减少延迟的用户采用。

本地重定向策略（LRP）允许使用 eBPF 将发往 IP 地址和端口/协议元组或 Kubernetes 服务的 Pod 流量重定向到本地节点内的后端 Pod。

首先，我们将考虑一个简单的示例，其中本地重定向策略将重定向发往 Kubernetes 服务的流量，并强制流量仅转发到本地后端（例如，在与源客户端相同的 Kubernetes 节点上）。

当我们展开方案时，将提供图表以帮助您理解概念。

3.1 机器人藏在哪里？

冲锋队并不满意。他们很确定欧比旺在隐瞒什么，并打算检查飞机上的机器人。

让我们检查当前 droids 服务的状态。首先，获取服务集群 IP：

root@server:~# kubectl -n landspeeder get svc droids
NAME     TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
droids   ClusterIP   10.96.56.89   <none>        80/TCP    16m

让我们在第一个 worker 节点上获取 Cilium 代理：

root@server:~# CILIUM1=$(kubectl -n kube-system get po -l k8s-app=cilium --field-selector spec.nodeName=kind-worker -o name)
echo $CILIUM1
pod/cilium-wrnjg

接下来，在该节点上验证 Cilium 的 eBPF kube-proxy 替换是否为此 IP 创建了 ClusterIP 服务条目：

root@server:~# kubectl exec -it -n kube-system $CILIUM1 -c cilium-agent -- \cilium-dbg service list | \grep -E '^ID|10.96.56.89:80' | \GREP_COLORS='mt=01;31' grep --color=always -B1 '10.96.56.89:80' | \GREP_COLORS='mt=01;32:sl=01;33' grep --color=always -B1 -E 'LocalRedirect|ClusterIP'
ID   Frontend                Service Type   Backend                               
6    10.96.56.89:80/TCP      ClusterIP      1 => 10.244.1.196:80/TCP (active)

你可以看到，它目前的行为就像一个标准的 ClusterIP 服务，指向一个 IP（黄色的，这是 obi-wan Pod，因为 luke Pod 已被删除）。

验证后端 IP 是否与 obi-wan Pod 的 IP 相对应：

root@server:~# kubectl -n landspeeder get pod obi-wan -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
obi-wan   1/1     Running   0          19m   10.244.1.196   kind-worker   <none>           <none>

3.2 创建 Cilium 本地重定向策略

本地重定向策略有两种类型：

ServiceMatcher （我们现在和下一个任务中将看到它）
AddressMatcher 中（您将在最终挑战中看到）。

使用 ServiceMatcher 时，发往特定服务的流量将仅重定向到节点本地后端 Pod。

当应用此类型的策略时，由 Cilium 的 eBPF kube-proxy 替换创建的现有服务条目将被 LocalRedirect 类型的新服务条目替换。此条目可能只有节点本地后端 Pod。

应用 CiliumLocalRedirectPolicy，它将覆盖 droids 服务，将其指向实际的 droid pods：

root@server:~# kubectl apply -f droids-lrp.yaml -o yaml | yq
apiVersion: cilium.io/v2
kind: CiliumLocalRedirectPolicy
metadata:name: droidsnamespace: landspeeder
spec:redirectFrontend:serviceMatcher:serviceName: droidsnamespace: landspeederredirectBackend:localEndpointSelector:matchLabels:role: droidtoPorts:- port: "80"protocol: TCP

你可以看到，这将导致发往服务 droid 的流量被重定向到带有 role=droid 标签的本地后端。

请添加图片描述

再次验证 Cilium 的 eBPF 服务表：

root@server:~# kubectl exec -it -n kube-system $CILIUM1 -c cilium-agent -- \cilium-dbg service list | \grep -E '^ID|10.96.56.89:80' | \GREP_COLORS='mt=01;31' grep --color=always -B1 '10.96.56.89:80' | \GREP_COLORS='mt=01;32:sl=01;33' grep --color=always -B1 -E 'LocalRedirect|ClusterIP'
ID   Frontend                Service Type    Backend                               
6    10.96.56.89:80/TCP      LocalRedirect   1 => 10.244.1.230:80/TCP (active)

该服务现在被标记为指向单个 IP 的本地重定向，该 IP 是标记为 role=droid 的本地 Pod 的 IP。验证这一点：

root@server:~# kubectl -n landspeeder get po -l role=droid --field-selector spec.nodeName=kind-worker -o wide
NAME    READY   STATUS    RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
r2-d2   1/1     Running   0          23m   10.244.1.230   kind-worker   <none>           <none>

让我们再次获取每个节点的 troopers：

root@server:~# TROOPER1=$(kubectl -n spaceport get po -l role=soldier --field-selector spec.nodeName=kind-worker -o name)
echo $TROOPER1
TROOPER2=$(kubectl -n spaceport get po -l role=soldier --field-selector spec.nodeName=kind-worker2 -o name)
echo $TROOPER2
pod/stormtroopers-gb9lr
pod/stormtroopers-bt7xs

然后，从第一个 worker 访问 droids 服务：

root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:45:32 GMT
Content-Type: application/octet-stream
Content-Length: 32
Connection: keep-alive
X-Identification: R2-D2Beep beep boop boop! *whistles*
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:45:41 GMT
Content-Type: application/octet-stream
Content-Length: 32
Connection: keep-alive
X-Identification: R2-D2Beep beep boop boop! *whistles*
root@server:~# kubectl -n spaceport exec $TROOPER1 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:45:44 GMT
Content-Type: application/octet-stream
Content-Length: 32
Connection: keep-alive
X-Identification: R2-D2Beep beep boop boop! *whistles*

现在所有响应都来自 R2-D2，因为它是第一个 worker 上标记为 role=droid 的 pod。

在第二个 worker 上尝试：

root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:46:15 GMT
Content-Type: application/octet-stream
Content-Length: 126
Connection: keep-alive
X-Identification: C3-POGreetings, Human! I am C-3PO, human-cyborg relations. Fluent in over six million forms of communication, and at your service!
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:46:18 GMT
Content-Type: application/octet-stream
Content-Length: 126
Connection: keep-alive
X-Identification: C3-POGreetings, Human! I am C-3PO, human-cyborg relations. Fluent in over six million forms of communication, and at your service!
root@server:~# kubectl -n spaceport exec $TROOPER2 -- \curl -si droids.landspeeder.svc.cluster.local
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 05 Jun 2025 02:46:20 GMT
Content-Type: application/octet-stream
Content-Length: 126
Connection: keep-alive
X-Identification: C3-POGreetings, Human! I am C-3PO, human-cyborg relations. Fluent in over six million forms of communication, and at your service!

所有响应都来自 C3-PO，即第二个节点的本地 droid pod。

4. 本地节点DNS 缓存

4.1 本地节点DNS缓存

在使用 LRP 重定向 DNS 流量和优化延迟之前，我们首先需要创建一个本地 DNS 节点缓存。

让我们回顾一下 DNS 节点缓存配置：

root@server:~# yq node-local-dns.yaml
---
apiVersion: "cilium.io/v2"
apiVersion: v1
kind: ServiceAccount
metadata:name: node-local-dnsnamespace: kube-system
---
apiVersion: v1
kind: Service
metadata:name: kube-dns-upstreamnamespace: kube-systemlabels:k8s-app: kube-dnskubernetes.io/name: "KubeDNSUpstream"
spec:ports:- name: dnsport: 53protocol: UDPtargetPort: 53- name: dns-tcpport: 53protocol: TCPtargetPort: 53selector:k8s-app: kube-dns
---
apiVersion: v1
kind: ConfigMap
metadata:name: node-local-dnsnamespace: kube-system
data:Corefile: |cluster.local:53 {errorscache {success 9984 30denial 9984 5}reloadloopbind 0.0.0.0forward . __PILLAR__CLUSTER__DNS__ {force_tcp}prometheus :9253health}in-addr.arpa:53 {errorscache 30reloadloopbind 0.0.0.0forward . __PILLAR__CLUSTER__DNS__ {force_tcp}prometheus :9253}ip6.arpa:53 {errorscache 30reloadloopbind 0.0.0.0forward . __PILLAR__CLUSTER__DNS__ {force_tcp}prometheus :9253}.:53 {errorscache 30reloadloopbind 0.0.0.0forward . __PILLAR__UPSTREAM__SERVERS__prometheus :9253}---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: node-local-dnsnamespace: kube-systemlabels:k8s-app: node-local-dns
spec:updateStrategy:rollingUpdate:maxUnavailable: 10%selector:matchLabels:k8s-app: node-local-dnstemplate:metadata:labels:k8s-app: node-local-dnsannotations:policy.cilium.io/no-track-port: "53"prometheus.io/port: "9253"prometheus.io/scrape: "true"spec:priorityClassName: system-node-criticalserviceAccountName: node-local-dnsdnsPolicy: Default # Don't use cluster DNS.tolerations:- key: "CriticalAddonsOnly"operator: "Exists"- effect: "NoExecute"operator: "Exists"- effect: "NoSchedule"operator: "Exists"containers:- name: node-cacheimage: registry.k8s.io/dns/k8s-dns-node-cache:1.15.16resources:requests:cpu: 25mmemory: 5Miargs: ["-localip", "169.254.20.10,10.96.0.10", "-conf", "/etc/Corefile", "-upstreamsvc", "kube-dns-upstream", "-skipteardown=true", "-setupinterface=false", "-setupiptables=false"]ports:- containerPort: 53name: dnsprotocol: UDP- containerPort: 53name: dns-tcpprotocol: TCP- containerPort: 9253name: metricsprotocol: TCPlivenessProbe:httpGet:path: /healthport: 8080initialDelaySeconds: 60timeoutSeconds: 5volumeMounts:- name: config-volumemountPath: /etc/coredns- name: kube-dns-configmountPath: /etc/kube-dnsvolumes:- name: kube-dns-configconfigMap:name: kube-dnsoptional: true- name: config-volumeconfigMap:name: node-local-dnsitems:- key: Corefilepath: Corefile.base

让我们总结一些核心组件：

名为 node-local-dns 的 ServiceAccount 将被 DaemonSet 用于在集群中以特定权限运行。
名为 kube-dns-upstream 的服务在端口 53 上侦听 UDP 和 TCP 上的 DNS 查询，并将其转发到由 k8s-app： kube-dns 标签选择的 DNS pod（coreDNS pods）上的目标端口。当节点本地 DNS 缓存无法解析请求时，此服务用于处理上游 DNS 查询。
ConfigMap 定义节点级别 DNS 缓存的 CoreDNS 配置（Corefile）。
- 它定义了 cluster.local 域（Kubernetes 服务发现）和外部 DNS 查询（cluster.local 外部）中的 DNS 请求的行为。
- 缓存配置了 cluster.local 和反向查找区域（in-addr.arpa 和 ip6.arpa）的成功和拒绝限制
- Prometheus 的端口 9253 上公开指标，并启用运行状况检查。
名为 node-local-dns 的 DaemonSet 在集群中的每个节点上运行一个 node-cache 容器。\
- 它确保每个节点都有一个本地 DNS 缓存服务，从而显著减少节点上工作负载的 DNS 查找时间。
- dnsPolicy： Default 确保 DaemonSet 不使用集群的默认 DNS 设置，而是依赖于自己的 DNS 配置。
- args 定义缓存的本地 IP 地址（169.254.20.10 和 10.96.0.10），这些地址是此服务使用的节点本地 IP。
- 它为 DNS（TCP 和 UDP）公开了端口 53，为指标公开了端口 9253。
- 运行状况检查通过 /health 端点进行配置。

部署 NodeLocal Daemonset 和关联的组件：

kubectl apply -f node-local-dns.yaml

4.2 用于节点本地 DNS 缓存的 LRP

查看本地重定向策略：

root@server:~# yq node-local-dns-lrp.yaml
---
apiVersion: "cilium.io/v2"
kind: CiliumLocalRedirectPolicy
metadata:name: "nodelocaldns"namespace: kube-system
spec:redirectFrontend:serviceMatcher:serviceName: kube-dnsnamespace: kube-systemredirectBackend:localEndpointSelector:matchLabels:k8s-app: node-local-dnstoPorts:- port: "53"name: dnsprotocol: UDP- port: "53"name: dns-tcpprotocol: TCP

它将重定向绑定到 CoreDNS 服务器的流量，并将其转发到节点本地 DNS 缓存代理。

现在让我们部署策略：

kubectl apply -f node-local-dns-lrp.yaml

DNS 流量现在将首先转到本地 node-cache。

您可以通过检查 DNS 缓存的指标来验证。

让我们从 Obi Wan Pod（在第一个 Worker 节点上）发出 DNS 请求，并检查本地 DNS 缓存和另一个 Worker 节点上运行的 DNS 缓存上的 DNS 请求计数。

首先，让我们获取第一个 worker 节点（Obi Wan 本地）上 DNS 缓存 Pod 的 IP：

root@server:~# DNS_LOCAL=$(kubectl -n kube-system get po -l k8s-app=node-local-dns --field-selector spec.nodeName=kind-worker -o jsonpath='{.items[].status.podIP}')
echo $DNS_LOCAL
10.244.1.100

我们还来获取 ‘non-local’ DNS 条目的 IP：

root@server:~# DNS_NON_LOCAL=$(kubectl -n kube-system get po -l k8s-app=node-local-dns --field-selector spec.nodeName=kind-worker2 -o jsonpath='{.items[].status.podIP}')
echo $DNS_NON_LOCAL
10.244.2.92

我们来检查一下这两个 DNS 条目的指标。

让我们从客户端 Pod 运行一些 DNS 命令：

root@server:~# echo "## Stats on Local DNS Cache ##"
kubectl -n landspeeder exec obi-wan -- curl $DNS_LOCAL:9253/metrics | grep coredns_dns_request_count_total
echo "## Stats on Non-Local DNS Cache ##"
kubectl -n landspeeder exec obi-wan -- curl $DNS_NON_LOCAL:9253/metrics | grep coredns_dns_request_count_total
## Stats on Local DNS Cache ##% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP coredns_dns_request_count_total Counter of DNS requests made per zone, protocol and family.
# TYPE coredns_dns_request_count_total counter
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="cluster.local."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="in-addr.arpa."} 10
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="ip6.arpa."} 10
100 37453    0 37453    0     0  17.3M      0 --:--:-- --:--:-- --:--:-- 17.8M
## Stats on Non-Local DNS Cache ##% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP coredns_dns_request_count_total Counter of DNS requests made per zone, protocol and family.
# TYPE coredns_dns_request_count_total counter
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="cluster.local."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="in-addr.arpa."} 10
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="ip6.arpa."} 10
100 37450    0 37450    0     0  13.6M      0 --:--:-- --:--:-- --:--:-- 17.8M

让我们运行一些 DNS 请求：

root@server:~# kubectl -n landspeeder exec obi-wan -- getent hosts swapi.dev
kubectl -n landspeeder exec obi-wan -- getent hosts isovalent.com
kubectl -n landspeeder exec obi-wan -- getent hosts cilium.io
52.58.110.120   swapi.dev
75.2.60.5       isovalent.com
104.198.14.52   cilium.io

现在验证它们是否仅在本地 VM 上增加：

root@server:~# echo "## Stats on Local DNS Cache ##"
kubectl -n landspeeder exec obi-wan -- curl $DNS_LOCAL:9253/metrics | grep coredns_dns_request_count_total
echo "## Stats on Non-Local DNS Cache ##"
kubectl -n landspeeder exec obi-wan -- curl $DNS_NON_LOCAL:9253/metrics | grep coredns_dns_request_count_total
## Stats on Local DNS Cache ##% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100 38144    0 38144    0     0  21.0M      0 --:--:-- --:--:-- --:--:-- 36.3M
# HELP coredns_dns_request_count_total Counter of DNS requests made per zone, protocol and family.
# TYPE coredns_dns_request_count_total counter
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="."} 19
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="cluster.local."} 25
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="in-addr.arpa."} 10
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="ip6.arpa."} 10
## Stats on Non-Local DNS Cache ##% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100 37443    0 37443    0     0  13.9M      0 --:--:-- --:--:-- --:--:-- 17.8M
# HELP coredns_dns_request_count_total Counter of DNS requests made per zone, protocol and family.
# TYPE coredns_dns_request_count_total counter
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="cluster.local."} 1
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="in-addr.arpa."} 10
coredns_dns_request_count_total{family="1",proto="udp",server="dns://0.0.0.0:53",zone="ip6.arpa."} 10

4.3 小测验

×	Local Redirect Policy requires Cilium's eBPF-based Kube Proxy Replacement.
√	Local Redirect Policy uses annotations to select the backend pods the traffic is redirected to.
√	The most popular use for Local Redirect Policy is to optimize DNS traffic and reduce latency.
×	Local Redirect Policy leverages iptables to redirect the traffic.

5. 最终实验

5.1 考题

如本实验前面所述，实际上有两种类型的 LocalRedirectPolicy：基于服务（LRP 重定向发往 Kubernetes 服务的流量）和基于 IP 地址的（LRP 重定向发往特定 IP 地址的流量）。

后一种使用案例不太常见，但在某些情况下很有用：云提供商（例如 EKS、GKE）通常有一个 DaemonSet，它运行节点本地缓存来为元数据请求提供服务，并将缓存未命中发送到在链接本地地址上运行的外部元数据服务器。

此类使用案例的 LRP 将描述如何将与前端地址匹配的流量重定向到节点本地缓存。 169.254.169.254 是云环境中通常用作元数据服务端点的 IP。

在最后一项挑战中，我们要求您拦截绑定到 IP 169.254.169.254 和端口 8081 的流量，该端口连接到后端 Pod（在此任务开始时部署），标签为 app： proxy-exam。

已为您预先创建了一个 lrp-addrmatcher-exam.yaml 文件。
更新缺少的参数。
应用它。您可以通过从现有 obi-wan Pod 对所需的元数据 IP/端口运行 curl -si 来验证它是否成功（假设您的配置正确，您应该会收到 200 OK 作为响应）。

5.2 解题

确认pod端口

k get po lrp-pod-exam -o yaml |grep port
.... 略....
# 获得以下内容containerPort: 80

根据题意写入内容将已知内容填入lrp-addrmatcher-exam.yaml

apiVersion: "cilium.io/v2"
kind: CiliumLocalRedirectPolicy
metadata:name: "lrp-addr-exam"
spec:redirectFrontend:addressMatcher:ip: "169.254.169.254"toPorts:- port: "8081"protocol: TCPredirectBackend:localEndpointSelector:matchLabels:app: proxy-examtoPorts:- port: "80"protocol: TCP