引言
最近听好多朋友说，cilium很强，势必将成为主流。因其使用了ebpf，性能好，而且支持网络策略。于是，决定花点时间学习一下。在通过官网文档学习过程中，发现使用cilium作为CNI，居然可以不用安装kube-proxy了。这让我想起来，之前在面试中被问到的一个问题，面试官问我:kube-proxy是否可以不用安装，是否有其他替代品。这下不就有答案了嘛。
顺便吐槽一下，看官方文档学习，是真的有点难(毕竟全英文);不过还是建议大家看官方文档学习，不要翻译成中文哦。那么接下来，就由我来实操一下。
环境说明
| 序号 | 事项 | 说明 |
|---|---|---|
| 1 | kubernetes version | v1.21.3 |
| 2 | cilium version | v1.10.3 |
| 3 | kubernetes安装方式 | kubeadm |
| 4 | cilium组网模式 | vxlan |
| 5 | os | ubuntu 18.04 |
| 6 | kubernetes集群规模 | 1master、2node |
正文
在master上初始化集群，并通过添加--skip-phases=addon/kube-proxy参数忽略kube-proxy的安装
- kubeadminit--apiserver-advertise-address=10.211.55.50--image-repositoryregistry.aliyuncs.com/google_containers--kubernetes-versionv1.21.3--service-cidr=10.96.0.0/12--pod-network-cidr=10.244.0.0/16--ignore-preflight-errors=all--skip-phases=addon/kube-proxy
在两个node上执行kubeadm join，加入集群
- kubeadmjoin10.211.55.50:6443--tokenouez6j.02ms269v8i4psl7p--discovery-token-ca-cert-hashsha256:5fdafe0fe1adb3b60cd7bc33f033f028279a94a3944816424cc7f5bb498f6868
使用helm(v3)来安装cilium。先添加cilium库
- helmrepoaddciliumhttps://helm.cilium.io/
使用如下命令安装cilium，添加kubeProxyReplacement=strict参数
- helminstallciliumcilium/cilium--version1.10.3--namespacekube-system--setkubeProxyReplacement=strict--setk8sServiceHost=10.211.55.50--setk8sServicePort=6443
检查cilium安装结果
- #查看ciliumagent,以daemonset方式部署在每个node节点上
- root@cilium1:/#kubectl-nkube-systemgetpods-lk8s-app=cilium
- NAMEREADYSTATUSRESTARTSAGE
- cilium-8gwg21/1Running08m4s
- cilium-t9ffc1/1Running08m39s
- cilium-x42r61/1Running08m16s
- #查看cilumoperator
- root@cilium1:~#kubectlgetpo-A-owide|grepcilium-operator
- kube-systemcilium-operator-5df88875-867hd1/1Running541h172.16.88.47cilium3<none><none>
- kube-systemcilium-operator-5df88875-9kx8c1/1Running541h172.16.88.253cilium2<none><none>
检查是否有kube-proxy组件。可以发现并没有该组件
- root@cilium1:/#kubectlgetpo-nkube-system
- NAMEREADYSTATUSRESTARTSAGE
- cilium-8gwg21/1Running010m
- cilium-operator-5df88875-867hd1/1Running527h
- cilium-operator-5df88875-9kx8c1/1Running527h
- cilium-t9ffc1/1Running011m
- cilium-x42r61/1Running010m
- coredns-59d64cd4d4-hbwg41/1Running127h
- coredns-59d64cd4d4-l2pmt1/1Running127h
- etcd-cilium11/1Running227h
- kube-apiserver-cilium11/1Running227h
- kube-controller-manager-cilium11/1Running227h
- kube-scheduler-cilium11/1Running227h
检查cilium状态，确保安装正确
- root@cilium1:/#kubectlexec-nkube-systemcilium-t9ffc--ciliumstatus
- Defaultedcontainer"cilium-agent"outof:cilium-agent,mount-cgroup(init),clean-cilium-state(init)
- KVStore:OkDisabled
- Kubernetes:Ok1.21(v1.21.3)[linux/amd64]
- KubernetesAPIs:["cilium/v2::CiliumClusterwideNetworkPolicy","cilium/v2::CiliumEndpoint","cilium/v2::CiliumNetworkPolicy","cilium/v2::CiliumNode","core/v1::Namespace","core/v1::Node","core/v1::Pods","core/v1::Service","discovery/v1::EndpointSlice","networking.k8s.io/v1::NetworkPolicy"]
- KubeProxyReplacement:Strict[eth010.211.55.50(DirectRouting)]
- Cilium:Ok1.10.3(v1.10.3-4145278)
- NodeMonitor:Listeningforeventson8CPUswith64x4096ofsharedmemory
- Ciliumhealthdaemon:Ok
- IPAM:IPv4:2/254allocatedfrom10.0.0.0/24,
- BandwidthManager:Disabled
- HostRouting:Legacy
- Masquerading:BPF[eth0]10.0.0.0/24[IPv4:Enabled,IPv6:Disabled]
- ControllerStatus:20/20healthy
- ProxyStatus:OK,ip10.0.0.41,0redirectsactiveonports10000-20000
- Hubble:OkCurrent/MaxFlows:817/4095(19.95%),Flows/s:0.95Metrics:Disabled
- Encryption:Disabled
- Clusterhealth:3/3reachable(2021-08-07T15:29:05Z)
部署nginx来测试一下网络联通性
- #nginxdeploymentyaml文件
- catdeployment-nginx.yaml
- apiVersion:apps/v1
- kind:Deployment
- metadata:
- name:nginx
- spec:
- selector:
- matchLabels:
- run:nginx
- replicas:4
- template:
- metadata:
- labels:
- run:nginx
- spec:
- containers:
- -name:nginx
- image:nginx
- ports:
- -containerPort:80
- #创建nginxdeployment
- kubectlcreate-fdeployment-nginx.yaml
- #查看部署结果
- root@cilium1:/#kubectlgetpo-owide
- NAMEREADYSTATUSRESTARTSAGEIPNODENOMINATEDNODEREADINESSGATES
- nginx-649c4b9857-8f2v51/1Running126h10.0.2.212cilium2<none><none>
- nginx-649c4b9857-mhsxs1/1Running126h10.0.1.23cilium3<none><none>
- nginx-649c4b9857-qw2jj1/1Running126h10.0.2.69cilium2<none><none>
- nginx-649c4b9857-vj9w21/1Running126h10.0.1.126cilium3
创建一个nodeport service来验证service的可访问
- #创建service
- kubectlexposedeploymentnginx--type=NodePort--port=80
- #查看service
- root@cilium1:/#kubectlgetsvcnginx
- NAMETYPECLUSTER-IPEXTERNAL-IPPORT(S)AGE
- nginxNodePort10.97.209.103<none>80:31126/TCP26h
验证nodeport、cluster可访问
- #通过nodeport
- root@cilium1:/#curl127.0.0.1:31126
- <!DOCTYPEhtml>
- <html>
- <head>
- <title>Welcometonginx!</title>
- #通过service:port
- root@cilium1:/#curl10.97.209.103
- <!DOCTYPEhtml>
- <html>
- <head>
- <title>Welcometonginx!</title>
- #检查iptables发现为空
- root@cilium1:/#iptables-save|grepKUBE-SVC
- root@cilium1:/#
- #检查ciliunservice
- root@cilium1:/#kubectlexec-nkube-systemcilium-t9ffc--ciliumservicelist
- Defaultedcontainer"cilium-agent"outof:cilium-agent,mount-cgroup(init),clean-cilium-state(init)
- IDFrontendServiceTypeBackend
- 110.96.0.1:443ClusterIP1=>172.16.88.57:6443
- 210.96.0.10:9153ClusterIP1=>10.0.2.229:9153
- 2=>10.0.2.80:9153
- 310.96.0.10:53ClusterIP1=>10.0.2.229:53
- 2=>10.0.2.80:53
- 410.97.209.103:80ClusterIP1=>10.0.2.69:80
- 2=>10.0.1.23:80
- 3=>10.0.1.126:80
- 4=>10.0.2.212:80
- 5172.16.88.57:31126NodePort1=>10.0.2.69:80
- 2=>10.0.1.23:80
- 3=>10.0.1.126:80
- 4=>10.0.2.212:80
- 60.0.0.0:31126NodePort1=>10.0.2.69:80
- 2=>10.0.1.23:80
- 3=>10.0.1.126:80
- 4=>10.0.2.212:80
从上面的安装和测试结果来，虽然我们没有安装k8s的kube-proxy组件，但是集群依然正常。说明kube-proxy组件确实是可以被替代的。
总结
以上虽然完成了kubernetes without kube-proxy的搭建和测试工作，但还是有很多事情没说明。比如使用cilium的系统要求、cilium是什么、有几种组网模式、网络策略。不过请不要着急，期待我后续的文章。
参考
https://docs.cilium.io/en/v1.10/gettingstarted/kubeproxy-free/#kubernetes-without-kube-proxy
https://kubernetes.io/docs/concepts/cluster-administration/addons/
https://helm.sh/docs/intro/install/
原文链接：https://mp.weixin.qq.com/s/riGA59hkViC8HR1CFbuFHA
如果您对该产品感兴趣，请填写办理（客服微信：xiaoxiongyidong）
发表评论
◎欢迎参与讨论，请在这里发表您的看法、交流您的观点。