Helm搭建etcd集群

1. 概述

这篇文章是使用 Helm 安装 Etcd 的记录，本次安装我使用的是 Bitnami 维护的仓库，详细的安装参见官方说明 .

安装前先说明下当前的环境，规划一下安装方案。

节点	角色	IP	配置	Label
master	master, etcd	192.168.1.100	4核4G50G	usefulness=schedule
node1	worker	192.168.1.101	8核32G100G	usefulness=devops
node2	worker	192.168.1.102	8核12G100G	usefulness=business
node3	worker	192.168.1.103	8核12G100G	usefulness=business

同 Kafka 与 ZooKeeper 一样，Etcd 也安装在 node1 节点上，因此安装的时候需要对 Chart 默认的参数进行简单修改。

2. 安装

添加 Bitnami 仓库

helm repo add bitnami https://charts.bitnami.com/bitnami

覆盖默认参数

auth:
  rbac:
    enabled: true
    allowNoneAuthentication: false
    rootPassword: "90CjPHPRlxw="
replicaCount: 3

安装

helm install etcd bitnami/etcd -n devops -f values.yaml

安装结果

NAME: etcd
LAST DEPLOYED: Sat Dec 11 16:15:37 2021
NAMESPACE: devops
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: etcd
CHART VERSION: 6.10.5
APP VERSION: 3.5.1

** Please be patient while the chart is being deployed **

etcd can be accessed via port 2379 on the following DNS name from within your cluster:

    etcd.devops.svc.cluster.local

To create a pod that you can use as a etcd client run the following command:

    kubectl run etcd-client --restart='Never' --image docker.io/bitnami/etcd:3.5.1-debian-10-r31 --env ROOT_PASSWORD=$(kubectl get secret --namespace devops etcd -o jsonpath="{.data.etcd-root-password}" | base64 --decode) --env ETCDCTL_ENDPOINTS="etcd.devops.svc.cluster.local:2379" --namespace devops --command -- sleep infinity

Then, you can set/get a key using the commands below:

    kubectl exec --namespace devops -it etcd-client -- bash
    etcdctl --user root:$ROOT_PASSWORD put /message Hello
    etcdctl --user root:$ROOT_PASSWORD get /message

To connect to your etcd server from outside the cluster execute the following commands:

    kubectl port-forward --namespace devops svc/etcd 2379:2379 &
    echo "etcd URL: http://127.0.0.1:2379"

 * As rbac is enabled you should add the flag `--user root:$ETCD_ROOT_PASSWORD` to the etcdctl commands. Use the command below to export the password:

    export ETCD_ROOT_PASSWORD=$(kubectl get secret --namespace devops etcd -o jsonpath="{.data.etcd-root-password}" | base64 --decode)

查看部署的资源

root@master:~# kubectl get statefulset -n devops | grep etcd
etcd                   3/3     3m7s
root@master:~# kubectl get pod -n devops | grep etcd
etcd-0                           1/1     Running   0          3m13s
etcd-1                           1/1     Running   0          3m13s
etcd-2                           1/1     Running   0          3m12s
root@master:~# kubectl get svc -n devops | grep etcd
etcd                            ClusterIP   10.233.47.112   <none>        2379/TCP,2380/TCP                            3m18s
etcd-headless                   ClusterIP   None            <none>        2379/TCP,2380/TCP                            3m18s

配置域名

cat >> /etc/hosts <<EOF
192.168.1.101 etcd.local.com
EOF

IngressRoute

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: etcd
  namespace: devops
spec:
  entryPoints:
    - web
  routes:
    - match: HostSNI(`*`)
      services:
        - name: etcd
          port: 2379

3. 测试

3.1 集群

查看集群状态

➜  ~ etcdctl --write-out=table --endpoints=etcd.local.com endpoint status
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|    ENDPOINT    |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| etcd.local.com | 7b8de1370db1ea57 |   3.5.1 |   20 kB |     false |      false |         2 |       1428 |               1428 |        |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
# 查看集群成员

查看集群健康状况

➜  ~ etcdctl --endpoints=etcd.local.com --user=root:90CjPHPRlxw= endpoint health
etcd.local.com is healthy: successfully committed proposal: took = 6.674885ms

查看集群成员

➜  ~ etcdctl --write-out=table --endpoints=etcd.local.com member list
+------------------+---------+--------+-----------------------------------------------------------+-----------------------------------------------------------------------------------------------------+------------+
|        ID        | STATUS  |  NAME  |                        PEER ADDRS                         |                                            CLIENT ADDRS                                             | IS LEARNER |
+------------------+---------+--------+-----------------------------------------------------------+-----------------------------------------------------------------------------------------------------+------------+
| 54eb31943ddc1e5a | started | etcd-1 | http://etcd-1.etcd-headless.devops.svc.cluster.local:2380 | http://etcd-1.etcd-headless.devops.svc.cluster.local:2379,http://etcd.devops.svc.cluster.local:2379 |      false |
| 7b8de1370db1ea57 | started | etcd-0 | http://etcd-0.etcd-headless.devops.svc.cluster.local:2380 | http://etcd-0.etcd-headless.devops.svc.cluster.local:2379,http://etcd.devops.svc.cluster.local:2379 |      false |
| 8a47754cbeead497 | started | etcd-2 | http://etcd-2.etcd-headless.devops.svc.cluster.local:2380 | http://etcd-2.etcd-headless.devops.svc.cluster.local:2379,http://etcd.devops.svc.cluster.local:2379 |      false |
+------------------+---------+--------+-----------------------------------------------------------+-----------------------------------------------------------------------------------------------------+------------+

3.2 基础操作

# 设置
➜  ~ etcdctl --endpoints=etcd.local.com --user=root:90CjPHPRlxw= put foo bar
OK
# 获取
➜  ~ etcdctl --endpoints=etcd.local.com --user=root:90CjPHPRlxw= get foo
foo
bar
# 删除
➜  ~ etcdctl --endpoints=etcd.local.com --user=root:90CjPHPRlxw= del foo
1