k8s-证书 - 学习笔记

所有master节点创建etcd证书目录 `mkdir /etc/etcd/ssl -p` 所有节点创建kubernetes相关目录 `mkdir -p /etc/kubernetes/pki` # Master1节点生成etcd证书生成证书的CSR文件：证书签名请求文件，配置了一些域名、公司、单位组织信息 ```shell cd /etc/kubernetes/pki # 生成Etcd CA证书和CA证书的KEY cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca ``` ```shell # 执行结果 [root@k8s-master1 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca 2022/08/01 09:44:04 [INFO] generating a new CA key and certificate from CSR 2022/08/01 09:44:04 [INFO] generate received request 2022/08/01 09:44:04 [INFO] received CSR 2022/08/01 09:44:04 [INFO] generating key: rsa-2048 2022/08/01 09:44:04 [INFO] encoded CSR 2022/08/01 09:44:04 [INFO] signed certificate with serial number 364920026958112443002224301757596366369850476928 [root@k8s-master1 pki]# ls /etc/etcd/ssl etcd-ca.csr etcd-ca-key.pem etcd-ca.pem ``` 颁发Etcd的客户端证书 ```shell cfssl gencert \ -ca=/etc/etcd/ssl/etcd-ca.pem \ -ca-key=/etc/etcd/ssl/etcd-ca-key.pem \ -config=ca-config.json \ -hostname=127.0.0.1,k8s-master1,k8s-master2,k8s-master3,k8s-master4,k8s-master5 \ -profile=kubernetes \ etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd ``` > hostname可以为etcd多预留几个IP或域名，方便以后进行扩容，减少以后可能再次签发证书的麻烦 ``` # 执行情况 [root@k8s-master1 pki]# cfssl gencert \ > -ca=/etc/etcd/ssl/etcd-ca.pem \ > -ca-key=/etc/etcd/ssl/etcd-ca-key.pem \ > -config=ca-config.json \ > -hostname=127.0.0.1,k8s-master1,k8s-master2,k8s-master3,k8s-master4,k8s-master5 \ > -profile=kubernetes \ > etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd 2022/08/01 10:10:15 [INFO] generate received request 2022/08/01 10:10:15 [INFO] received CSR 2022/08/01 10:10:15 [INFO] generating key: rsa-2048 2022/08/01 10:10:15 [INFO] encoded CSR 2022/08/01 10:10:15 [INFO] signed certificate with serial number 3829254656452712074263405800459719618168964154 ``` 将证书复制到其他节点 ```shell MasterNodes='k8s-master2 k8s-master3' WorkNodes='k8s-node1 k8s-node2' for NODE in $MasterNodes;do ssh $NODE "mkdir /etc/etcd/ssl -p" for FILE in etcd-ca-key.pem etcd-ca.pem etcd-key.pem etcd.pem;do scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE} done; done ``` # Master1节点生成kubernetes证书 > Etcd的证书和k8s的证书是相互独立的，可以使用一个ca文件生成k8s相关的(API Server、Controller-manager、scheduler、kubelet)客户端证书，它就相当于一个根证书 ```shell cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca ``` ```shell # 执行结果 [root@k8s-master1 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca 2022/08/01 10:28:24 [INFO] generating a new CA key and certificate from CSR 2022/08/01 10:28:24 [INFO] generate received request 2022/08/01 10:28:24 [INFO] received CSR 2022/08/01 10:28:24 [INFO] generating key: rsa-2048 2022/08/01 10:28:24 [INFO] encoded CSR 2022/08/01 10:28:24 [INFO] signed certificate with serial number 552016950580965560689461615026678627797257716092 ``` > k8s三个网段： service的网段：10.10.0.x，如果需要更改，则将10.10.0.1改成对于网段的第一个即可 pod的网段：172.168.0.0，相当于容器的IP地址宿主机的网段：192.168.xx.xxx 注：三个网段尽量不要重复 ## API Server证书 ```shell cfssl gencert \ -ca=/etc/kubernetes/pki/ca.pem \ -ca-key=/etc/kubernetes/pki/ca-key.pem \ -config=ca-config.json \ -hostname=10.10.0.1,127.0.0.1,k8s-master1,k8s-master2,k8s-master3,k8s-master4,k8s-master5,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.10.151,192.168.10.152 \ -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserve ``` ```shell # 执行结果 [root@k8s-master1 pki]# cfssl gencert \ > -ca=/etc/kubernetes/pki/ca.pem \ > -ca-key=/etc/kubernetes/pki/ca-key.pem \ > -config=ca-config.json \ > -hostname=10.10.0.1,127.0.0.1,k8s-master1,k8s-master2,k8s-master3,k8s-master4,k8s-master5,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.10.151,192.168.10.152 \ > -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserve 2022/08/01 11:20:33 [INFO] generate received request 2022/08/01 11:20:33 [INFO] received CSR 2022/08/01 11:20:33 [INFO] generating key: rsa-2048 2022/08/01 11:20:33 [INFO] encoded CSR 2022/08/01 11:20:33 [INFO] signed certificate with serial number 621628441439586849527097548883795381780989357019 ``` ## 生成API Server的聚合证书生成CA证书： ```shell cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca ``` ```shell # 执行情况 [root@k8s-master1 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca 2022/08/01 11:21:14 [INFO] generating a new CA key and certificate from CSR 2022/08/01 11:21:14 [INFO] generate received request 2022/08/01 11:21:14 [INFO] received CSR 2022/08/01 11:21:14 [INFO] generating key: rsa-2048 2022/08/01 11:21:14 [INFO] encoded CSR 2022/08/01 11:21:14 [INFO] signed certificate with serial number 429167082732624020844761757603099289472056382975 ``` ```shell cfssl gencert \ -ca=/etc/kubernetes/pki/front-proxy-ca.pem \ -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client ``` > 所有(删除、创建资源)请求都会经过API Server进行验证和过滤，在这个过程中API Server相当于代理服务器后面会有Requestheader-client-xxx的配置，API Server会通过这个指定的证书进行验证请求是否合法，怎么判断合法：还有一个：Requestheader-allowd-xxx:aggerator判断请求是否是允许的 ```shell # 执行情况 [root@k8s-master1 pki]# cfssl gencert \ > -ca=/etc/kubernetes/pki/front-proxy-ca.pem \ > -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem \ > -config=ca-config.json \ > -profile=kubernetes \ > front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client 2022/08/01 11:28:01 [INFO] generate received request 2022/08/01 11:28:01 [INFO] received CSR 2022/08/01 11:28:01 [INFO] generating key: rsa-2048 2022/08/01 11:28:02 [INFO] encoded CSR 2022/08/01 11:28:02 [INFO] signed certificate with serial number 578465796960400487332946505853093900973678690894 2022/08/01 11:28:02 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); specifically, section 10.2.3 ("Information Requirements"). ``` ## 生成Controller-manager的证书使用前面生成的k8s证书进行生成 ```shell cfssl gencert \ -ca=/etc/kubernetes/pki/ca.pem \ -ca-key=/etc/kubernetes/pki/ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ controller-manager-csr.json | cfssljson -bare /etc/kubernetes/controller-manager ``` ```shell # 执行情况 [root@k8s-master1 pki]# cfssl gencert \ > -ca=/etc/kubernetes/pki/ca.pem \ > -ca-key=/etc/kubernetes/pki/ca-key.pem \ > -config=ca-config.json \ > -profile=kubernetes \ > controller-manager-csr.json | cfssljson -bare /etc/kubernetes/controller-manager 2022/08/01 11:42:29 [INFO] generate received request 2022/08/01 11:42:29 [INFO] received CSR 2022/08/01 11:42:29 [INFO] generating key: rsa-2048 2022/08/01 11:42:29 [INFO] encoded CSR 2022/08/01 11:42:30 [INFO] signed certificate with serial number 202355660092475516505431482093878224708735991380 2022/08/01 11:42:30 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); specifically, section 10.2.3 ("Information Requirements"). ``` ## kubeconfig的文件配置 > kubectl 会通过 kubeconfig 文件操作集群，其他组件连接API Server的时候也能使用到kubeconfig文件连接到API Server kubeconfig文件包含了：API Server的地址、连接API Server的证书 ```shell # set-cluster:设置一个集群项 kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/pki/ca.pem \ --embed-certs=true \ --server=https://192.168.10.151:8443 \ --kubeconfig=/etc/kubernetes/controller-manager.kubeconfig # 设置一个环境项，一个上下文 kubectl config set-context system:kube-controller-manager@kubernetes \ --cluster=kubernetes \ --user=system:kube-controller-manager \ --kubeconfig=/etc/kubernetes/controller-manager.kubeconfig # set-credentials:设置一个用户项 kubectl config set-credentials system:kube-controller-manager \ --client-certificate=/etc/kubernetes/pki/controller-manager.pem \ --client-key=/etc/kubernetes/pki/controller-manager-key.pem \ --embed-certs=true \ --kubeconfig=/etc/kubernetes/controller-manager.kubeconfig # 将环境设置成默认的环境 kubectl config use-context system:kube-controller-manager@kubernetes \ --kubeconfig=/etc/kubernetes/controller-manager.kubeconfig ``` ```shell cfssl gencert \ -ca=/etc/kubernetes/pki/ca.pem \ -ca-key=/etc/kubernetes/pki/ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ scheduler-csr.json | cfssljson -bare /etc/kubernetes/scheduler ``` ```shell kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/pki/ca.pem \ --embed-certs=true \ --server=https://192.168.10.151:8443 \ --kubeconfig=/etc/kubernetes/scheduler.kubeconfig ``` ```shell kubectl config set-credentials system:kube-scheduler \ --client-certificate=/etc/kubernetes/pki/scheduler.pem \ --client-key=/etc/kubernetes/pki/scheduler-key.pem \ --embed-certs=true \ --kubeconfig=/etc/kubernetes/scheduler.kubeconfig ``` ```shell kubectl config set-context system:kube-scheduler@kubernetes \ --cluster=kubernetes \ --user=system:kube-scheduler \ --kubeconfig=/etc/kubernetes/scheduler.kubeconfig # 将环境设置成默认的环境 kubectl config use-context system:kube-scheduler@kubernetes \ --kubeconfig=/etc/kubernetes/scheduler.kubeconfig ``` ## 生成Admin的证书，admin是用于管理集群的 ```shell cfssl gencert \ -ca=/etc/kubernetes/pki/ca.pem \ -ca-key=/etc/kubernetes/pki/ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ admin-csr.json | cfssljson -bare /etc/kubernetes/admin ``` ```shell kubectl config set-cluster kubernetes \ --certificate-authority=/etc/kubernetes/pki/ca.pem \ --embed-certs=true \ --server=https://192.168.10.151:8443 \ --kubeconfig=/etc/kubernetes/admin.kubeconfig ``` ```shell # 设置上下文 kubectl config set-context kubernetes-admin@kubernetes \ --cluster=kubernetes \ --user=kubernetes-admin \ --kubeconfig=/etc/kubernetes/admin.kubeconfig ``` ```shell # 将环境设置成默认的环境 kubectl config use-context kubernetes-admin@kubernetes \ --kubeconfig=/etc/kubernetes/admin.kubeconfig ``` # 创建ServiceAccount Key > 创建ServiceAccount之后会生成一个和ServiceAccount绑定的secret secret会产生一个token。 token使用`openssl genrsa -out /etc/kubernetes/pki/sa.key 2048`证书去生成的 ```shell openssl genrsa -out /etc/kubernetes/pki/sa.key 2048 openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub ``` 将证书复制到其他节点 ```shell MasterNodes='k8s-master2' WorkNodes='k8s-node1 k8s-node2' for NODE in $MasterNodes;do for FILE in $(ls /etc/kubernetes/pki | grep -v etcd);do scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE}; done; for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig;do scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE}; done; done ``` 证书签发完成返回安装步骤：[k8s的安装](doc:m2hBw1Ix)