Kubernetes TLS Bootstrap是CKA考试的一个模块，这里系统学习一下如何做这种题目

Q&A

bootstrap-kubeconfig是怎么生成的，里面包含了啥
- 不知道咋生成的
- apiserver url 和 token
创建CSR
- 需要管理员参与么？还是kubelet自己就会
同意该CSR
- 如何配置才能默认让controller-manager同意
- 手动使用kubectl怎么同意
apiserver得给kubelet权限
- --enable-bootstrap-token-auth=true
- 怎么使用token授权？让其进入system:bootstrapers权限
- 使用RoleBinding给kubelet权限来创建CSR
如何创建 token呢？
- 第一种是普通token，按照–token-auth-file的形式
- 第二种是bootstrap token，是一个特殊的secret
kubelet的参数怎么配置呢？
- 不知道
- 使用bootstrap token是否和使用普通token一样呢

做题流程

选择方法

是否有kubeadm，如果有选择kubeadm的方法，该方法会使用bootstrp token的方式
检查kube-apiserver参数是否支持了--enable-bootstrap-token-auth=true，若支持了，则采用bootstrap token的模式
否则，采用普通token的认证方式--token-auth-file=FILENAME

安装bin

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

kubeadm 启动

查看kubeadm init创建的token，是否过期了

kubeadm token list

kubeadm join 三种模式 1. kubeadm join -–discovery-token abcdef.1234567890abcdef apiserver_ip:6443 2. kubeadm join -–discovery-file path/to/file.conf 3. kubeadm join -–discovery-file https://url/file.conf

kubeadm join

计算sha256的$hash用于验证CA，这是第一种模式必须的

# 计算CA的hash，kubeadm init也会自动输出
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

创建用于discovery和tlsbootstrap的token，这两个过程公用一个$token

kubeadm token create
# generate一个符合格式的token，并创建合法的secret

执行Join

kubeadm join --discovery-token $token --discovery-token-ca-cert-hash sha256:$hash  $apiserver_ip:6443

bootstrap token 启动

准备bootstrap token

要求：名字有特殊规定bootstrap-token-<token id>，存在于kube-system中的secret

检查是否已经有bootstrap token了

kubectl get secret -n kube-system | grep "bootstrap-token-"

创建一个bootstrap token
格式： [a-z0-9]{6}\.[a-z0-9]{16}
example: abcdef.$(head -c 8 /dev/random | od -An -t x | tr -d ' ')

example: kubeadm generate token仅输出符合格式的token

apiVersion: v1
kind: Secret
metadata:
# Name MUST be of form "bootstrap-token-<token id>"
name: bootstrap-token-07401b
namespace: kube-system

# Type MUST be 'bootstrap.kubernetes.io/token'
type: bootstrap.kubernetes.io/token
stringData:
# Human readable description. Optional.
description: "The default bootstrap token generated by 'kubeadm init'."

# Token ID and secret. Required.
token-id: abcdef
token-secret: f395accd246ae52d

# Expiration. Optional.
# expiration: 2017-03-10T03:22:11Z

# Allowed usages.
usage-bootstrap-authentication: "true"
usage-bootstrap-signing: "true"

# Extra groups to authenticate the token as. Must start with "system:bootstrappers:"
auth-extra-groups: system:bootstrappers:worker,system:bootstrappers:ingress

准备bootstrap-kubeconfig

从master上copy过来ca.pem

使用之前创建好的bootstrap-token

#/var/lib/kubelet/bootstrap-kubeconfig
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority: /var/lib/kubernetes/ca.pem
server: https://my.server.example.com:6443
name: bootstrap
contexts:
- context:
cluster: bootstrap
user: kubelet-bootstrap
name: bootstrap
current-context: bootstrap
preferences: {}
users:
- name: kubelet-bootstrap
user:
token: 07401b.f395accd246ae52d
# 使用之前创建好的token

准备ClusterRoleBinding

1.8之后的版本系统都默认创建了一些ClusterRole，我们只需要绑定到system:bootstrappers这个Group上就行了

system:node-bootstrapper
- 具备申请CSR的权限

下面两个是用于自动approve的，也就是只要有权限，就会自动approve - system:certificates.k8s.io:certificatesigningrequests:nodeclient - 具备创建certificates的权限 - system:certificates.k8s.io:certificatesigningrequests:selfnodeclient - 具备更新certificates的权限，所以这个绑定到system:nodes上

或者可以选择手动approve，kubectl get csr,kubectl certificate approve <name>

---
# enable bootstrapping nodes to create CSR
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: create-csrs-for-bootstrapping
subjects:
- kind: Group
  name: system:bootstrappers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: system:node-bootstrapper
  apiGroup: rbac.authorization.k8s.io
---
# Approve all CSRs for the group "system:bootstrappers"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: auto-approve-csrs-for-group
subjects:
- kind: Group
  name: system:bootstrappers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: system:certificates.k8s.io:certificatesigningrequests:nodeclient
  apiGroup: rbac.authorization.k8s.io
---
# Approve renewal CSRs for the group "system:nodes"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: auto-approve-renewals-for-nodes
subjects:
- kind: Group
  name: system:nodes
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
  apiGroup: rbac.authorization.k8s.io

配置kubelet参数

# /lib/systemd/system/kubelet.service
--bootstrap-kubeconfig="/var/lib/kubelet/bootstrap-kubeconfig" --kubeconfig="/var/lib/kubelet/kubeconfig"

swapoff -a
systemctl daemon-reload
systemctl restart kubelet
systemctl status kubelet
# 这步可以看到是否加载了你的参数，有时候会从/etc/systemd/system/kubelet.service.d 读取
kubectl get nodes
kubectl get csr

普通 token 启动

准备token

token=$(head -c 16 /dev/random | od -An -t x | tr -d ' ')

apiserver接收token

cat <<EOF>> tokenfile
$token,kubelet-bootstrap,10001,"system:bootstrappers"
EOF

# apiserver
--token-auth-file=tokenfile

Kubernetes TLS Bootstrap

Q&A