K8S CKS认证考试内容分析 --Part 1

Context
针对 kubeadm 创建的 cluster 运行 CIS 基准测试工具时，发现了多个必须立即解决的问题
Task
通过配置修复所有问题并重新启动受影响的组件以确保新的设置生效
修复针对 API 服务器发现的所有以下违规行为:
1.2.7 Enlure that the --authorization-mode argument is not set to AlwaysAllow   FAIL
1.2.8 Ensure that the --authorization-mode argument includes Node               FAIL
1.2.9 Ensure that the --authorization-mode argument includes RBAC               FAIL

修复针对 kubelet 发现的所有以下违规行为:
Fix all of the following violations that were found against the kubelet:
4.2.1 Ensure that the anonymous-auth argument is set to false                   FAIL
4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow   FAIL

注意:尽可能使用 Webhook 身份验证/授权。
修复针对etcd发现的所有以下违规行为:
Fix all of the following violations that were found against etcd:
2.2 Ensure that the -client-cert-auth argument is set to true                  FAIL

在安全端口上进行鉴权的插件的顺序列表。 逗号分隔的列表：AlwaysAllow、AlwaysDeny、ABAC、Webhook、RBAC、Node。默认值："AlwaysAllow"

##ssh 到master节点
#cp /etc/kubernetes/manifests/kube-apiserver.yaml ~/bak1/kube-apiserver.yaml   ##先备份
#vi /etc/kubernetes/manifests/kube-apiserver.yaml 
    - --authorization-mode=Node,RBAC    ## 修改这里的值

# ./kube-bench  run --targets=master --config-dir=/app/cfg |grep authorization-mode -A 2
[FAIL] 1.2.7 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated)
[FAIL] 1.2.8 Ensure that the --authorization-mode argument includes Node (Automated)
[FAIL] 1.2.9 Ensure that the --authorization-mode argument includes RBAC (Automated)
[WARN] 1.2.10 Ensure that the admission control plugin EventRateLimit is set (Manual)
[FAIL] 1.2.11 Ensure that the admission control plugin AlwaysAdmit is not set (Automated)
--
on the control plane node and set the --authorization-mode parameter to values other than AlwaysAllow.
One such example could be as below.
--authorization-mode=RBAC

1.2.8 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
on the control plane node and set the --authorization-mode parameter to a value that includes Node.
--authorization-mode=Node,RBAC

1.2.9 Edit the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml
on the control plane node and set the --authorization-mode parameter to a value that includes RBAC,
for example `--authorization-mode=Node,RBAC`.

#./kube-bench  run --targets=node --config-dir=/app/cfg
...
4.2.1 If using a Kubelet config file, edit the file to set `authentication: anonymous: enabled` to
`false`.
If using executable arguments, edit the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
`--anonymous-auth=false`
Based on your system, restart the kubelet service. For example,
systemctl daemon-reload
systemctl restart kubelet.service

4.2.2 If using a Kubelet config file, edit the file to set `authorization.mode` to Webhook. If
using executable arguments, edit the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
set the below parameter in KUBELET_AUTHZ_ARGS variable.
--authorization-mode=Webhook
Based on your system, restart the kubelet service. For example,
systemctl daemon-reload
systemctl restart kubelet.service

# cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
......

# cp /var/lib/kubelet/config.yaml ~/bak1/
#vi /var/lib/kubelet/config.yaml
...
authentication:
  anonymous:
    enabled: false   ## 这里如果是true，则要修改为falas，表示禁止匿名用户登录。
  webhook:
    cacheTTL: 0s
    enabled: true   ## 这里正常是true的，一般不用改，但如果是false，则需要修改成true，表示启用webhook认证
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook    ## 认证模式，如果是AlwaysAllow需要修改成 Webhook ，注意大小写。
  webhook:
    cacheAuthorizedTTL: 0s
    cacheUnauthorizedTTL: 0s
...

# ./kube-bench run --targets=etcd --config-dir=/app/cfg
...
2.2 Edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml on the master
node and set the below parameter.
--client-cert-auth="true"

#vi  /etc/kubernetes/manifests/etcd.yaml
...
    - --client-cert-auth=true  ## 把这个参数修改成true

# systemctl daemon-reload 
# systemctl restart kubelet

Context
您组织的安全策略包括:
- ServiceAccount 不得自动挂载 API 凭据
- ServiceAccount 名称必须以“-sa”结尾
清单文件 /cks/sa/pod1.yaml 中指定的 Pod 由于 ServiceAccount 指定错误而无法调度
请完成以下项目:
Task
1. 在现有 namespace qa 中创建一个名为 backend-sa 的新 ServiceAccount,确保此 ServiceAccount 不自动挂载API 凭据。
2. 使用 /cks/sa/pod1.yaml 中的清单文件来创建一个 Pod。
3. 最后，清理 namespace qa 中任何未使用的 ServiceAccount。

apiversion: V1
kind: Pod
metadata:
  name: backend
  namespace: qa
spec:
  serviceAccountName: test01
  containers:
  - image: nginx:1.9
    imagePullPolicy: IfNotPresent
    name: backend

automountServiceAccountToken (boolean)

AutomountServiceAccountToken 指示作为此服务帐户运行的 Pod 是否应自动挂载 API 令牌， 可以在 Pod 级别覆盖。

# kubectl -n qa create sa backend-sa
# kubectl -n qa edit sa backend-sa 
apiVersion: v1
kind: ServiceAccount
automountServiceAccountToken: false  ## 增加这一个参数，并且值设置为false
metadata:
...

# kubectl explain serviceaccount |grep auto
   automountServiceAccountToken <boolean>

# vi /cks/sa/pod1.yaml
metadata:
  name: backend
  namespace: qa  ## 确认命名空间正确
spec:
  serviceAccountName: backend-sa  ## 修改为backend-sa这个账号
# kubectl apply -f /cks/sa/pod1.yaml  ## 修改完记得创建

#kubectl get pod -n qa -o yaml|grep -i serviceaccountname
#kubectl get sa -n qa
#kubectl -n qa delete sa test01

Context
一个默认拒绝 (default-deny)的 NetworkPolicy 可避免在未定义任何其他 NetworkPoliy 的 namespace 中意外公开 Pod。

Task
为所有类型为 ingress+Egress 的流量在 namespace testing 中创建一个名为 denypolicy 的新默认拒绝 NetworkPolicy.
此新的 NetworkPolicy 必须拒绝 namespace testing 中的所有的 Ingress + Egress 流量
将新创建的默认拒绝NetworkPolicy 应用与在namespace testing 中运行的所有 Pod。

你可以在 /cks/net/p1.yaml 找到一个模板清单文件.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: denypolicy
  namesapce: testing
spec:
  podSelector: {}
  policyTypes:
  - Ingress   ## 注意，我考试时碰到的是只要求拒绝Egress流量，所以这行是要删除。每次考试可能要求的不一样，要注意应对。
  - Egress

#kubectl apply -f denypolicy.yaml
#kubectl describe NetworkPolicy -n testing

Context
绑定到 Pod的 ServiceAocount 的 Role 授予过度宽松的权限,完成以下项目以减少权限集。
Task
一个名为 web-pod 的现有 Pod 已在 namespace db 中运行。
编辑绑定到 Pod 的 ServiceAccount service-account-web 的现有 Role，仅允许只对 services 类型的资源执行get操作。
在namespace db 中创建一个名为 role-2 ，并仅允许只对 namespaces 类型的资源执行 delete 操作的新 Role。
创建一个名为 role-2-binding 的新 RoleBinding.将新创建的 Role 绑定到 Pod 的 ServiceAccount。
注意:请勿删除现有的 RoleBinding。

#k describe rolebindings -n db
...
Role:
  Kind:  Role
  Name:  role-1
Subjects:
  Kind            Name                 Namespace
  ----            ----                 ---------
  ServiceAccount  service-account-web  db

# kubectl edit role role-1 -n db
...
rules:
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get

rules:
- apiGroups: [""] # "" 标明 core API 组
  resources: ["services"]
  verbs: ["get"]

# kubectl create role role-2 --verb=delete --resource=namespaces -n db
role.rbac.authorization.k8s.io/role-2 created
# kubectl create rolebinding role-2-binding --serviceaccount=db:service-account-web --role=role-2 -n db
rolebinding.rbac.authorization.k8s.io/role-2-binding created

# kubectl describe rolebindings.rbac.authorization.k8s.io role-2-binding -n db
Name:         role-2-binding
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  Role
  Name:  role-2
Subjects:
  Kind            Name                 Namespace
  ----            ----                 ---------
  ServiceAccount  service-account-web  db