Setup K8s With Kubeadm

安装docker

检查dns

sudo cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.8.3.1
nameserver 114.114.114.114
nameserver 8.8.8.8
nameserver 114.114.114.114

详细说明

Letting iptables see bridged traffic

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system

安装kubeadm

kubeadm install mirror in china

安装

kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers
sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers

kubectl apply -f https://docs.projectcalico.org/v3.11/manifests/calico.yaml

kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true


kubeadm join 10.8.3.222:6443 --token awon9z.bcw8z \
    --discovery-token-ca-cert-hash sha256:7b90bca7225915f07179fd2ad31820533

检查DNS

kubectl run -it busybox --image=busybox --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ # nslookup kubernetes
Server:         10.96.0.10
Address:        10.96.0.10:53

为calico的KDD安装calicoctl(calico/ctl:v3.11.3 POD)

kubectl apply -f https://docs.projectcalico.org/manifests/calicoctl.yaml
alias calicoctl="kubectl exec -i -n kube-system calicoctl -- /calicoctl"
calicoctl get node

calicoctl node status用法提示

  • calicoctl的版本必须和calico-node版本一致才能正常工作

  • 必须在calico-node所在的节点上运行该命令

    curl -O -L  https://github.com/projectcalico/calicoctl/releases/download/v3.11.3/calicoctl
    export DATASTORE_TYPE=kubernetes
    export KUBECONFIG=~/.kube/config
    sudo ./calicoctl node status
    Calico process is running.
    
    IPv4 BGP status
    +--------------+-------------------+-------+------------+-------------+
    | PEER ADDRESS |     PEER TYPE     | STATE |   SINCE    |    INFO     |
    +--------------+-------------------+-------+------------+-------------+
    | 10.8.3.211   | node-to-node mesh | up    | 2020-10-30 | Established |
    +--------------+-------------------+-------+------------+-------------+
    
    IPv6 BGP status
    No IPv6 peers found.
    
    bigo@bigo-s3:~$ sudo ./calicoctl version
    Client Version:    v3.11.3
    Git commit:        05f36cc8
    no etcd endpoints specified
    bigo@bigo-s3:~$ alias calicoctl="kubectl exec -i -n kube-system calicoctl -- /calicoctl"
    bigo@bigo-s3:~$ calicoctl version
    Client Version:    v3.16.4
    Git commit:        51418082
    Cluster Version:   v3.11.3
    Cluster Type:      k8s,bgp,kdd
    bigo@bigo-s3:~$ calicoctl node status
    Calico process is not running.
    command terminated with exit code 1
    
    

https://docs.projectcalico.org/getting-started/clis/calicoctl/install

ubuntu 20

安装完后确保没有使用本地DNS缓存

unlink /etc/resolv.conf
ln -s /run/systemd/resolve/resolv.conf  /etc/resolv.conf
[FATAL][1783] int_dataplane.go 1035: Kernel's RPF check is set to 'loose'.  This would allow endpoints to spoof their IP address.  Calico requires net.ipv4.conf.all.rp_filter to be set to 0 or 1. If you require loose RPF and you are not concerned about spoofing, this check can be disabled by setting the IgnoreLooseRPF configuration parameter to 'true'.
kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true

确保br_netfilter模块已经加载

lsmod | grep br_netfilter
sudo modprobe br_netfilter

启用IP转发

sysctl -w net.ipv4.ip_forward = 1

重新安装

kubeadm reset
  • 刷新所有的链(-F),删除所有的非默认链(-X)

    iptables -F
    iptables -X
    
    iptables -nvL
    
    kubeadm token create --print-join-command