kube-proxy开启ipvs的前置条件

  1. 加载netfilter模块

    1
    [[email protected] ~]# modprobe br_netfilter  
  2. 添加配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    [[email protected] ~]# cat  >  /etc/sysconfig/modules/ipvs.modules  <<EOF

    #!/bin/bash

    modprobe -- ip_vs

    modprobe -- ip_vs_rr

    modprobe -- ip_vs_wrr

    modprobe -- ip_vs_sh

    modprobe -- nf_conntrack_ipv4

    EOF
  3. 赋予权限并引导

    1
    2
    3
    4
    5
    [[email protected] ~]# chmod 755 /etc/sysconfig/modules/ipvs.modules

    [[email protected] ~]# bash /etc/sysconfig/modules/ipvs.modules

    [[email protected] ~]# lsmod | grep -e ip_vs -e nf_conntrack_ipv4

安装docker软件

  1. docker依赖

    1
    [[email protected] ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
  2. 导入阿里云的docker-ce仓库

    1
    [[email protected] ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
  3. 更新系统安装docker-ce

    1
    [[email protected] ~]# yum update -y && yum install -y docker-ce
  4. uname -r 检测版本,再设置版本,后又重启reboot

    1
    [[email protected] ~]# grub2-set-default "CentOS Linux (4.4.182-1.el7.elrepo.x86_64) 7 (Core)"
  5. 启动

    1
    [[email protected] ~]# systemctl start docker
  6. 开机自启

    1
    [[email protected] ~]# systemctl enable docker
  7. 配置deamon

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    [[email protected] ~]# cat > /etc/docker/daemon.json <<EOF
    {

    "exec-opts": ["native.cgroupdriver=systemd"],

    "log-driver": "json-file",

    "log-opts": {

    "max-size": "100m"

    }

    }

    EOF
  8. 创建目录存放docker配置文件

    1
    [[email protected] ~]# mkdir -p  /etc/systemd/system/docker.service.d
  9. 重启docker

    1
    [[email protected] ~]# systemctl daemon-reload && systemctl restart docker && systemctl enable docker

安装Kubeadm(主从配置)

  1. 导入阿里云的YUM仓库

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21

    [[email protected] ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo

    [kubernetes]

    name=Kubernetes

    baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64

    enabled=1

    gpgcheck=0

    repo_gpgcheck=0

    gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

    http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

    EOF

  2. 在每个节点安装kubeadm(初始化工具)、kubectl(命令行管理工具)、kubelet(与docker的cri交互创建容器)

    1
    [[email protected] ~]# yum -y install kubeadm kubectl kubelet  
  3. k8s开机自启.kubelet需要与容器接口进行交互启动容器,而k8s通过Kubeadm安装出来以后都是以pod方式存在,也就是底层以容器的方式运行,所以一定要开机自启

    1
    [[email protected] ~]# systemctl enable kubelet.service

初始化主节点(master)

  1. 初始化文件生成

    显示init默认的初始化文件,并打印出来到kubeadm-config.yaml文件中

    1
    [[email protected] ~]# kubeadm config print init-defaults > kubeadm-config.yaml
  2. 修改默认配置

    vi kubeadm-config.yaml
    &nbsp;

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    apiVersion: kubeadm.k8s.io/v1beta2
    bootstrapTokens:
    - groups:
    - system:bootstrappers:kubeadm:default-node-token
    token: abcdef.0123456789abcdef
    ttl: 24h0m0s
    usages:
    - signing
    - authentication
    kind: InitConfiguration
    localAPIEndpoint:
    advertiseAddress: 1.2.3.4
    bindPort: 6443
    nodeRegistration:
    criSocket: /var/run/dockershim.sock
    name: k8s-master-01
    taints:
    - effect: NoSchedule
    key: node-role.kubernetes.io/master
    ---
    apiServer:
    timeoutForControlPlane: 4m0s
    apiVersion: kubeadm.k8s.io/v1beta2
    certificatesDir: /etc/kubernetes/pki
    clusterName: kubernetes
    controllerManager: {}
    dns:
    type: CoreDNS
    etcd:
    local:
    dataDir: /var/lib/etcd
    imageRepository: k8s.gcr.io
    kind: ClusterConfiguration
    kubernetesVersion: v1.18.0
    networking:
    dnsDomain: cluster.local
    serviceSubnet: 10.96.0.0/12
    scheduler: {}
    • 修改ip为自己的ip地址
      advertiseAddress: 1.2.3.4 => advertiseAddress: 172.18.98.3

    • 版本号修改
      kubernetesVersion: v1.18.0 => kubernetesVersion: v1.18.1

    • 添加pod网段

      1
      2
      networking:
      podSubnet: 10.122.0.0/16
    • 修改默认调度

      1
      2
      3
      4
      5
      apiVersion: kubeproxy.config.k8s.io/v1alpha1
      kind: KubeProxyConfiguration
      featureGates:
      SupportIPVSProxyMode: true
      mode: ipvs
  3. 初始化master

    • 指定配置文件启动
    1
    [[email protected] ~]# kubeadm init --config=kubeadm-config.yaml --experimental-upload-certs | tee kubeadm-init.log
    • 可以在启动命令里指定参数配置
    1
    2
    3
    4
    [[email protected] ~]# kubeadm init --kubernetes-version=1.18.0  \
    --apiserver-advertise-address=172.18.98.3 \
    --image-repository registry.aliyuncs.com/google_containers \
    --service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16

    这一步很关键,由于kubeadm 默认从官网k8s.grc.io下载所需镜像,国内无法访问,因此需要通过–image-repository指定阿里云镜像仓库地址。

初始化K8s集群(node)

  1. 在工作节点执行注册

    1
    2
    3
    4
    5
    6
    7
    8
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
    https://kubernetes.io/docs/concepts/cluster-administration/addons/

    Then you can join any number of worker nodes by running the following on each as root:

    kubeadm join 172.18.98.7:6443 --token wi7way.kt2398q4g7jp6528 \
    --discovery-token-ca-cert-hash sha256:e46d8fd7c9c7f6d1b4a4bcd1a3591f8b364d2c3c80366269a054fadaf1d79c07

    记录生成的最后部分内容,此内容需要在其它节点加入Kubernetes集群时执行。
    根据提示创建kubectl

    1
    2
    3
    [[email protected] ~]# mkdir -p $HOME/.kube
    [[email protected] ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    [[email protected] ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

    执行下面命令,使kubectl可以自动补充

    1
    source <(kubectl completion bash)

部署网络

  1. 查看节点

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    [[email protected] ~]# kubectl get node
    NAME STATUS ROLES AGE VERSION
    k8s-master-01 NotReady master 2m29s v1.18.1
    k8s-node-01 NotReady <none> 1m17s v1.18.1
    k8s-node-02 NotReady <none> 37s v1.18.1
    [[email protected] ~]# kubectl get pod --all-namespaces
    NAMESPACE NAME READY STATUS RESTARTS AGE
    kube-system coredns-7ff77c879f-6fpj4 0/1 Pending 0 2m12s
    kube-system coredns-7ff77c879f-7tjcn 0/1 Pending 0 2m12s
    kube-system etcd-k8s-master-01 1/1 Running 1 2m12s
    kube-system kube-apiserver-k8s-master-01 1/1 Running 1 2m12s
    kube-system kube-controller-manager-k8s-master-01 1/1 Running 1 2m12s
    kube-system kube-proxy-2c2fs 1/1 Running 1 2m12s
    kube-system kube-proxy-cvmxt 1/1 Running 1 2m12s
    kube-system kube-proxy-k6ddp 1/1 Running 1 2m12s
    kube-system kube-scheduler-k8s-master-01 1/1 Running 1 2m12s
  2. 安装calico网络

    • node节点为NotReady,因为corednspod没有启动,缺少网络pod
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    [[email protected] ~]# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
    configmap/calico-config created
    customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
    customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
    clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
    clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
    clusterrole.rbac.authorization.k8s.io/calico-node created
    clusterrolebinding.rbac.authorization.k8s.io/calico-node created
    daemonset.apps/calico-node created
    serviceaccount/calico-node created
    deployment.apps/calico-kube-controllers created
    serviceaccount/calico-kube-controllers created
    • 查看node和pod
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    [[email protected] ~]# kubectl get pod --all-namespaces
    NAME READY STATUS RESTARTS AGE
    calico-kube-controllers-555fc8cc5c-q2cc8 1/1 Running 0 36s
    calico-node-4968l 1/1 Running 0 36s
    calico-node-7qr9w 1/1 Running 0 36s
    calico-node-tszrb 1/1 Running 0 36s
    coredns-7ff77c879f-6fpj4 1/1 Running 0 5m22s
    coredns-7ff77c879f-7tjcn 1/1 Running 0 5m22s
    etcd-k8s-master-01 1/1 Running 0 5m32s
    kube-apiserver-k8s-master-01 1/1 Running 0 5m32s
    kube-controller-manager-k8s-master-01 1/1 Running 0 5m32s
    kube-proxy-2c2fs 1/1 Running 0 5m32s
    kube-proxy-cvmxt 1/1 Running 0 5m32s
    kube-proxy-k6ddp 1/1 Running 0 5m32s
    kube-scheduler-k8s-master-01 1/1 Running 0 5m32s
    [[email protected] ~]# kubectl get node
    NAME STATUS ROLES AGE VERSION
    k8s-master-01 Ready master 5m47s v1.18.1
    k8s-node-01 Ready <none> 4m35s v1.18.1
    k8s-node-02 Ready <none> 3m55s v1.18.1

其他指令

  • 查看日志

    vim kubeadm-init.log

  • 查看节点信息

    kubectl get pod -n kube-system

  • 监视

    kubectl get pod -n kube-system -w

  • 详细信息

    kubectl get pod -n kube-system -o wide