k8s就不用过多介绍了，是Google开源的一款强大的容器管理工具，现在更多的场景是用来管理docker集群，服务编排。
这里需要强调一下，本文是基于yum安装的k8s，通过yum安装的k8s的api并不完整，或者说不能使用全部k8s的功能（比如deployment），想在centos上使用k8s全部功能，建议源码编译安装
k8s的一些基本概念
这里部分参考了这篇文章 http://dockone.io/article/932
Master：kubernetes的主节点，上面运行着kube-apiserver, kube-controller-manager, and kube-scheduler, etcd服务
Minion: node节点，上面运行着kubelet, proxy, cadvisor and docker.
Kubectl
- 通过cli方式可以直接与k8s交互，具体可以参考 kubectl CLI reference
- kubectl的具体配置实用见下文
Kube-Apiserver
- k8s的管理接口
Pods
- 一组容器叫做pod，一个pod中的容器一起部署，开始，停止，以整个group的方式被复制。
- Pod是kubernetes的最小操作单元，一个Pod可以由一个或多个容器组成；
- 同一个Pod只能运行在同一个主机上，共享相同的volumes、network、namespace、ip（pid namespace共享在docker里还没有实现）；
Desired State
- Desired State 是k8s中一个很重要的概念，许多组件自己定义了一个desired state，k8s的责任是让组件当前的状态匹配上desired state
- 拿pod为例，当创建一个pod，desired state希望这个pod里的containers处于运行状态，当这个pod里的containers因某些原因停止了，k8s会持续的重新创建containers以达到desired state，这个过程会一直持续到Pod删除
- Like individual application containers, pods are considered to be relatively ephemeral (rather than durable) entities. As discussed in life of a pod, pods are created, assigned a unique ID (UID), and scheduled to nodes where they remain until termination (according to restart policy) or deletion. If a node dies, the pods scheduled to that node are deleted, after a timeout period. A given pod (as defined by a UID) is not “rescheduled” to a new node; instead, it can be replaced by an identical pod, with even the same name if desired, but with a new UID (see replication controller for more details). (In the future, a higher-level API may support pod migration.)
When something is said to have the same lifetime as a pod, such as a volume, that means that it exists as long as that pod (with that UID) exists. If that pod is deleted for any reason, even if an identical replacement is created, the related thing (e.g. volume) is also destroyed and created anew.
Labels
- key-value键值对，可以用来分组
- 在k8s中所有资源都支持Label,资源包括（pod，service，replication controller，deployment）
Replication Controller
- Replication Controller确保任意时间都有指定数量的Pod“副本”在运行。如果为某个Pod创建了Replication Controller并且指定3个副本，它会创建3个Pod，并且持续监控它们。如果某个Pod不响应，那么Replication Controller会替换它，保持总数为3.
- 当创建Replication Controller时，需要指定两个东西：
- Pod模板：用来创建Pod副本的模板
- Label：Replication Controller需要监控的Pod的标签。
Service
- . A service provides a way to refer to a set of pods (selected by labels) with a single static IP address. It may also provide load balancing, if supported by the provider.
Health Checking
- k8s提供了健康检查的机制，我们知道传统上的健康检查可以在进程层面，k8s要求docker daemon去检查容器进程是否在运行，如果发现某个容器进程死亡，那么docker daemon就会重启容器。但是如果进程层面是好的，但是实际上运行状态不一定是好的，比如发生了进程仍在运行，但是服务其实没有正常工作。
- 因此k8s为此提供了一种应用级别的健康检查机制，你需要自己定义应用在什么状态下才是正常的，目前k8s提供三种形式
- HTTP的方式
- 在容器内部执行命令的方式
- TCP socket方式
- 一旦kubelet发现健康检查失败了，容器重启
Pod Infrastructure Container
The infrastructure container is where the kubelet puts all the resources that are shared across containers in the pod. This includes the IP, as well as any volumes we’ve defined
Deployment
Ensures that a specified number of pod “replicas” are running at any one time.
Unlike in the case where you directly create pods, a Deployment replaces pods that are deleted or terminated for any reason, such as in the case of node failure. For this reason, we recommend that you use a Deployment for a continuously running application even if your application requires only a single pod, in which case you can omit replicas and it will default to a single replica.
安装环境
- 系统版本：Centos 7
- Docker版本：1.10.3
- k8s版本：1.2.0
- 三台台机子：一台master（kube-apiserver, kube-controller-manager, and kube-scheduler, etcd），二台minion(每台都要安装docker, kube-proxy, kubelet)
部署流程
Master
yum安装软件包
创建一个virt7-docker-common-release.repo
[virt7-docker-common-release] name=virt7-docker-common-release baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/ gpgcheck=0 $ yum -y update $ yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd
添加hosts
$ echo "192.168.xx.xx centos-master 192.168.xx.xx centos-minion" >> /etc/hosts修改/etc/etcd/etcd.conf
$ grep -v ^# /etc/etcd/etcd.conf ETCD_NAME=default ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_CLIENT_URLS="http://centos-master:2379" ETCD_ADVERTISE_CLIENT_URLS="http://centos-master:2379"修改/etc/kubernetes/config
$ grep -v "^#" /etc/kubernetes/config KUBE_LOGTOSTDERR="--logtostderr=true" KUBE_LOG_LEVEL="--v=0" KUBE_ALLOW_PRIV="--allow-privileged=false" KUBE_MASTER="--master=http://centos-master:8080"修改/etc/kubernetes/apiserver
$ grep -v "^#" /etc/kubernetes/apiserver KUBE_API_ADDRESS="--address=0.0.0.0" KUBE_API_PORT="--port=8080" KUBE_MASTER="--master=http://centos-master:8080" KUBE_ETCD_SERVERS="--etcd-servers=http://centos-master:2379" KUBELET_PORT="--kubelet-port=10250" KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota" KUBE_API_ARGS=""
注意,这里把KUBE_ADMISSION_CONTROL里默认的SecurityContextDeny,ServiceAccount去掉，否则后面创建pod时候，可能会报错Pod "nginx" is forbidden: no API token found for service account default/default, retry after the token is automatically created and added to the service account
关闭防火墙
$ systemctl disable firewalld $ systemctl stop firewalld启动各项服务
写个shell脚本启动
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done如果没有报什么错误，各项服务就启动成功啦 :)
两台Minion
先执行master的步骤1(etcd不用装)，2,4,6
修改/etc/kubenetes/proxy
$ grep -v "^#" /etc/kubernetes/proxy KUBE_PROXY_ARGS="--proxy-mode=iptables"重要选项说明：–proxy-mode，代理模式，默认使用userspace，会如下报错，
kube-proxy[11895]: E0331 09:33:21.041898 11895 proxier.go:193] Error removing pure-iptables proxy rule: error checking rule: exit status 2: iptables v1.4.21: Couldn't load target `KUBE-SERVICES':No such file or directory
我们需要修改为iptables做网络转发
**这里参考了这篇博客** [http://lizhenliang.blog.51cto.com/7876557/1736572](http://lizhenliang.blog.51cto.com/7876557/1736572))
关于网络部分的配置
Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on
To achieve this we must impose some requirements on how you set up your cluster networking.
为什么要进行网络配置
我们知道目前docker的网络模式：宿主机上新建了一个
docker0的网桥，在这台虚拟机上的容器桥接到docker0，默认docker0的ip是172.17.x.xk8s本身对底层container或者说pod网络互联不做实现，而是由其他工具如docker本身自带的overlay，或者ovs，flannel等工具。k8s中每个Pod都有能与其它Pod通信的ip，同一个Pod之内的containers共享网络的namespaces，包括ip地址，同一个Pod内的容器之间可以通过本地localhost之间的各个端口通信，这也就是说同一个Pod内的容器之间端口不能重复，这跟一个VM内的进程很像。
已知的一些网络实现方案
- docker自带的overlay network
- Flannel
- OVS
- Weave
- Calico
- Romana
ovs网络模型
【图片来源：http://kubernetes.io/images/docs/ovs-networking】
- kbr0要替代默认的docker0网桥(要在docker启动配置文件中加入
-br=kbr0) - pod之间网络通信都走ovs
- 但是使用ovs搭建网络，若要保证两两node之间网络都通需要
n*(n-1)/2个隧道，所以在node比较多的情况下若想用ovs，需要考虑自动化配置。
- kbr0要替代默认的docker0网桥(要在docker启动配置文件中加入
这里以任意一个minion0为例，minion1也做类似的步骤，网络结构参考上面的ovs那张图
建立一个obr0网桥
$ ovs-vsctl add-br obr0建立隧道gre0，并添加到obr0上
$ ovs-vsctl add-port obr0 gre0 -- set Interface gre0 type=gre options:remote_ip=xx.xxremote_ip写对端ip，若是在minion0上配置，就写minion1的ip
创建linux bridge
$brctl addbr kbr0添加obr0为kbr0的接口
$ brctl addif kbr0 obr0设置docker0为down
$ ip link set dev docker0 down删除docker0
$ ip link del dev docker0保证kbr0在重启也生效，/etc/sysconfig/network-scripts/目录下新建minion1的ifcfg-kbr0如下
DEVICE=kbr0 ONBOOT=yes BOOTPROTO=static IPADDR=172.xx NETMASK=255.255.255.0 GATEWAY=172.xx USERCTL=no TYPE=Bridge IPV6INIT=no将docker启动网络选项改成kbr0
在docker启动配置文件中加入
-b=kbr0重启网络
$ systemctl restart networkminion1做类似的配置
minion0和minion1 配置好以后，验证网络互通，首先要分别在minion0和minion1上增加路由
$ ip route add 172.17.xx.xx/24 via 192.168.xx.xx dev eth1192.168.xx.xx为对端ip
在minion0上ping minon1的kbr0网段ip（172开头的）通，在minion0上 ping minion0也通。
ovs网络部分配置完毕！