第15关 k8s架构师课程之CICD自动化devops大结局
CI/CD生产实战项目
大家好，我是博哥爱运维。这节课我们开始最终CI/CD自动化流程实战，终于要到打大BOSS大结局了，博哥自从2021年3月1日开始分享这套K8S架构师课程以来，坚持每天整理文档录制视频，一直坚持到今天，在这期间，博哥认识了不少喜欢K8S的朋友，也收到了很多朋友的鼓励和建议，这对博哥都是宝贵的财富。有些人可能会想，在现如今这个社会，免费的东西还存在嘛？免费的东西就是最贵的东西，诚然，这些博哥也认同，但也不能排除网上也有很多热爱技术，执着分享的人，像国内外很多大牛开源出来很多优化的代码项目，像优秀的操作系统LINUX，像谷歌开源的这套K8S系统等等，博哥虽然做不到这么优秀，但也想把自己工作中的一些踩坑经验积累分享给大家，要说私心嘛，就是博哥想锻炼下自己的讲课经验，拓宽下自己的职业发展路线，但这个和我分享给大家的内容不相冲突，反而我认为它们是有利的，相辅相成的，博哥分享的所有东西都是实实在在工作中拿下来的生产经验，再精心整理来作分享。
大家一定要仔细观看，多多操作，把整个流程都掌握透彻。这里我会采用目前企业较常见的编程语言python的flask模块来实施完整的项目自动化流程步骤，其他语言都可以参照这个项目来实施自动化流程。
先把k8s的二进制命令行工具kubectl容器化备用
FROM harbor.boge.com/library/alpine:3.13
MAINTAINER boge
ENV TZ "Asia/Shanghai"
RUN sed -ri 's+dl-cdn.alpinelinux.org+mirrors.aliyun.com+g' /etc/apk/repositories\
&& apk add --no-cache curl tzdata ca-certificates\
&& cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime\
&& apk upgrade\
&& rm -rf /var/cache/apk/*
COPY kubectl /usr/local/bin/
RUN chmod +x /usr/local/bin/kubectl
ENTRYPOINT ["kubectl"]
CMD ["help"]
python的flask模块
准备好flask相关的代码文件上传到gitlab代码仓库
app.py
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello, boge! 21.04.11.01'
@app.route('/gg/<username>')
def hello(username):
return 'welcome' + ': ' + username + '!'
Dockerfile
FROM harbor.boge.com/library/python:3.5-slim-stretch
MAINTAINER boge
WORKDIR /kae/app
COPY requirements.txt .
RUN sed -i 's/deb.debian.org/ftp.cn.debian.org/g' /etc/apt/sources.list\
&& sed -i 's/security.debian.org/ftp.cn.debian.org/g' /etc/apt/sources.list\
&& apt-get update -y\
&& apt-get install -y wget gcc libsm6 libxext6 libglib2.0-0 libxrender1 make\
&& apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple -r requirements.txt\
&& rm requirements.txt
COPY . .
EXPOSE 5000
HEALTHCHECK CMD curl --fail http://localhost:5000 || exit 1
ENTRYPOINT ["gunicorn", "app:app", "-c", "gunicorn_config.py"]gunicorn_config.py
bind = '0.0.0.0:5000'
graceful_timeout = 3600
timeout = 1200
max_requests = 1200
workers = 1
worker_class = 'gevent'
requirements.txt
flask
gevent
gunicorn在代码仓库变量配置里面配置如下变量值
Type Key Value State Masked
Variable DOCKER_USER admin 下面都关闭 下面都关闭
Variable DOCKER_PASS boge666
Variable REGISTRY_URL harbor.boge.com
Variable REGISTRY_NS product
File KUBE_CONFIG_TEST k8s相关config配置文件内容
准备项目自动化配置文件.gitlab-ci.yml
stages:
- build
- deploy
- rollback
# tag name need: 20.11.21.01
variables:
namecb: "flask-test"
svcport: "5000"
replicanum: "2"
ingress: "flask-test.boge.com"
certname: "mytls"
CanarylIngressNum: "20"
.deploy_k8s: &deploy_k8s |
if [ $CANARY_CB -eq 1 ];then cp -arf .project-name-canary.yaml ${namecb}-${CI_COMMIT_TAG}.yaml; sed -ri "s+CanarylIngressNum+${CanarylIngressNum}+g" ${namecb}-${CI_COMMIT_TAG}.yaml; sed -ri "s+NomalIngressNum+$(expr 100 - ${CanarylIngressNum})+g" ${namecb}-${CI_COMMIT_TAG}.yaml ;else cp -arf .project-name.yaml ${namecb}-${CI_COMMIT_TAG}.yaml;fi
sed -ri "s+projectnamecb.boge.com+${ingress}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+projectnamecb+${namecb}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+5000+${svcport}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+replicanum+${replicanum}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+mytls+${certname}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+mytagcb+${CI_COMMIT_TAG}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
sed -ri "s+harbor.boge.com/library+${IMG_URL}+g" ${namecb}-${CI_COMMIT_TAG}.yaml
cat ${namecb}-${CI_COMMIT_TAG}.yaml
[ -d ~/.kube ] || mkdir ~/.kube
echo "$KUBE_CONFIG" > ~/.kube/config
if [ $NORMAL_CB -eq 1 ];then if kubectl get deployments.|grep -w ${namecb}-canary &>/dev/null;then kubectl delete deployments.,svc ${namecb}-canary ;fi;fi
kubectl apply -f ${namecb}-${CI_COMMIT_TAG}.yaml --record
echo
echo
echo "============================================================="
echo " Rollback Indx List"
echo "============================================================="
kubectl rollout history deployment ${namecb}|tail -5|awk -F"[ =]+" '{print $1"\t"$5}'|sed '$d'|sed '$d'|sort -r|awk '{print $NF}'|awk '$0=""NR". "$0'
.rollback_k8s: &rollback_k8s |
[ -d ~/.kube ] || mkdir ~/.kube
echo "$KUBE_CONFIG" > ~/.kube/config
last_version_command=$( kubectl rollout history deployment ${namecb}|tail -5|awk -F"[ =]+" '{print $1"\t"$5}'|sed '$d'|sed '$d'|tail -${ROLL_NUM}|head -1 )
last_version_num=$( echo ${last_version_command}|awk '{print $1}' )
last_version_name=$( echo ${last_version_command}|awk '{print $2}' )
kubectl rollout undo deployment ${namecb} --to-revision=$last_version_num
echo $last_version_num
echo $last_version_name
kubectl rollout history deployment ${namecb}
build:
stage: build
retry: 2
variables:
# use dind.yaml to depoy dind'service on k8s
DOCKER_HOST: tcp://10.68.86.33:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
##services:
##- docker:dind
before_script:
- docker login ${REGISTRY_URL} -u "$DOCKER_USER" -p "$DOCKER_PASS"
script:
- docker pull ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:latest || true
- docker build --network host --cache-from ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:latest --tag ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:$CI_COMMIT_TAG --tag ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:latest .
- docker push ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:$CI_COMMIT_TAG
- docker push ${REGISTRY_URL}/${REGISTRY_NS}/${namecb}:latest
after_script:
- docker logout ${REGISTRY_URL}
tags:
- "docker"
only:
- tags
#--------------------------K8S DEPLOY--------------------------------------------------
BOGE-deploy:
stage: deploy
image: harbor.boge.com/library/kubectl:v1.19.9
variables:
KUBE_CONFIG: "$KUBE_CONFIG_TEST"
IMG_URL: "${REGISTRY_URL}/${REGISTRY_NS}"
NORMAL_CB: 1
script:
- *deploy_k8s
when: manual
only:
- tags
# canary start
BOGE-canary-deploy:
stage: deploy
image: harbor.boge.com/library/kubectl:v1.19.9
variables:
KUBE_CONFIG: "$KUBE_CONFIG_TEST"
IMG_URL: "${REGISTRY_URL}/${REGISTRY_NS}"
CANARY_CB: 1
script:
- *deploy_k8s
when: manual
only:
- tags
# canary end
BOGE-rollback-1:
stage: rollback
image: harbor.boge.com/library/kubectl:v1.19.9
variables:
KUBE_CONFIG: "$KUBE_CONFIG_TEST"
ROLL_NUM: 1
script:
- *rollback_k8s
when: manual
only:
- tags
BOGE-rollback-2:
stage: rollback
image: harbor.boge.com/library/kubectl:v1.19.9
variables:
KUBE_CONFIG: "$KUBE_CONFIG_TEST"
ROLL_NUM: 2
script:
- *rollback_k8s
when: manual
only:
- tags
BOGE-rollback-3:
stage: rollback
image: harbor.boge.com/library/kubectl:v1.19.9
variables:
KUBE_CONFIG: "$KUBE_CONFIG_TEST"
ROLL_NUM: 3
script:
- *rollback_k8s
when: manual
only:
- tags
准备k8s的deployment模板文件 .project-name.yaml
这里要注意提前在K8S把harbor拉取的凭证secret给创建好，命令如下：
kubectl -n test create secret docker-registry boge-secret --docker-server=harbor.boge.com --docker-username=admin --docker-password=boge666 --docker-email=admin@boge.com
---
# SVC
kind: Service
apiVersion: v1
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb
kae-type: app
name: projectnamecb
spec:
selector:
kae: "true"
kae-app-name: projectnamecb
kae-type: app
ports:
- name: http-port
port: 80
protocol: TCP
targetPort: 5000
# nodePort: 12345
# type: NodePort
---
# Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb
kae-type: app
name: projectnamecb
spec:
tls:
- hosts:
- projectnamecb.boge.com
secretName: mytls
rules:
- host: projectnamecb.boge.com
http:
paths:
- path: /
backend:
serviceName: projectnamecb
servicePort: 80
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: projectnamecb
labels:
kae: "true"
kae-app-name: projectnamecb
kae-type: app
spec:
replicas: replicanum
selector:
matchLabels:
kae-app-name: projectnamecb
template:
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb
kae-type: app
spec:
containers:
- name: projectnamecb
image: harbor.boge.com/library/projectnamecb:mytagcb
env:
- name: TZ
value: Asia/Shanghai
ports:
- containerPort: 5000
readinessProbe:
httpGet:
scheme: HTTP
path: /
port: 5000
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 3
livenessProbe:
httpGet:
scheme: HTTP
path: /
port: 5000
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 3
resources:
requests:
cpu: 0.3
memory: 0.5Gi
limits:
cpu: 0.3
memory: 0.5Gi
imagePullSecrets:
- name: boge-secret
准备好K8S上金丝雀部署的模板文件 .project-name-canary.yaml
---
# SVC
kind: Service
apiVersion: v1
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb-canary
kae-type: app
name: projectnamecb-canary
spec:
selector:
kae: "true"
kae-app-name: projectnamecb-canary
kae-type: app
ports:
- name: http-port
port: 80
protocol: TCP
targetPort: 5000
# nodePort: 12345
# type: NodePort
---
# Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb-canary
kae-type: app
name: projectnamecb
annotations:
nginx.ingress.kubernetes.io/service-weight: |
projectnamecb: NomalIngressNum, projectnamecb-canary: CanarylIngressNum
spec:
tls:
- hosts:
- projectnamecb.boge.com
secretName: mytls
rules:
- host: projectnamecb.boge.com
http:
paths:
- path: /
backend:
serviceName: projectnamecb
servicePort: 80
- path: /
backend:
serviceName: projectnamecb-canary
servicePort: 80
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: projectnamecb-canary
labels:
kae: "true"
kae-app-name: projectnamecb-canary
kae-type: app
spec:
replicas: replicanum
selector:
matchLabels:
kae-app-name: projectnamecb-canary
template:
metadata:
labels:
kae: "true"
kae-app-name: projectnamecb-canary
kae-type: app
spec:
containers:
- name: projectnamecb-canary
image: harbor.boge.com/library/projectnamecb:mytagcb
env:
- name: TZ
value: Asia/Shanghai
ports:
- containerPort: 5000
readinessProbe:
httpGet:
scheme: HTTP
path: /
port: 5000
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 3
livenessProbe:
httpGet:
scheme: HTTP
path: /
port: 5000
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 3
resources:
requests:
cpu: 0.3
memory: 0.5Gi
limits:
cpu: 0.3
memory: 0.5Gi
imagePullSecrets:
- name: boge-secret
第15关 k8s架构师课程之CICD自动化devops大结局视频
最后，在修改完代码，提交tag版本号后，即会触发CI/CD自动化流程，详细操作可以看博哥录制的同名视频教程好了。
好啦，到这里为止整个历时40多天的K8S架构师课程也就告一个段落了，希望本套课程能对的大家有所帮助，如果你觉得博哥视频教程不错的话，请分享给你的朋友，让更多人能掌握K8S，掌握CI/CD自动化。