CI/CD DevOps 自动化实践指南(持续更新)
前言
CI/CD 是什么?
想象一下:你要把一篇写好的文章发布到博客网站。传统方式:
- 手动检查错别字(代码检查)
- 手动排版(构建)
- 手动上传到网站(部署)
这太麻烦了!如果你写完文章后,按一个按钮,以上步骤全部自动完成,那该多好!
CI/CD 就是这个"自动发布按钮"! 它让代码从写好到上线,全部自动化。
DevOps 是一种文化和方法论,CI/CD 是 DevOps 的核心技术实践。在现代软件开发中,CI/CD 已成为标配,它能大幅提升交付效率、减少人为错误。
本文档会告诉你:
- CI/CD 核心概念
- GitLab CI 详解
- Jenkins 详解
- GitHub Actions 详解
- Kubernetes 部署集成
- 流水线最佳实践
目录
核心概念详解
1. CI/CD 基本概念
CI(持续集成)是什么?
想象一下:多人一起写一本书。每次有人写完一章,就自动检查:错别字、格式问题、逻辑错误。这样就能保证书的质量。
CI 就是代码的"自动检查员",每次代码提交都自动运行测试和构建。
CD(持续交付/部署)是什么?
想象一下:书检查没问题后,自动排版、自动上传到网站。读者马上就能看到新内容。
CD 就是代码的"自动发布员",把通过检查的代码自动部署到服务器。
CI/CD 流程图:
┌─────────────────────────────────────────────────────────────┐
│ CI/CD 完整流程 │
│ │
│ 代码提交 → 编译构建 → 单元测试 → 集成测试 → 镜像构建 → 部署测试 → 上线 │
│ │ │ │ │ │ │ │
│ ▼ ▼ ▼ ▼ ▼ ▼ │
│ Git Hook Maven JUnit Postman Docker K8s │
│ 自动触发 Gradle pytest Newman Build Deploy │
└─────────────────────────────────────────────────────────────┘
2. 核心概念速查
| 概念 | 通俗解释 | 类比 |
|---|---|---|
| CI | 持续集成,每次提交自动检查 | 出版社的初审 |
| CD | 持续交付,自动准备好发布 | 自动排版待发布 |
| CDE | 持续部署,自动发布到生产 | 自动上线 |
| Pipeline | 流水线,定义自动化流程 | 生产流水线 |
| Stage | 阶段,流水线的步骤 | 生产线的工站 |
| Job | 任务,具体的操作 | 工人执行的任务 |
| Artifact | 制品,构建产出的文件 | 流水线产出的产品 |
| Trigger | 触发器,触发流水线的事件 | 生产线的启动按钮 |
3. DevOps 工具生态
┌─────────────────────────────────────────────────────────────┐
│ DevOps 工具生态 │
│ │
│ 规划 ───▶ 代码 ───▶ 构建 ───▶ 测试 ───▶ 部署 ───▶ 监控 │
│ │ │ │ │ │ │ │
│ Jira GitLab Maven JUnit K8s Prometheus │
│ Jira GitHub Gradle Pytest Helm Grafana │
│ Trello Gitea Docker Postman Ansible Jaeger │
│ │
│ CI/CD 工具: GitLab CI, Jenkins, GitHub Actions, ArgoCD │
│ 容器: Docker, Podman, containerd │
│ 编排: Kubernetes, Docker Swarm │
│ 配置管理: Ansible, Terraform, Puppet │
│ 监控: Prometheus, Grafana, ELK │
└─────────────────────────────────────────────────────────────┘
GitLab CI 详解
GitLab CI 是什么?
GitLab CI 是什么?
想象一下:GitLab 是一个"代码图书馆",CI 就是图书馆里的"自动服务机器人"。每次有人还书或借书,机器人自动检查书有没有损坏。
GitLab CI 集成在 GitLab 中,是最流行的 CI/CD 工具之一。
核心组件:
| 组件 | 说明 |
|---|---|
| GitLab Runner | 执行 CI 任务的代理 |
| .gitlab-ci.yml | 流水线配置文件 |
| Pipeline | 完整的流水线 |
| Stage | 流水线中的阶段 |
| Job | 具体任务 |
.gitlab-ci.yml 基础配置
# 👀 .gitlab-ci.yml 基础示例
# 👀 定义流水线 Stages
stages:
- build # 构建阶段
- test # 测试阶段
- deploy # 部署阶段
# 👀 定义变量
variables:
DOCKER_IMAGE: registry.example.com/app
DOCKER_TAG: $CI_COMMIT_SHORT_SHA
# 👀 构建阶段任务
build:
stage: build
image: maven:3.8-openjdk-17
script:
- mvn clean package -DskipTests
artifacts:
paths:
- target/*.jar
expire_in: 1 hour
# 👀 测试阶段任务
test-unit:
stage: test
image: maven:3.8-openjdk-17
script:
- mvn test
coverage: '/Total:.*?([0-9]{1,3})%/'
test-integration:
stage: test
image: maven:3.8-openjdk-17
script:
- mvn verify -Dspring.profiles.active=test
services:
- postgres:14
# 👀 部署阶段任务
deploy-staging:
stage: deploy
script:
- echo "Deploying to staging..."
- kubectl config set-cluster staging --server=$K8S_STAGING
- kubectl config set-credentials deploy --token=$K8S_TOKEN
- kubectl apply -f k8s/
environment:
name: staging
url: https://staging.example.com
only:
- develop
deploy-production:
stage: deploy
script:
- echo "Deploying to production..."
- kubectl config set-cluster prod --server=$K8S_PROD
- kubectl apply -f k8s/
environment:
name: production
url: https://example.com
when: manual # 手动触发
only:
- main
完整流水线示例
# 👀 完整的 Java Spring Boot 项目流水线
stages:
- build
- test
- analyze
- build-image
- deploy
# 👀 全局变量
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
DOCKER_DRIVER: overlay2
# 👀 Maven 缓存
cache:
paths:
- .m2/repository
- target/
# 👀 阶段1:构建
build:
stage: build
image: maven:3.8-openjdk-17
script:
- mvn clean package -DskipTests=false
artifacts:
name: ${CI_PROJECT_NAME}-${CI_COMMIT_SHORT_SHA}
paths:
- target/*.jar
expire_in: 7 days
tags:
- docker
# 👀 阶段2:单元测试
test-unit:
stage: test
image: maven:3.8-openjdk-17
script:
- mvn test
coverage: '/([0-9]{1,3})%/'
reports:
junit: target/surefire-reports/TEST-*.xml
coverage_report:
coverage_format: cobertura
path: target/site/cobertura/coverage.xml
tags:
- docker
# 👀 阶段3:代码分析
sonarqube-check:
stage: analyze
image: sonarsource/sonar-scanner-cli:latest
script:
- sonar-scanner
-Dsonar.host.url=$SONAR_HOST_URL
-Dsonar.projectKey=$CI_PROJECT_NAME
-Dsonar.java.binaries=target/classes
variables:
SONAR_USER_HOME: ${CI_PROJECT_DIR}/.sonar
cache:
key: "${CI_PROJECT_PATH}"
paths:
- .sonar/cache
allow_failure: true
tags:
- docker
# 👀 阶段4:构建 Docker 镜像
build-image:
stage: build-image
image: docker:latest
services:
- docker:dind
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
- docker build -t $CI_REGISTRY_IMAGE:latest .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
only:
- main
- develop
tags:
- docker
# 👀 阶段5:部署到 Kubernetes
deploy-staging:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl config set-cluster staging --server=$K8S_STAGING_URL
- kubectl config set-credentials deploy --token=$K8S_STAGING_TOKEN
- kubectl config set-context staging --cluster=staging --user=deploy
- kubectl config use-context staging
- kubectl set image deployment/app app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -n staging
- kubectl rollout status deployment/app -n staging
environment:
name: staging
url: https://staging.example.com
only:
- develop
tags:
- kubernetes
deploy-production:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl config set-cluster prod --server=$K8S_PROD_URL
- kubectl config set-credentials deploy --token=$K8S_PROD_TOKEN
- kubectl config set-context prod --cluster=prod --user=deploy
- kubectl config use-context prod
- kubectl set image deployment/app app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -n production
- kubectl rollout status deployment/app -n production
- kubectl create deployment app-old --image=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -n production --dry-run=server -o yaml | kubectl apply -f -
environment:
name: production
url: https://example.com
when: manual
only:
- main
tags:
- kubernetes
GitLab Runner 配置
# 👀 安装 GitLab Runner
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner
# 👀 注册 Runner
sudo gitlab-runner register \
--url "https://gitlab.example.com/" \
--registration-token "xxxxx" \
--description "docker-runner" \
--tag-list "docker" \
--executor "docker" \
--docker-image "docker:latest"
# 👀 查看 Runner 状态
sudo gitlab-runner list
# 👀 Runner 配置示例(Docker Machine)
[[runners]]
name = "docker-runner"
url = "https://gitlab.example.com"
token = "xxxxx"
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:latest"
privileged = true
disable_cache = false
volumes = ["/cache"]
[runners.machine]
MachineDriver = "digitalocean"
MachineSize = "s-2vcpu-4gb"
Autoscaling = true
MaxBuilds = 10
Jenkins 详解
Jenkins 是什么?
Jenkins 是什么?
想象一下:Jenkins 是一个"万能装修队"。你告诉它要做什么(安装地板、粉刷墙壁),它自动完成所有工作,还能同时装修多套房子。
Jenkins 是最流行的 CI/CD 服务器,插件丰富,几乎能满足所有自动化需求。
Jenkins 架构:
┌─────────────────────────────────────────────────────────────┐
│ Jenkins 架构 │
│ │
│ ┌─────────────┐ │
│ │ Jenkins │ │
│ │ Master │ │
│ │ (调度中心) │ │
│ └──────┬──────┘ │
│ │ │
│ SSH/WMI/RPC │
│ │ │
│ ┌──────┴──────┐ │
│ │ Agents │ │
│ │ (执行节点) │ │
│ └──────┬──────┘ │
│ │ │
│ ┌──────┴──────┐ │
│ │ Nodes │ │
│ │ (构建环境) │ │
└─────────────────────────────────────────────────────────────┘
Jenkins 安装
# 👀 Docker 方式安装(推荐)
docker run -d \
--name jenkins \
-p 8080:8080 \
-p 50000:50000 \
-v jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
jenkins/jenkins:lts
# 👀 查看初始密码
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword
# 👀 或者 apt 安装
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins
Jenkinsfile 示例
// 👀 Jenkinsfile (Declarative Pipeline)
pipeline {
agent {
docker {
image 'maven:3.8-openjdk-17'
args '-v /root/.m2:/root/.m2'
}
}
environment {
DOCKER_IMAGE = 'registry.example.com/app'
DOCKER_TAG = "${env.BUILD_NUMBER}"
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Build') {
steps {
sh 'mvn clean package -DskipTests=false'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit 'target/surefire-reports/*.xml'
cobertura coberturaReportFile: 'target/site/cobertura/coverage.xml'
}
}
}
stage('Security Scan') {
steps {
sh 'mvn dependency:tree'
sh 'trivy image --severity HIGH,CRITICAL $DOCKER_IMAGE:$DOCKER_TAG || true'
}
}
stage('Build Image') {
steps {
script {
def image = docker.build("${DOCKER_IMAGE}:${DOCKER_TAG}")
docker.withRegistry('https://registry.example.com', 'docker-registry') {
image.push()
image.push('latest')
}
}
}
}
stage('Deploy to Staging') {
when {
branch 'develop'
}
steps {
sh '''
kubectl config use-context staging
kubectl set image deployment/app app=${DOCKER_IMAGE}:${DOCKER_TAG}
kubectl rollout status deployment/app -n staging
'''
}
}
stage('Deploy to Production') {
when {
branch 'main'
}
steps {
input message: 'Deploy to Production?', ok: 'Deploy'
sh '''
kubectl config use-context production
kubectl set image deployment/app app=${DOCKER_IMAGE}:${DOCKER_TAG}
kubectl rollout status deployment/app -n production
'''
}
}
}
post {
always {
cleanWs()
}
success {
echo 'Pipeline succeeded!'
}
failure {
echo 'Pipeline failed!'
}
}
}
GitHub Actions 详解
GitHub Actions 是什么?
GitHub Actions 是什么?
想象一下:GitHub 是一个"代码出版社",Actions 就是出版社的"自动编辑系统"。作者提交书稿后,系统自动检查、排版、发布。
GitHub Actions 集成在 GitHub 中,免费额度充足,是开源项目的首选。
核心概念:
| 概念 | 说明 |
|---|---|
| Workflow | 完整的工作流程 |
| Job | 任务(由多个 Step 组成) |
| Step | 具体步骤(可以是命令或 Action) |
| Action | 可复用的 Action |
workflow 示例
# 👀 .github/workflows/ci.yml
name: CI/CD Pipeline
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
env:
DOCKER_IMAGE: ghcr.io/${{ github.repository }}
JAVA_VERSION: '17'
jobs:
# 👀 构建 job
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK ${{ env.JAVA_VERSION }}
uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: 'temurin'
cache: 'maven'
- name: Build with Maven
run: mvn clean package -DskipTests=false
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: jar-file
path: target/*.jar
retention-days: 7
# 👀 测试 job
test:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4
- name: Set up JDK
uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: 'temurin'
cache: 'maven'
- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: jar-file
path: target/
- name: Run tests
run: mvn test
- name: Upload test results
uses: actions/upload-artifact@v4
if: always()
with:
name: test-results
path: target/surefire-reports/*.xml
- name: Publish Test Report
uses: scalamando/phony-action@v1
if: always()
with:
files: target/surefire-reports/*.xml
# 👀 构建 Docker 镜像
docker:
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/main'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ env.DOCKER_IMAGE }}:${{ github.sha }}, ${{ env.DOCKER_IMAGE }}:latest
cache-from: type=registry,ref=${{ env.DOCKER_IMAGE }}:latest
cache-to: type=inline
# 👀 部署到 Kubernetes
deploy:
runs-on: ubuntu-latest
needs: docker
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Set up kubectl
uses: azure/setup-kubectl@v3
- name: Configure kubectl
run: |
echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig
echo "KUBECONFIG=$(pwd)/kubeconfig" >> $GITHUB_ENV
- name: Deploy to production
run: |
kubectl set image deployment/app app=${{ env.DOCKER_IMAGE }}:${{ github.sha }}
kubectl rollout status deployment/app -n production
Kubernetes 集成
ArgoCD 概述
ArgoCD 是什么?
想象一下:Kubernetes 是一个"自动装修系统",ArgoCD 就像装修系统的"遥控器"。你按一下按钮,它自动把新版本部署到所有房间。
ArgoCD 是 GitOps 的实现工具,能自动同步 Git 仓库中的配置到 Kubernetes。
ArgoCD 架构:
┌─────────────────────────────────────────────────────────────┐
│ ArgoCD 架构 │
│ │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Git Repo │────▶│ ArgoCD │ │
│ │ (配置存储) │ │ Server │ │
│ └─────────────┘ └──────┬──────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ Kubernetes │ │
│ │ Cluster │ │
│ └─────────────┘ │
└─────────────────────────────────────────────────────────────┘
ArgoCD 安装
# 👀 安装 ArgoCD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# 👀 安装 CLI
curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x /usr/local/bin/argocd
# 👀 获取初始密码
argocd admin initial-password -n argocd
# 👀 端口转发访问
kubectl port-forward svc/argocd-server -n argocd 8080:443
ArgoCD Application 配置
# 👀 argocd-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/example/myapp.git
targetRevision: HEAD
path: k8s/overlays/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
流水线最佳实践
分支策略
| 策略 | 说明 | 适用场景 |
|---|---|---|
| GitFlow | main/develop/feature/hotfix 分支 | 大型项目 |
| GitHub Flow | main + feature | 简单项目 |
| Trunk-based | 所有开发者基于 main 开发 | 敏捷团队 |
流水线优化
# 👀 优化1:并行执行
stages:
- test
test:
parallel:
matrix:
- JDK_VERSION: [17, 21]
DATABASE: [mysql8, postgres15]
# 👀 优化2:依赖缓存
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .m2/repository
- node_modules/
- build/
# 👀 优化3:失败重试
deploy:
script:
- kubectl rollout status deployment/app
retry:
max: 3
when:
- runner_system_failure
- stuck_or_timeout_failure
安全最佳实践
# 👀 1. 使用 Secret 管理敏感信息
env:
DOCKER_REGISTRY_PASSWORD: ${{ secrets.DOCKER_REGISTRY_PASSWORD }}
# 👀 2. 扫描依赖漏洞
- name: Scan dependencies
uses: snyk/actions/node@master
with:
args: --severity-threshold=high
# 👀 3. 不在日志中输出敏感信息
script:
- echo "Deploying version $VERSION" # 不输出密码
实战案例
案例:Spring Boot 项目完整 CI/CD
项目结构:
myapp/
├── src/
│ └── main/
├── k8s/
│ ├── base/
│ │ ├── deployment.yaml
│ │ └── service.yaml
│ └── overlays/
│ ├── staging/
│ │ └── kustomization.yaml
│ └── production/
│ └── kustomization.yaml
├── Dockerfile
├── Jenkinsfile
└── pom.xml
Kubernetes 配置:
# 👀 k8s/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
Jenkinsfile:
// 👀 完整 Jenkinsfile
pipeline {
agent {
label 'docker'
}
options {
buildDiscarder logRotator(numToKeepStr: '10')
timeout(time: 30, unit: 'MINUTES')
}
environment {
DOCKER_IMAGE = "registry.example.com/myapp"
DOCKER_TAG = "${env.BUILD_NUMBER}"
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Build') {
steps {
sh 'mvn clean package -DskipTests'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit 'target/surefire-reports/**/*.xml'
}
}
}
stage('Security Scan') {
steps {
sh 'mvn dependency:analyze -DfailBuild=false'
}
}
stage('Build Docker') {
steps {
script {
def image = docker.build("${DOCKER_IMAGE}:${DOCKER_TAG}")
docker.withRegistry('https://registry.example.com', 'docker-hub') {
image.push()
image.push('latest')
}
}
}
}
stage('Deploy to Staging') {
when {
branch 'develop'
}
steps {
sh '''
kubectl config use-context staging
kubectl set image deployment/myapp myapp=${DOCKER_IMAGE}:${DOCKER_TAG}
kubectl rollout status deployment/myapp -n staging
'''
}
}
stage('Deploy to Production') {
when {
branch 'main'
}
steps {
input message: 'Ready to deploy to production?', ok: 'Deploy'
sh '''
kubectl config use-context production
kubectl set image deployment/myapp myapp=${DOCKER_IMAGE}:${DOCKER_TAG}
kubectl rollout status deployment/myapp -n production
'''
}
}
}
post {
success {
echo 'CI/CD Pipeline completed successfully!'
}
failure {
echo 'CI/CD Pipeline failed!'
}
}
}
总结
工具选择建议
| 场景 | 推荐工具 |
|---|---|
| GitHub 仓库 | GitHub Actions |
| GitLab 仓库 | GitLab CI |
| 多仓库统一管理 | Jenkins |
| GitOps 部署 | ArgoCD |
| 企业级全链路 | GitLab + ArgoCD + Jenkins |
黄金法则
- 提交触发 - 代码提交即触发 CI
- 快速反馈 - CI 流程控制在 10 分钟内
- 失败停止 - 测试失败立即停止流水线
- 制品追溯 - 每个版本都可回溯
- 安全扫描 - 集成安全检查
- GitOps - 基础设施代码化
持续更新中… 如有问题或建议,欢迎交流讨论!
评论区