第11章 分布式构建
11.1 分布式构建概述
什么是分布式构建
分布式构建定义:
Jenkins分布式构建是指将构建任务分散到多个节点(Agent)上执行的架构模式。
这种模式可以提高构建效率、资源利用率和系统可扩展性。核心概念:
- Controller(控制器):Jenkins主节点,负责调度和管理
- Agent(代理):执行构建任务的工作节点
- Executor(执行器):Agent上的并发执行单元
- Label(标签):用于标识和选择特定类型的Agent
- Node(节点):Controller和Agent的统称
分布式构建优势:
性能优势:
- 并行执行:多个构建任务同时运行
- 负载分散:避免单点性能瓶颈
- 资源隔离:不同项目使用独立资源
- 弹性扩展:根据需求动态增减节点管理优势:
- 环境隔离:不同环境的构建互不影响
- 专用资源:特定任务使用专门配置的节点
- 故障隔离:单个节点故障不影响整体
- 成本优化:按需使用云资源
架构模式:
经典架构:
┌─────────────────┐
│ Controller │ ← 主节点(调度、管理、UI)
│ (Master) │
└─────────┬───────┘│┌─────┴─────┐│ │
┌───▼───┐ ┌───▼───┐
│Agent 1│ │Agent 2│ ← 工作节点(执行构建)
└───────┘ └───────┘现代架构:
┌─────────────────┐
│ Controller │ ← 无状态控制器
└─────────┬───────┘│┌─────┴─────┐│ │
┌───▼───┐ ┌───▼───┐
│ Pod 1 │ │ Pod 2 │ ← Kubernetes Pod
└───────┘ └───────┘
节点类型和特性
Controller节点:
职责:
- 用户界面服务
- 构建调度和分发
- 插件管理
- 系统配置管理
- 构建历史存储
- 安全认证和授权特性:
- 通常不执行构建任务
- 需要持久化存储
- 高可用性要求
- 网络连通性要求
Agent节点:
类型分类:1. 永久Agent(Permanent Agent)- 长期运行的物理机或虚拟机- 稳定的网络连接- 预配置的构建环境- 适合频繁构建的项目2. 云Agent(Cloud Agent)- 按需创建和销毁- 弹性扩展能力- 成本优化- 适合间歇性构建3. 容器Agent(Container Agent)- 基于Docker或Kubernetes- 快速启动和清理- 环境一致性- 资源隔离4. 静态Agent(Static Agent)- 手动配置和管理- 固定资源分配- 简单可靠- 适合小规模环境
11.2 Agent配置与管理
添加永久Agent
通过SSH连接:
配置步骤:1. 准备Agent机器- 安装Java运行环境- 配置SSH服务- 创建Jenkins用户- 设置工作目录权限2. 在Jenkins中添加节点- 管理Jenkins -> 节点管理 -> 新建节点- 节点名称:agent-linux-01- 类型:永久代理3. 节点配置- 远程工作目录:/home/jenkins/workspace- 标签:linux java maven- 用法:尽可能使用这个节点- 启动方式:通过SSH启动代理- 主机:192.168.1.100- 凭据:jenkins-ssh-key- 主机密钥验证策略:已知主机文件验证策略
SSH Agent配置示例:
# 1. 在Agent机器上创建Jenkins用户
sudo useradd -m -s /bin/bash jenkins
sudo mkdir -p /home/jenkins/.ssh
sudo mkdir -p /home/jenkins/workspace# 2. 配置SSH密钥认证
# 在Controller上生成密钥对
ssh-keygen -t rsa -b 4096 -f jenkins-agent-key# 3. 将公钥复制到Agent
sudo cp jenkins-agent-key.pub /home/jenkins/.ssh/authorized_keys
sudo chown -R jenkins:jenkins /home/jenkins
sudo chmod 700 /home/jenkins/.ssh
sudo chmod 600 /home/jenkins/.ssh/authorized_keys# 4. 测试SSH连接
ssh -i jenkins-agent-key jenkins@192.168.1.100
通过JNLP连接:
配置步骤:1. 创建JNLP节点- 启动方式:通过Java Web Start启动代理- 或:让Jenkins控制这个Windows从节点2. 下载agent.jarwget http://jenkins-server:8080/jnlpJars/agent.jar3. 启动Agentjava -jar agent.jar -jnlpUrl http://jenkins-server:8080/computer/agent-name/slave-agent.jnlp -secret <secret-key>4. 创建启动脚本#!/bin/bashJENKINS_URL="http://jenkins-server:8080"AGENT_NAME="agent-linux-02"SECRET="your-secret-here"java -jar agent.jar \-jnlpUrl "${JENKINS_URL}/computer/${AGENT_NAME}/slave-agent.jnlp" \-secret "${SECRET}" \-workDir "/home/jenkins/workspace"
Windows Agent配置
Windows服务方式:
# 1. 下载并安装Jenkins Agent
$jenkinsUrl = "http://jenkins-server:8080"
$agentName = "windows-agent-01"
$secret = "your-secret-here"# 2. 下载agent.jar
Invoke-WebRequest -Uri "$jenkinsUrl/jnlpJars/agent.jar" -OutFile "agent.jar"# 3. 创建启动脚本
@"
java -jar agent.jar ^-jnlpUrl "$jenkinsUrl/computer/$agentName/slave-agent.jnlp" ^-secret "$secret" ^-workDir "C:\Jenkins\workspace"
"@ | Out-File -FilePath "start-agent.bat" -Encoding ASCII# 4. 安装为Windows服务
# 使用WinSW或NSSM工具
nssm install JenkinsAgent "C:\Program Files\Java\jdk-11\bin\java.exe"
nssm set JenkinsAgent Parameters "-jar C:\Jenkins\agent.jar -jnlpUrl $jenkinsUrl/computer/$agentName/slave-agent.jnlp -secret $secret"
nssm set JenkinsAgent AppDirectory "C:\Jenkins"
nssm start JenkinsAgent
PowerShell DSC配置:
Configuration JenkinsAgent {param([string]$JenkinsUrl,[string]$AgentName,[string]$Secret)Import-DscResource -ModuleName PSDesiredStateConfigurationNode localhost {# 创建Jenkins目录File JenkinsDirectory {DestinationPath = "C:\Jenkins"Type = "Directory"Ensure = "Present"}# 下载agent.jarScript DownloadAgent {SetScript = {Invoke-WebRequest -Uri "$using:JenkinsUrl/jnlpJars/agent.jar" -OutFile "C:\Jenkins\agent.jar"}TestScript = {Test-Path "C:\Jenkins\agent.jar"}GetScript = {@{ Result = (Test-Path "C:\Jenkins\agent.jar") }}DependsOn = "[File]JenkinsDirectory"}# 创建启动脚本File StartScript {DestinationPath = "C:\Jenkins\start-agent.bat"Contents = @"
java -jar C:\Jenkins\agent.jar ^-jnlpUrl "$JenkinsUrl/computer/$AgentName/slave-agent.jnlp" ^-secret "$Secret" ^-workDir "C:\Jenkins\workspace"
"@Type = "File"Ensure = "Present"DependsOn = "[Script]DownloadAgent"}}
}# 应用配置
JenkinsAgent -JenkinsUrl "http://jenkins-server:8080" -AgentName "windows-agent-01" -Secret "your-secret"
Start-DscConfiguration -Path .\JenkinsAgent -Wait -Verbose
标签和节点选择
标签策略:
标签分类:1. 操作系统标签- linux, windows, macos- ubuntu, centos, rhel- windows-2019, windows-20222. 架构标签- x86_64, arm64, aarch64- amd64, i3863. 软件环境标签- java-8, java-11, java-17- maven, gradle, npm- docker, kubernetes- python-3.8, node-164. 硬件资源标签- high-memory, high-cpu- gpu-enabled- ssd-storage5. 功能特性标签- build, test, deploy- security-scan- performance-test6. 环境标签- dev, staging, prod- internal, external- trusted, untrusted
Pipeline中的节点选择:
pipeline {agent nonestages {stage('Build') {agent {label 'linux && java-11 && maven'}steps {sh 'mvn clean compile'}}stage('Test') {parallel {stage('Unit Tests') {agent {label 'linux && java-11'}steps {sh 'mvn test'}}stage('Integration Tests') {agent {label 'linux && docker'}steps {sh 'docker-compose up -d'sh 'mvn verify -P integration-tests'sh 'docker-compose down'}}stage('Performance Tests') {agent {label 'high-memory && performance-test'}steps {sh 'jmeter -n -t performance-test.jmx'}}}}stage('Security Scan') {agent {label 'security-scan && trusted'}steps {sh 'sonar-scanner'sh 'dependency-check.sh'}}stage('Deploy') {agent {label 'deploy && prod'}when {branch 'main'}steps {sh 'kubectl apply -f k8s/'}}}
}
动态标签选择:
pipeline {agent nonestages {stage('Dynamic Agent Selection') {steps {script {// 根据条件动态选择Agentdef agentLabel = ''if (env.BRANCH_NAME == 'main') {agentLabel = 'prod && deploy'} else if (env.BRANCH_NAME.startsWith('feature/')) {agentLabel = 'dev && build'} else if (env.BRANCH_NAME.startsWith('release/')) {agentLabel = 'staging && test'}// 根据文件变化选择Agentdef changes = currentBuild.changeSetsdef hasDockerfile = falsedef hasJavaFiles = falsechanges.each { changeSet ->changeSet.each { change ->change.affectedFiles.each { file ->if (file.path.contains('Dockerfile')) {hasDockerfile = true}if (file.path.endsWith('.java')) {hasJavaFiles = true}}}}if (hasDockerfile) {agentLabel += ' && docker'}if (hasJavaFiles) {agentLabel += ' && java-11'}echo "选择的Agent标签: ${agentLabel}"// 使用选择的Agent执行构建node(agentLabel) {checkout scmsh 'echo "在Agent ${NODE_NAME} 上执行构建"'// 执行构建步骤}}}}}
}
11.3 云Agent配置
AWS EC2 Agent
EC2插件配置:
插件安装:
- 安装 "Amazon EC2" 插件
- 管理Jenkins -> 系统配置 -> Cloud配置步骤:
1. 添加新的Cloud- 类型:Amazon EC2- 名称:AWS-EC2-Cloud2. AWS配置- Access Key ID:使用IAM凭据- Secret Access Key:使用IAM凭据- Region:us-west-2- EC2 Key Pair's Private Key:上传私钥文件3. AMI配置- AMI ID:ami-0abcdef1234567890- Instance Type:t3.medium- Security group names:jenkins-agents- Remote user:ec2-user- AMI Type:unix- Labels:aws linux docker- Usage:Use this node as much as possible
EC2 Agent模板:
{"amiType": "unix","associatePublicIp": false,"connectBySSHProcess": false,"connectUsingPublicIp": false,"customDeviceMapping": "","deleteRootOnTermination": true,"description": "Jenkins Agent for CI/CD","ebsOptimized": false,"iamInstanceProfile": "jenkins-agent-role","idleTerminationMinutes": "30","initScript": "#!/bin/bash\nyum update -y\nyum install -y docker git\nservice docker start\nusermod -a -G docker ec2-user","instanceCapStr": "10","jvmopts": "-Xmx1024m","labelString": "aws linux docker maven","launchTimeoutStr": "300","numExecutors": "2","remoteAdmin": "ec2-user","remoteFS": "/home/ec2-user/jenkins","securityGroups": "jenkins-agents","stopOnTerminate": false,"subnetId": "subnet-12345678","tags": [{"name": "Name","value": "jenkins-agent"},{"name": "Environment","value": "ci-cd"}],"tmpDir": "/tmp","type": "t3.medium","useEphemeralDevices": false,"userData": ""
}
Terraform配置EC2 Agent:
# variables.tf
variable "jenkins_controller_ip" {description = "Jenkins Controller IP address"type = string
}variable "key_pair_name" {description = "EC2 Key Pair name"type = string
}# main.tf
resource "aws_security_group" "jenkins_agents" {name_prefix = "jenkins-agents-"description = "Security group for Jenkins agents"ingress {from_port = 22to_port = 22protocol = "tcp"cidr_blocks = ["${var.jenkins_controller_ip}/32"]}egress {from_port = 0to_port = 0protocol = "-1"cidr_blocks = ["0.0.0.0/0"]}tags = {Name = "jenkins-agents"}
}resource "aws_iam_role" "jenkins_agent_role" {name = "jenkins-agent-role"assume_role_policy = jsonencode({Version = "2012-10-17"Statement = [{Action = "sts:AssumeRole"Effect = "Allow"Principal = {Service = "ec2.amazonaws.com"}}]})
}resource "aws_iam_role_policy" "jenkins_agent_policy" {name = "jenkins-agent-policy"role = aws_iam_role.jenkins_agent_role.idpolicy = jsonencode({Version = "2012-10-17"Statement = [{Effect = "Allow"Action = ["s3:GetObject","s3:PutObject","ecr:GetAuthorizationToken","ecr:BatchCheckLayerAvailability","ecr:GetDownloadUrlForLayer","ecr:BatchGetImage"]Resource = "*"}]})
}resource "aws_iam_instance_profile" "jenkins_agent_profile" {name = "jenkins-agent-profile"role = aws_iam_role.jenkins_agent_role.name
}resource "aws_launch_template" "jenkins_agent" {name_prefix = "jenkins-agent-"image_id = "ami-0abcdef1234567890" # Amazon Linux 2instance_type = "t3.medium"key_name = var.key_pair_namevpc_security_group_ids = [aws_security_group.jenkins_agents.id]iam_instance_profile {name = aws_iam_instance_profile.jenkins_agent_profile.name}user_data = base64encode(templatefile("${path.module}/user_data.sh", {jenkins_controller_ip = var.jenkins_controller_ip}))tag_specifications {resource_type = "instance"tags = {Name = "jenkins-agent"Environment = "ci-cd"}}
}
用户数据脚本:
#!/bin/bash
# user_data.sh# 更新系统
yum update -y# 安装必要软件
yum install -y java-11-openjdk docker git maven# 启动Docker
systemctl start docker
systemctl enable docker
usermod -a -G docker ec2-user# 安装Node.js
curl -fsSL https://rpm.nodesource.com/setup_16.x | bash -
yum install -y nodejs# 创建Jenkins工作目录
mkdir -p /home/ec2-user/jenkins
chown ec2-user:ec2-user /home/ec2-user/jenkins# 配置SSH
echo "${jenkins_controller_ip} jenkins-controller" >> /etc/hosts# 安装AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install# 配置日志
echo "Jenkins Agent initialized at $(date)" >> /var/log/jenkins-agent.log
Kubernetes Agent
Kubernetes插件配置:
# jenkins-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: jenkinsnamespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: jenkins
rules:
- apiGroups: [""]resources: ["pods"]verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]resources: ["pods/exec"]verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]resources: ["pods/log"]verbs: ["get", "list", "watch"]
- apiGroups: [""]resources: ["secrets"]verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: jenkins
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: jenkins
subjects:
- kind: ServiceAccountname: jenkinsnamespace: jenkins
Jenkins Kubernetes配置:
插件安装:
- 安装 "Kubernetes" 插件
- 管理Jenkins -> 系统配置 -> Cloud配置步骤:
1. 添加Kubernetes Cloud- 名称:kubernetes- Kubernetes URL:https://kubernetes.default.svc.cluster.local- Kubernetes Namespace:jenkins- Credentials:使用ServiceAccount Token2. Pod模板配置- 名称:jenkins-agent- 命名空间:jenkins- 标签:kubernetes docker maven- 用法:Use this node as much as possible
Pipeline中使用Kubernetes Agent:
pipeline {agent {kubernetes {yaml """
apiVersion: v1
kind: Pod
spec:containers:- name: mavenimage: maven:3.8.1-openjdk-11command:- cattty: truevolumeMounts:- name: maven-cachemountPath: /root/.m2- name: dockerimage: docker:20.10.7-dindsecurityContext:privileged: truevolumeMounts:- name: docker-sockmountPath: /var/run/docker.sock- name: kubectlimage: bitnami/kubectl:latestcommand:- cattty: truevolumes:- name: maven-cachehostPath:path: /tmp/maven-cache- name: docker-sockhostPath:path: /var/run/docker.sock
"""}}stages {stage('Build') {steps {container('maven') {sh 'mvn clean compile'}}}stage('Test') {steps {container('maven') {sh 'mvn test'}}}stage('Package') {steps {container('maven') {sh 'mvn package'}}}stage('Build Image') {steps {container('docker') {sh 'docker build -t myapp:${BUILD_NUMBER} .'}}}stage('Deploy') {steps {container('kubectl') {sh 'kubectl apply -f k8s/deployment.yaml'}}}}
}
动态Pod模板:
def createPodTemplate(Map config) {def podYaml = """
apiVersion: v1
kind: Pod
metadata:labels:jenkins: agent
spec:containers:
"""// 添加基础容器podYaml += """- name: jnlpimage: jenkins/inbound-agent:latestargs: ['\$(JENKINS_SECRET)', '\$(JENKINS_NAME)']
"""// 根据配置添加容器if (config.maven) {podYaml += """- name: mavenimage: maven:3.8.1-openjdk-11command: ['cat']tty: truevolumeMounts:- name: maven-cachemountPath: /root/.m2
"""}if (config.docker) {podYaml += """- name: dockerimage: docker:20.10.7-dindsecurityContext:privileged: truevolumeMounts:- name: docker-sockmountPath: /var/run/docker.sock
"""}if (config.node) {podYaml += """- name: nodeimage: node:16-alpinecommand: ['cat']tty: true
"""}// 添加卷podYaml += """volumes:
"""if (config.maven) {podYaml += """- name: maven-cachehostPath:path: /tmp/maven-cache
"""}if (config.docker) {podYaml += """- name: docker-sockhostPath:path: /var/run/docker.sock
"""}return podYaml
}pipeline {agent {kubernetes {yaml createPodTemplate([maven: true,docker: true,node: false])}}stages {stage('Build') {steps {container('maven') {sh 'mvn clean package'}}}stage('Docker Build') {steps {container('docker') {sh 'docker build -t myapp:${BUILD_NUMBER} .'}}}}
}
11.4 负载均衡与调度
构建调度策略
调度算法:
Jenkins调度策略:1. 标签匹配(Label Matching)- 精确匹配:agent标签完全匹配job要求- 子集匹配:agent标签包含job要求的所有标签- 表达式匹配:支持逻辑运算符(&&, ||, !)2. 负载均衡(Load Balancing)- 轮询调度:依次分配到可用节点- 最少连接:分配到当前负载最轻的节点- 加权轮询:根据节点性能分配权重3. 亲和性调度(Affinity Scheduling)- 节点亲和性:优先使用特定节点- 任务亲和性:相关任务在同一节点执行- 反亲和性:避免在同一节点执行冲突任务4. 资源感知调度(Resource-Aware Scheduling)- CPU使用率:避免过载节点- 内存使用率:确保足够内存- 磁盘空间:检查可用存储空间- 网络带宽:考虑网络性能
自定义调度策略:
// 在共享库中实现智能调度
def selectOptimalAgent(Map requirements) {def availableNodes = Jenkins.instance.nodes.findAll { node ->node.computer.online && !node.computer.temporarilyOffline}def suitableNodes = availableNodes.findAll { node ->def nodeLabels = node.labelString.split(' ') as Setdef requiredLabels = requirements.labels as Set// 检查标签匹配if (!requiredLabels.every { nodeLabels.contains(it) }) {return false}// 检查资源要求def computer = node.computerdef executors = computer.executorsdef busyExecutors = executors.findAll { it.busy }def cpuUsage = (busyExecutors.size() / executors.size()) * 100if (cpuUsage > requirements.maxCpuUsage ?: 80) {return false}// 检查内存使用def memoryMonitor = computer.getMonitorData()['hudson.node_monitors.SwapSpaceMonitor']if (memoryMonitor && memoryMonitor.availablePhysicalMemory < (requirements.minMemoryMB ?: 1024) * 1024 * 1024) {return false}return true}if (suitableNodes.empty) {error "没有找到满足要求的节点: ${requirements}"}// 选择负载最轻的节点def optimalNode = suitableNodes.min { node ->def computer = node.computerdef executors = computer.executorsdef busyExecutors = executors.findAll { it.busy }return busyExecutors.size() / executors.size()}return optimalNode.nodeName
}// 在Pipeline中使用
pipeline {agent nonestages {stage('Smart Scheduling') {steps {script {def requirements = [labels: ['linux', 'docker', 'high-memory'],maxCpuUsage: 70,minMemoryMB: 4096]def selectedAgent = selectOptimalAgent(requirements)echo "选择的Agent: ${selectedAgent}"node(selectedAgent) {checkout scmsh 'echo "在最优Agent上执行构建"'// 执行构建步骤}}}}}
}
队列管理
构建队列优化:
// 队列管理脚本
import jenkins.model.Jenkins
import hudson.model.Queue
import hudson.model.queue.QueueTaskFuturedef optimizeBuildQueue() {def jenkins = Jenkins.instancedef queue = jenkins.queue// 获取队列中的任务def queuedItems = queue.itemsecho "当前队列中有 ${queuedItems.length} 个任务"// 按优先级排序def prioritizedItems = queuedItems.sort { item ->def priority = 0// 主分支构建优先级更高if (item.task.name.contains('main') || item.task.name.contains('master')) {priority += 100}// 生产部署优先级最高if (item.task.name.contains('deploy') && item.task.name.contains('prod')) {priority += 200}// 热修复优先级很高if (item.task.name.contains('hotfix')) {priority += 150}// 测试任务优先级较低if (item.task.name.contains('test')) {priority -= 50}return -priority // 负数用于降序排列}// 重新排列队列(这是一个简化的示例)prioritizedItems.eachWithIndex { item, index ->echo "优先级 ${index + 1}: ${item.task.name}"}
}// 定期执行队列优化
pipeline {agent anytriggers {cron('*/5 * * * *') // 每5分钟执行一次}stages {stage('Queue Optimization') {steps {script {optimizeBuildQueue()}}}}
}
队列监控和告警:
// 队列监控脚本
def monitorBuildQueue() {def jenkins = Jenkins.instancedef queue = jenkins.queuedef queuedItems = queue.items// 检查队列长度if (queuedItems.length > 20) {def message = "⚠️ 构建队列过长: ${queuedItems.length} 个任务等待执行"// 发送告警slackSend(channel: '#ops-alerts',color: 'warning',message: message)// 分析队列中的任务类型def taskTypes = [:]queuedItems.each { item ->def taskName = item.task.namedef taskType = 'other'if (taskName.contains('build')) taskType = 'build'else if (taskName.contains('test')) taskType = 'test'else if (taskName.contains('deploy')) taskType = 'deploy'taskTypes[taskType] = (taskTypes[taskType] ?: 0) + 1}def analysis = taskTypes.collect { type, count ->"${type}: ${count}"}.join(', ')echo "队列任务分析: ${analysis}"}// 检查长时间等待的任务def now = System.currentTimeMillis()def longWaitingTasks = queuedItems.findAll { item ->(now - item.inQueueSince) > 30 * 60 * 1000 // 30分钟}if (!longWaitingTasks.empty) {def message = "🕐 发现长时间等待的任务: ${longWaitingTasks.size()} 个"longWaitingTasks.each { item ->def waitTime = (now - item.inQueueSince) / (60 * 1000) // 分钟echo "任务 ${item.task.name} 已等待 ${waitTime.intValue()} 分钟"}slackSend(channel: '#ops-alerts',color: 'danger',message: message)}
}
资源监控
节点资源监控:
// 节点监控脚本
import jenkins.model.Jenkins
import hudson.node_monitors.*def monitorNodeResources() {def jenkins = Jenkins.instancedef nodes = jenkins.nodes + [jenkins] // 包括master节点nodes.each { node ->def computer = node.computerif (!computer.online) {echo "节点 ${node.nodeName} 离线"return}def monitors = computer.getMonitorData()// CPU监控def cpuMonitor = monitors['hudson.node_monitors.ArchitectureMonitor']// 内存监控def memoryMonitor = monitors['hudson.node_monitors.SwapSpaceMonitor']if (memoryMonitor) {def totalMemory = memoryMonitor.totalPhysicalMemorydef availableMemory = memoryMonitor.availablePhysicalMemorydef memoryUsage = ((totalMemory - availableMemory) / totalMemory) * 100echo "节点 ${node.nodeName} 内存使用率: ${memoryUsage.round(2)}%"if (memoryUsage > 90) {slackSend(channel: '#ops-alerts',color: 'danger',message: "🚨 节点 ${node.nodeName} 内存使用率过高: ${memoryUsage.round(2)}%")}}// 磁盘空间监控def diskMonitor = monitors['hudson.node_monitors.DiskSpaceMonitor']if (diskMonitor) {def freeSpace = diskMonitor.sizedef totalSpace = diskMonitor.size // 这里需要根据实际API调整if (freeSpace < 1024 * 1024 * 1024) { // 小于1GBslackSend(channel: '#ops-alerts',color: 'warning',message: "⚠️ 节点 ${node.nodeName} 磁盘空间不足: ${(freeSpace / (1024*1024*1024)).round(2)}GB")}}// 响应时间监控def responseMonitor = monitors['hudson.node_monitors.ResponseTimeMonitor']if (responseMonitor && responseMonitor.average > 5000) { // 5秒slackSend(channel: '#ops-alerts',color: 'warning',message: "🐌 节点 ${node.nodeName} 响应时间过长: ${responseMonitor.average}ms")}// 执行器状态def executors = computer.executorsdef busyExecutors = executors.findAll { it.busy }def executorUsage = (busyExecutors.size() / executors.size()) * 100echo "节点 ${node.nodeName} 执行器使用率: ${executorUsage.round(2)}%"}
}// 生成资源报告
def generateResourceReport() {def jenkins = Jenkins.instancedef nodes = jenkins.nodes + [jenkins]def report = [timestamp: new Date().toString(),nodes: []]nodes.each { node ->def computer = node.computerdef nodeInfo = [name: node.nodeName,online: computer.online,labels: node.labelString,executors: computer.executors.size(),busyExecutors: computer.executors.findAll { it.busy }.size()]if (computer.online) {def monitors = computer.getMonitorData()// 添加监控数据def memoryMonitor = monitors['hudson.node_monitors.SwapSpaceMonitor']if (memoryMonitor) {nodeInfo.memory = [total: memoryMonitor.totalPhysicalMemory,available: memoryMonitor.availablePhysicalMemory,usage: ((memoryMonitor.totalPhysicalMemory - memoryMonitor.availablePhysicalMemory) / memoryMonitor.totalPhysicalMemory) * 100]}def diskMonitor = monitors['hudson.node_monitors.DiskSpaceMonitor']if (diskMonitor) {nodeInfo.disk = [free: diskMonitor.size]}}report.nodes.add(nodeInfo)}// 保存报告def reportJson = groovy.json.JsonBuilder(report).toPrettyString()writeFile file: "resource-report-${env.BUILD_NUMBER}.json", text: reportJson// 归档报告archiveArtifacts artifacts: "resource-report-${env.BUILD_NUMBER}.json"return report
}
本章小结
本章详细介绍了Jenkins分布式构建的各个方面:
- 分布式构建概述:理解分布式架构的优势和基本概念
- Agent配置与管理:学习各种类型Agent的配置方法
- 云Agent配置:掌握AWS EC2和Kubernetes Agent的使用
- 负载均衡与调度:了解构建调度策略和资源管理
分布式构建是Jenkins的核心特性之一,正确配置和管理分布式环境对于提高构建效率和系统可靠性至关重要。
下一章预告
下一章我们将学习Jenkins的监控与日志管理,包括性能监控、日志分析和故障排除。
练习与思考
理论练习
-
架构设计:
- 设计适合团队的分布式构建架构
- 分析不同Agent类型的适用场景
- 规划节点标签和调度策略
-
资源规划:
- 评估构建资源需求
- 设计弹性扩展策略
- 考虑成本优化方案
实践练习
-
Agent配置:
- 配置SSH和JNLP Agent
- 设置云Agent(AWS或Kubernetes)
- 实现动态Agent调度
-
监控实现:
- 部署资源监控脚本
- 配置告警机制
- 生成资源使用报告
思考题
-
性能优化:
- 如何优化分布式构建的性能?
- 如何处理Agent之间的负载不均衡?
- 如何减少构建等待时间?
-
可靠性保障:
- 如何确保分布式环境的高可用性?
- 如何处理Agent故障和网络问题?
- 如何实现构建任务的故障转移?