当前位置： 首页 > news >正文

【使用LLM搭建系统】5 处理输入: 链式 Prompt Chaining Prompts

news 2025/6/10 9:48:58

本章内容主要介绍了将复杂任务拆分为多个子任务（链式Prompt）的方法及其优势。尽管高级语言模型像GPT - 4擅长一次性遵循复杂指令，但有时拆分任务更可取。通过两个比喻来阐述原因：

  1. 一次烹饪复杂菜肴与分阶段烹饪：一次性复杂Prompt像一次性烹饪复杂菜肴，易出错；链式Prompt像分阶段烹饪，逐步确保每个部分正确。

  2. 一次性完成任务与分阶段完成任务：复杂单步任务像一长串代码，难调试；而链式Prompt如同模块化程序，当有需要在各节点根据状态调整操作的工作流程时，能提高效率。

一、环境设置

from zhipuai import ZhipuAI
# 导入第三方库key = "f5cd91f2528fed334b9dfd75015791c3.GuLdvM9tXWrGQnAg"
client = ZhipuAI(api_key = key)
def get_completion_from_messages(messages, model="glm-3-turbo", temperature=0, max_tokens=500):'''封装一个访问 OpenAI GPT3.5 的函数参数: messages: 这是一个消息列表，每个消息都是一个字典，包含 role(角色）和 content(内容)。角色可以是'system'、'user' 或 'assistant’，内容是角色的消息。model: 调用的模型，默认为 gpt-3.5-turbo(ChatGPT)，有内测资格的用户可以选择 gpt-4temperature: 这决定模型输出的随机程度，默认为0，表示输出将非常确定。增加温度会使输出更随机。max_tokens: 这决定模型输出的最大的 token 数。'''response = client.chat.completions.create(model=model,messages=messages,temperature=temperature, # 这决定模型输出的随机程度max_tokens=max_tokens, # 这决定模型输出的最大的 token 数)return response.choices[0].message.content

二、 实现一个包含多个提示的复杂任务

2.1 提取相关产品和类别名称

对客户查询进行分类后，采用链式Prompt方法的优势：

  1. 任务明确与管理简便：每个子任务只包含必要指令，使系统易于管理，确保模型具备执行任务所需信息，降低出错率。

  2. 降低成本：避免使用过长的Prompt和过多的tokens，减少运行成本，且在某些情况下无需概述所有步骤。

  3. 便于测试与干预：更容易测试哪些步骤可能失败，或在特定步骤中需要人工干预。

  4. 培养策略运用直觉：随着与模型的深入交互，逐渐形成何时使用该策略的直觉。

  5. 支持外部工具调用：允许模型在必要时调用外部工具，如在产品目录中查找内容、调用API或搜索知识库，这是单个Prompt无法实现的。

delimiter = "####"
system_message = f"""
你将提供服务查询。
服务查询将使用{delimiter}字符分隔。仅输出一个 Python 对象列表，其中每个对象具有以下格式：'category': <计算机和笔记本电脑、智能手机和配件、电视和家庭影院系统、游戏机和配件、音频设备、相机和摄像机中的一个>,
或者'products': <必须在下面的允许产品列表中找到的产品列表>类别和产品必须在客户服务查询中找到。
如果提及了产品，则必须将其与允许产品列表中的正确类别相关联。
如果未找到产品或类别，则输出空列表。允许的产品：计算机和笔记本电脑类别：
TechPro Ultrabook
BlueWave Gaming Laptop
PowerLite Convertible
TechPro Desktop
BlueWave Chromebook智能手机和配件类别：
SmartX ProPhone
MobiTech PowerCase
SmartX MiniPhone
MobiTech Wireless Charger
SmartX EarBuds电视和家庭影院系统类别：
CineView 4K TV
SoundMax Home Theater
CineView 8K TV
SoundMax Soundbar
CineView OLED TV
c
游戏机和配件类别：
GameSphere X
ProGamer Controller
GameSphere Y
ProGamer Racing Wheel
GameSphere VR Headset音频设备类别：
AudioPhonic Noise-Canceling Headphones
WaveSound Bluetooth Speaker
AudioPhonic True Wireless Earbuds
WaveSound Soundbar
AudioPhonic Turntable相机和摄像机类别：
FotoSnap DSLR Camera
ActionCam 4K
FotoSnap Mirrorless Camera
ZoomMaster Camcorder
FotoSnap Instant Camera仅输出 Python 对象列表，不包含其他字符信息。
"""
user_message_1 = f"""请查询 SmartX ProPhone 智能手机和 FotoSnap 相机，包括单反相机。另外，请查询关于电视产品的信息。 """
messages =  [  
{'role':'system', 'content': system_message},    
{'role':'user', 'content': f"{delimiter}{user_message_1}{delimiter}"},  
] 
category_and_product_response_1 = get_completion_from_messages(messages)
print(category_and_product_response_1)
[{'products': ['SmartX ProPhone']},{'category': '智能手机和配件'},{'products': ['FotoSnap DSLR Camera']},{'category': '相机和摄像机'},{'category': '电视和家庭影院系统'}
]

输出是一个对象列表，每个对象包含一个类别和一些产品，例如“SmartX ProPhone”和“Fotosnap DSLR Camera”。在某些情况下，可能只有一个类别而没有具体的产品，例如最后一个对象中没有提到任何具体的电视。这种结构化的输出可以轻松地读入Python的列表中，便于后续处理。

user_message_2 = f"""我的路由器坏了"""
messages =  [  
{'role':'system','content': system_message},    
{'role':'user','content': f"{delimiter}{user_message_2}{delimiter}"},  
] 
response = get_completion_from_messages(messages)
print(response)

没有包含任何路由器的信息，输出是一个空列表。

2.2 检索提取的产品和类别的详细信息

提供大量的产品信息作为示例，要求模型提取产品和对应的详细信息

def get_product_by_name(name):"""根据产品名称获取产品参数:name: 产品名称"""return products.get(name, None)def get_products_by_category(category):"""根据类别获取产品参数:category: 产品类别"""return [product for product in products.values() if product["category"] == category]
# 产品信息
products = {"TechPro Ultrabook": {"name": "TechPro 超极本","category": "电脑和笔记本","brand": "TechPro","model_number": "TP-UB100","warranty": "1 year","rating": 4.5,"features": ["13.3-inch display", "8GB RAM", "256GB SSD", "Intel Core i5 处理器"],"description": "一款时尚轻便的超极本，适合日常使用。","price": 799.99},"BlueWave Gaming Laptop": {"name": "BlueWave 游戏本","category": "电脑和笔记本","brand": "BlueWave","model_number": "BW-GL200","warranty": "2 years","rating": 4.7,"features": ["15.6-inch display", "16GB RAM", "512GB SSD", "NVIDIA GeForce RTX 3060"],"description": "一款高性能的游戏笔记本电脑，提供沉浸式体验。","price": 1199.99},"PowerLite Convertible": {"name": "PowerLite Convertible","category": "电脑和笔记本","brand": "PowerLite","model_number": "PL-CV300","warranty": "1 year","rating": 4.3,"features": ["14-inch touchscreen", "8GB RAM", "256GB SSD", "360-degree hinge"],"description": "一款多功能的可转换笔记本电脑，具有灵敏的触摸屏。","price": 699.99},"TechPro Desktop": {"name": "TechPro Desktop","category": "电脑和笔记本","brand": "TechPro","model_number": "TP-DT500","warranty": "1 year","rating": 4.4,"features": ["Intel Core i7 processor", "16GB RAM", "1TB HDD", "NVIDIA GeForce GTX 1660"],"description": "一款功能强大的台式电脑，适用于工作和娱乐。","price": 999.99},"BlueWave Chromebook": {"name": "BlueWave Chromebook","category": "电脑和笔记本","brand": "BlueWave","model_number": "BW-CB100","warranty": "1 year","rating": 4.1,"features": ["11.6-inch display", "4GB RAM", "32GB eMMC", "Chrome OS"],"description": "一款紧凑而价格实惠的Chromebook，适用于日常任务。","price": 249.99},"SmartX ProPhone": {"name": "SmartX ProPhone","category": "智能手机和配件","brand": "SmartX","model_number": "SX-PP10","warranty": "1 year","rating": 4.6,"features": ["6.1-inch display", "128GB storage", "12MP dual camera", "5G"],"description": "一款拥有先进摄像功能的强大智能手机。","price": 899.99},"MobiTech PowerCase": {"name": "MobiTech PowerCase","category": "专业手机","brand": "MobiTech","model_number": "MT-PC20","warranty": "1 year","rating": 4.3,"features": ["5000mAh battery", "Wireless charging", "Compatible with SmartX ProPhone"],"description": "一款带有内置电池的保护手机壳，可延长使用时间。","price": 59.99},"SmartX MiniPhone": {"name": "SmartX MiniPhone","category": "专业手机","brand": "SmartX","model_number": "SX-MP5","warranty": "1 year","rating": 4.2,"features": ["4.7-inch display", "64GB storage", "8MP camera", "4G"],"description": "一款紧凑而价格实惠的智能手机，适用于基本任务。","price": 399.99},"MobiTech Wireless Charger": {"name": "MobiTech Wireless Charger","category": "专业手机","brand": "MobiTech","model_number": "MT-WC10","warranty": "1 year","rating": 4.5,"features": ["10W fast charging", "Qi-compatible", "LED indicator", "Compact design"],"description": "一款方便的无线充电器，使工作区域整洁无杂物。","price": 29.99},"SmartX EarBuds": {"name": "SmartX EarBuds","category": "专业手机","brand": "SmartX","model_number": "SX-EB20","warranty": "1 year","rating": 4.4,"features": ["True wireless", "Bluetooth 5.0", "Touch controls", "24-hour battery life"],"description": "通过这些舒适的耳塞体验真正的无线自由。","price": 99.99},"CineView 4K TV": {"name": "CineView 4K TV","category": "电视和家庭影院系统","brand": "CineView","model_number": "CV-4K55","warranty": "2 years","rating": 4.8,"features": ["55-inch display", "4K resolution", "HDR", "Smart TV"],"description": "一款色彩鲜艳、智能功能丰富的惊艳4K电视。","price": 599.99},"SoundMax Home Theater": {"name": "SoundMax Home Theater","category": "电视和家庭影院系统","brand": "SoundMax","model_number": "SM-HT100","warranty": "1 year","rating": 4.4,"features": ["5.1 channel", "1000W output", "Wireless subwoofer", "Bluetooth"],"description": "一款强大的家庭影院系统，提供沉浸式音频体验。","price": 399.99},"CineView 8K TV": {"name": "CineView 8K TV","category": "电视和家庭影院系统","brand": "CineView","model_number": "CV-8K65","warranty": "2 years","rating": 4.9,"features": ["65-inch display", "8K resolution", "HDR", "Smart TV"],"description": "通过这款惊艳的8K电视，体验未来。","price": 2999.99},"SoundMax Soundbar": {"name": "SoundMax Soundbar","category": "电视和家庭影院系统","brand": "SoundMax","model_number": "SM-SB50","warranty": "1 year","rating": 4.3,"features": ["2.1 channel", "300W output", "Wireless subwoofer", "Bluetooth"],"description": "使用这款时尚而功能强大的声音，升级您电视的音频体验。","price": 199.99},"CineView OLED TV": {"name": "CineView OLED TV","category": "电视和家庭影院系统","brand": "CineView","model_number": "CV-OLED55","warranty": "2 years","rating": 4.7,"features": ["55-inch display", "4K resolution", "HDR", "Smart TV"],"description": "通过这款OLED电视，体验真正的五彩斑斓。","price": 1499.99},"GameSphere X": {"name": "GameSphere X","category": "游戏机和配件","brand": "GameSphere","model_number": "GS-X","warranty": "1 year","rating": 4.9,"features": ["4K gaming", "1TB storage", "Backward compatibility", "Online multiplayer"],"description": "一款下一代游戏机，提供终极游戏体验。","price": 499.99},"ProGamer Controller": {"name": "ProGamer Controller","category": "游戏机和配件","brand": "ProGamer","model_number": "PG-C100","warranty": "1 year","rating": 4.2,"features": ["Ergonomic design", "Customizable buttons", "Wireless", "Rechargeable battery"],"description": "一款高品质的游戏手柄，提供精准和舒适的操作。","price": 59.99},"GameSphere Y": {"name": "GameSphere Y","category": "游戏机和配件","brand": "GameSphere","model_number": "GS-Y","warranty": "1 year","rating": 4.8,"features": ["4K gaming", "500GB storage", "Backward compatibility", "Online multiplayer"],"description": "一款体积紧凑、性能强劲的游戏机。","price": 399.99},"ProGamer Racing Wheel": {"name": "ProGamer Racing Wheel","category": "游戏机和配件","brand": "ProGamer","model_number": "PG-RW200","warranty": "1 year","rating": 4.5,"features": ["Force feedback", "Adjustable pedals", "Paddle shifters", "Compatible with GameSphere X"],"description": "使用这款逼真的赛车方向盘，提升您的赛车游戏体验。","price": 249.99},"GameSphere VR Headset": {"name": "GameSphere VR Headset","category": "游戏机和配件","brand": "GameSphere","model_number": "GS-VR","warranty": "1 year","rating": 4.6,"features": ["Immersive VR experience", "Built-in headphones", "Adjustable headband", "Compatible with GameSphere X"],"description": "通过这款舒适的VR头戴设备，进入虚拟现实的世界。","price": 299.99},"AudioPhonic Noise-Canceling Headphones": {"name": "AudioPhonic Noise-Canceling Headphones","category": "音频设备","brand": "AudioPhonic","model_number": "AP-NC100","warranty": "1 year","rating": 4.6,"features": ["Active noise-canceling", "Bluetooth", "20-hour battery life", "Comfortable fit"],"description": "通过这款降噪耳机，体验沉浸式的音效。","price": 199.99},"WaveSound Bluetooth Speaker": {"name": "WaveSound Bluetooth Speaker","category": "音频设备","brand": "WaveSound","model_number": "WS-BS50","warranty": "1 year","rating": 4.5,"features": ["Portable", "10-hour battery life", "Water-resistant", "Built-in microphone"],"description": "一款紧凑而多用途的蓝牙音箱，适用于随时随地收听音乐。","price": 49.99},"AudioPhonic True Wireless Earbuds": {"name": "AudioPhonic True Wireless Earbuds","category": "音频设备","brand": "AudioPhonic","model_number": "AP-TW20","warranty": "1 year","rating": 4.4,"features": ["True wireless", "Bluetooth 5.0", "Touch controls", "18-hour battery life"],"description": "通过这款舒适的真无线耳塞，无需线缆即可享受音乐。","price": 79.99},"WaveSound Soundbar": {"name": "WaveSound Soundbar","category": "音频设备","brand": "WaveSound","model_number": "WS-SB40","warranty": "1 year","rating": 4.3,"features": ["2.0 channel", "80W output", "Bluetooth", "Wall-mountable"],"description": "使用这款纤薄而功能强大的声音吧，升级您电视的音频体验。","price": 99.99},"AudioPhonic Turntable": {"name": "AudioPhonic Turntable","category": "音频设备","brand": "AudioPhonic","model_number": "AP-TT10","warranty": "1 year","rating": 4.2,"features": ["3-speed", "Built-in speakers", "Bluetooth", "USB recording"],"description": "通过这款现代化的唱片机，重拾您的黑胶唱片收藏。","price": 149.99},"FotoSnap DSLR Camera": {"name": "FotoSnap DSLR Camera","category": "相机和摄像机","brand": "FotoSnap","model_number": "FS-DSLR200","warranty": "1 year","rating": 4.7,"features": ["24.2MP sensor", "1080p video", "3-inch LCD", "Interchangeable lenses"],"description": "使用这款多功能的单反相机，捕捉惊艳的照片和视频。","price": 599.99},"ActionCam 4K": {"name": "ActionCam 4K","category": "相机和摄像机","brand": "ActionCam","model_number": "AC-4K","warranty": "1 year","rating": 4.4,"features": ["4K video", "Waterproof", "Image stabilization", "Wi-Fi"],"description": "使用这款坚固而紧凑的4K运动相机，记录您的冒险旅程。","price": 299.99},"FotoSnap Mirrorless Camera": {"name": "FotoSnap Mirrorless Camera","category": "相机和摄像机","brand": "FotoSnap","model_number": "FS-ML100","warranty": "1 year","rating": 4.6,"features": ["20.1MP sensor", "4K video", "3-inch touchscreen", "Interchangeable lenses"],"description": "一款具有先进功能的小巧轻便的无反相机。","price": 799.99},"ZoomMaster Camcorder": {"name": "ZoomMaster Camcorder","category": "相机和摄像机","brand": "ZoomMaster","model_number": "ZM-CM50","warranty": "1 year","rating": 4.3,"features": ["1080p video", "30x optical zoom", "3-inch LCD", "Image stabilization"],"description": "使用这款易于使用的摄像机，捕捉生活的瞬间。","price": 249.99},"FotoSnap Instant Camera": {"name": "FotoSnap Instant Camera","category": "相机和摄像机","brand": "FotoSnap","model_number": "FS-IC10","warranty": "1 year","rating": 4.1,"features": ["Instant prints", "Built-in flash", "Selfie mirror", "Battery-powered"],"description": "使用这款有趣且便携的即时相机，创造瞬间回忆。","price": 69.99}
}
print(get_product_by_name("TechPro Ultrabook"))
{'name': 'TechPro 超极本', 'category': '电脑和笔记本', 'brand': 'TechPro', 'model_number': 'TP-UB100', 'warranty': '1 year', 'rating': 4.5, 'features': ['13.3-inch display', '8GB RAM', '256GB SSD', 'Intel Core i5 处理器'], 'description': '一款时尚轻便的超极本，适合日常使用。', 'price': 799.99}
print(get_products_by_category("电脑和笔记本"))
[{'name': 'TechPro 超极本', 'category': '电脑和笔记本', 'brand': 'TechPro', 'model_number': 'TP-UB100', 'warranty': '1 year', 'rating': 4.5, 'features': ['13.3-inch display', '8GB RAM', '256GB SSD', 'Intel Core i5 处理器'], 'description': '一款时尚轻便的超极本，适合日常使用。', 'price': 799.99}, {'name': 'BlueWave 游戏本', 'category': '电脑和笔记本', 'brand': 'BlueWave', 'model_number': 'BW-GL200', 'warranty': '2 years', 'rating': 4.7, 'features': ['15.6-inch display', '16GB RAM', '512GB SSD', 'NVIDIA GeForce RTX 3060'], 'description': '一款高性能的游戏笔记本电脑，提供沉浸式体验。', 'price': 1199.99}, {'name': 'PowerLite Convertible', 'category': '电脑和笔记本', 'brand': 'PowerLite', 'model_number': 'PL-CV300', 'warranty': '1 year', 'rating': 4.3, 'features': ['14-inch touchscreen', '8GB RAM', '256GB SSD', '360-degree hinge'], 'description': '一款多功能的可转换笔记本电脑，具有灵敏的触摸屏。', 'price': 699.99}, {'name': 'TechPro Desktop', 'category': '电脑和笔记本', 'brand': 'TechPro', 'model_number': 'TP-DT500', 'warranty': '1 year', 'rating': 4.4, 'features': ['Intel Core i7 processor', '16GB RAM', '1TB HDD', 'NVIDIA GeForce GTX 1660'], 'description': '一款功能强大的台式电脑，适用于工作和娱乐。', 'price': 999.99}, {'name': 'BlueWave Chromebook', 'category': '电脑和笔记本', 'brand': 'BlueWave', 'model_number': 'BW-CB100', 'warranty': '1 year', 'rating': 4.1, 'features': ['11.6-inch display', '4GB RAM', '32GB eMMC', 'Chrome OS'], 'description': '一款紧凑而价格实惠的Chromebook，适用于日常任务。', 'price': 249.99}]
print(user_message_1)
print(category_and_product_response_1)

2.3 将 Python 字符串读取为 Python 字典列表

import json def read_string_to_list(input_string):"""将输入的字符串转换为 Python 列表。参数:input_string: 输入的字符串，应为有效的 JSON 格式。返回:list 或 None: 如果输入字符串有效，则返回对应的 Python 列表，否则返回 None。"""if input_string is None:return Nonetry:# 将输入字符串中的单引号替换为双引号，以满足 JSON 格式的要求input_string = input_string.replace("'", "\"")  data = json.loads(input_string)return dataexcept json.JSONDecodeError:print("Error: Invalid JSON string")return None   
category_and_product_list = read_string_to_list(category_and_product_response_1)
print(category_and_product_list)
[{'products': ['SmartX ProPhone']}, {'category': '智能手机和配件'}, {'products': ['FotoSnap DSLR Camera']}, {'category': '相机和摄像机'}, {'category': '电视和家庭影院系统'}]
2.3.1 召回相关产品和类别的详细信息
def generate_output_string(data_list):"""根据输入的数据列表生成包含产品或类别信息的字符串。参数:data_list: 包含字典的列表，每个字典都应包含 "products" 或 "category" 的键。返回:output_string: 包含产品或类别信息的字符串。"""if data_list is None:return output_stringoutput_string = ''for data in data_list:try:if "products" in data:products_list = data["products"]for product_name in products_list:print(product_name)product = get_product_by_name(product_name)if product:output_string += json.dumps(product, indent=4) + "\n"else:print(f"Error: Product '{product_name}' not found")elif "category" in data:category_name = data["category"]category_products = get_products_by_category(category_name)for product in category_products:output_string += json.dumps(product, indent=4) + "\n"else:print("Error: Invalid object format")except Exception as e:print(f"Error: {e}")return output_string 
product_information_for_user_message_1 = generate_output_string(category_and_product_list)
print(product_information_for_user_message_1)

2.4 根据详细的产品信息生成用户查询的答案

system_message = f"""
您是一家大型电子商店的客服助理。
请以友好和乐于助人的口吻回答问题，并尽量简洁明了。
请确保向用户提出相关的后续问题。
"""
user_message_1 = f"""
请介绍一下 SmartX ProPhone 智能手机和 FotoSnap 相机，包括单反相机。
另外，介绍关于电视产品的信息。"""
messages =  [  
{'role':'system','content': system_message},   
{'role':'user','content': user_message_1},  
{'role':'assistant','content': f"""相关产品信息:\n\{product_information_for_user_message_1}"""},   
]
final_response = get_completion_from_messages(messages)
print(final_response)
当然可以！以下是关于您提到的产品的简要介绍：### SmartX ProPhone 智能手机
- **特点**：高性能处理器、高清显示屏、强大的摄像头系统、长续航电池。
- **适用人群**：适合追求高性能和优质摄影体验的用户。### FotoSnap 相机（包括单反相机）
- **特点**：高分辨率传感器、多种拍摄模式、可更换镜头、耐用机身。
- **适用人群**：适合摄影爱好者和专业摄影师。### 电视产品
- **特点**：高清或4K分辨率、智能操作系统、多种尺寸可选、支持流媒体服务。
- **适用人群**：适合家庭娱乐和游戏玩家。您对哪款产品特别感兴趣，或者有其他具体问题吗？
http://www.xdnf.cn/news/957799.html

相关文章：

  • 运行vue项目报错 errors and 0 warnings potentially fixable with the `--fix` option.
  • React hook之useRef
  • STM32---外部32.768K晶振（LSE）无法起振问题
  • 模型重展UV后绘制纹理
  • 什么是人工智能（AI）？大学想学AI，该如何入手？
  • 【CATIA的二次开发24】抽象对象Document涉及文档生命周期的方法
  • Python文本序列的类型
  • IDC 观察：一体化数据库如何支撑 GenAI 的数据需求
  • WebGL与Three.js：从基础到应用的关系与原理解析
  • MybatisPlus枚举类的应用与转换
  • L1和L2核心区别 ！！--part 2
  • 基于人工智能的闸机人脸识别门禁系统
  • 昇腾CANN集合通信技术解读——细粒度分级流水算法
  • CMake 从 GitHub 下载第三方库并使用
  • 高端性能封装正在突破性能壁垒，其芯片集成技术助力人工智能革命。
  • ABAP设计模式之---“童子军法则(The Boy Scout Rule)”
  • animate.css详解：轻松实现网页动画效果
  • 制作一款打飞机游戏68：地面敌人
  • CopyOnWriteArrayList和CopyOnWriteArraySet ：并发安全的写时复制机制
  • 新手指南：如何轻松将文件压缩为RAR格式
  • Android多媒体——音/视频数据播放（十八）
  • 如何实现高可用评论服务
  • gtxe2_channel内部参数和寄存器配置-CPLL超频设计，超过6.6Gbps的最高速率
  • OpenHarmony按键分发流程（60%）
  • 4.redis集群
  • rk3568的data分区修改
  • 以太网PHY布局布线指南
  • Houdini POP入门学习07 - 分组
  • 热门Chrome扩展程序存在明文传输风险，用户隐私安全受威胁
  • 论文解读：交大港大上海AI Lab开源论文 | 宇树机器人多姿态起立控制强化学习框架（二）