ROS2下YOLO+Moveit+PCL机械臂自主避障抓取方案

整体运行架构

1.运行相机取像节点
. ./install/setup.bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true
2.运行根据图像x,y获取z的service

基本操作记录：

1. 创建python包,在src目录下
   ros2 pkg create test_python_topic --build-type ament_python --dependencies rclpy --license Apache-2.0
   2.创建service
   ros2 pkg create GetZService --dependencies sensor_msgs rosidl_default_generators --license Apache-2.0

一、奥比中光Gemini335相机使用

https://www.orbbec.com.cn/index/Gemini330/info.html?cate=119&id=74
https://github.com/orbbec/OrbbecSDK_v2/releases
1.

cd ~/ros2_ws/
# 构建 Release 版本，默认为 Debug
colcon build --event-handlers  console_direct+  --cmake-args  -DCMAKE_BUILD_TYPE=Release

. ./install/setup.bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true

话题是 /camera/depth_registered/points
得到的图像是sensor_msgs/msg/PointCloud2
ros2 interface show sensor_msgs/msg/PointCloud2 可以看到图像数据格式

. ./install/setup.bash
rviz2

写一个节点

二、YOLO ROS的使用

https://github.com/mgonzs13/yolo_ros/tree/main

终端1：

. ./install/setup.bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true

终端2：

. ./install/setup.bash
ros2 launch yolo_bringup yolo.launch.py input_image_topic:=/camera/color/image_raw  device:=cpu

终端3：

. ./install/setup.bash
ros2 topic info /camera/color/image_raw
rviz2

三、睿尔曼RM65的使用

https://develop.realman-robotics.com/robot/quickUseManual/

快速启动：

colcon build
source ~/ros2_ws/install/setup.bash
ros2 launch rm_bringup rm_65_bringup.launch.py

一个一个启动：

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/zc9527/ros2_ws/src/ros2_rm_robot/rm_driver/lib

ros2 launch rm_driver rm_65_driver.launch.pyros2 launch rm_description rm_65_display.launch.pyros2 launch rm_control rm_65_control.launch.pyros2 launch rm_65_config real_moveit_demo.launch.py

joint_states代表机械臂当前的状态，我们的rm_driver功能包运行时会发布该话题，这样rviz中的模型就会根据实际的机械臂状态进行运动。

rm_control功能包为实现moveit2控制真实机械臂时所必须的一个功能包，该功能包的主要作用为将moveit2规划好的路径点进行进一步的细分，将细分后的路径点以透传的方式给到rm_driver，实现机械臂的规划运行。

rm_description功能包为显示机器人模型和TF变换的功能包，通过该功能包，我们可以实现电脑中的虚拟机械臂与现实中的实际机械臂的联动的效果
/robot_state_publisher

rm_moveit2_config是实现Moveit2控制真实机械臂的功能包，该功能包的主要作用为调用官方的Moveit2框架，结合我们机械臂本身的URDF生成适配于我们机械臂的moveit2的配置和启动文件，通过该功能包我们可以实现moveit2控制虚拟机械臂和控制真实机械臂。
/interactive_marker_display_99263946702096
/move_group
/move_group_private_105684501566768
/moveit_simple_controller_manager
/rviz
/rviz_ssp_robot_description
/transform_listener_impl_5a47b008a910
/transform_listener_impl_601e972d1ab0

ros2 run rqt_graph rqt_graph

rm_control/rm_control
rm_description/robot_state_publisher
rm_65_config /moveit_simple_controller_manager
rm_65_config //transform_listener_impl

ompl_planning不会自动生成，需要自己添加文件内容如下：

planning_plugin: ompl_interface/OMPLPlanner
request_adapters: >-default_planner_request_adapters/AddTimeOptimalParameterizationdefault_planner_request_adapters/ResolveConstraintFramesdefault_planner_request_adapters/FixWorkspaceBoundsdefault_planner_request_adapters/FixStartStateBoundsdefault_planner_request_adapters/FixStartStateCollisiondefault_planner_request_adapters/FixStartStatePathConstraints
start_state_max_bounds_error: 0.1
planner_configs:SBLkConfigDefault:type: geometric::SBLrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()ESTkConfigDefault:type: geometric::ESTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0 setup()goal_bias: 0.05  # When close to goal select goal, with this probability. default: 0.05LBKPIECEkConfigDefault:type: geometric::LBKPIECErange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()border_fraction: 0.9  # Fraction of time focused on boarder default: 0.9min_valid_path_fraction: 0.5  # Accept partially valid moves above fraction. default: 0.5BKPIECEkConfigDefault:type: geometric::BKPIECErange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()border_fraction: 0.9  # Fraction of time focused on boarder default: 0.9failed_expansion_score_factor: 0.5  # When extending motion fails, scale score by factor. default: 0.5min_valid_path_fraction: 0.5  # Accept partially valid moves above fraction. default: 0.5KPIECEkConfigDefault:type: geometric::KPIECErange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability. default: 0.05border_fraction: 0.9  # Fraction of time focused on boarder default: 0.9 (0.0,1.]failed_expansion_score_factor: 0.5  # When extending motion fails, scale score by factor. default: 0.5min_valid_path_fraction: 0.5  # Accept partially valid moves above fraction. default: 0.5RRTkConfigDefault:type: geometric::RRTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability? default: 0.05RRTConnectkConfigDefault:type: geometric::RRTConnectrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()RRTstarkConfigDefault:type: geometric::RRTstarrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability? default: 0.05delay_collision_checking: 1  # Stop collision checking as soon as C-free parent found. default 1TRRTkConfigDefault:type: geometric::TRRTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability? default: 0.05max_states_failed: 10  # when to start increasing temp. default: 10temp_change_factor: 2.0  # how much to increase or decrease temp. default: 2.0min_temperature: 10e-10  # lower limit of temp change. default: 10e-10init_temperature: 10e-6  # initial temperature. default: 10e-6frountier_threshold: 0.0  # dist new state to nearest neighbor to disqualify as frontier. default: 0.0 set in setup()frountierNodeRatio: 0.1  # 1/10, or 1 nonfrontier for every 10 frontier. default: 0.1k_constant: 0.0  # value used to normalize expresssion. default: 0.0 set in setup()PRMkConfigDefault:type: geometric::PRMmax_nearest_neighbors: 10  # use k nearest neighbors. default: 10PRMstarkConfigDefault:type: geometric::PRMstarFMTkConfigDefault:type: geometric::FMTnum_samples: 1000  # number of states that the planner should sample. default: 1000radius_multiplier: 1.1  # multiplier used for the nearest neighbors search radius. default: 1.1nearest_k: 1  # use Knearest strategy. default: 1cache_cc: 1  # use collision checking cache. default: 1heuristics: 0  # activate cost to go heuristics. default: 0extended_fmt: 1  # activate the extended FMT*: adding new samples if planner does not finish successfully. default: 1BFMTkConfigDefault:type: geometric::BFMTnum_samples: 1000  # number of states that the planner should sample. default: 1000radius_multiplier: 1.0  # multiplier used for the nearest neighbors search radius. default: 1.0nearest_k: 1  # use the Knearest strategy. default: 1balanced: 0  # exploration strategy: balanced true expands one tree every iteration. False will select the tree with lowest maximum cost to go. default: 1optimality: 1  # termination strategy: optimality true finishes when the best possible path is found. Otherwise, the algorithm will finish when the first feasible path is found. default: 1heuristics: 1  # activates cost to go heuristics. default: 1cache_cc: 1  # use the collision checking cache. default: 1extended_fmt: 1  # Activates the extended FMT*: adding new samples if planner does not finish successfully. default: 1PDSTkConfigDefault:type: geometric::PDSTSTRIDEkConfigDefault:type: geometric::STRIDErange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability. default: 0.05use_projected_distance: 0  # whether nearest neighbors are computed based on distances in a projection of the state rather distances in the state space itself. default: 0degree: 16  # desired degree of a node in the Geometric Near-neightbor Access Tree (GNAT). default: 16max_degree: 18  # max degree of a node in the GNAT. default: 12min_degree: 12  # min degree of a node in the GNAT. default: 12max_pts_per_leaf: 6  # max points per leaf in the GNAT. default: 6estimated_dimension: 0.0  # estimated dimension of the free space. default: 0.0min_valid_path_fraction: 0.2  # Accept partially valid moves above fraction. default: 0.2BiTRRTkConfigDefault:type: geometric::BiTRRTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()temp_change_factor: 0.1  # how much to increase or decrease temp. default: 0.1init_temperature: 100  # initial temperature. default: 100frountier_threshold: 0.0  # dist new state to nearest neighbor to disqualify as frontier. default: 0.0 set in setup()frountier_node_ratio: 0.1  # 1/10, or 1 nonfrontier for every 10 frontier. default: 0.1cost_threshold: 1e300  # the cost threshold. Any motion cost that is not better will not be expanded. default: infLBTRRTkConfigDefault:type: geometric::LBTRRTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability. default: 0.05epsilon: 0.4  # optimality approximation factor. default: 0.4BiESTkConfigDefault:type: geometric::BiESTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()ProjESTkConfigDefault:type: geometric::ProjESTrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()goal_bias: 0.05  # When close to goal select goal, with this probability. default: 0.05LazyPRMkConfigDefault:type: geometric::LazyPRMrange: 0.0  # Max motion added to tree. ==> maxDistance_ default: 0.0, if 0.0, set on setup()LazyPRMstarkConfigDefault:type: geometric::LazyPRMstarSPARSkConfigDefault:type: geometric::SPARSstretch_factor: 3.0  # roadmap spanner stretch factor. multiplicative upper bound on path quality. It does not make sense to make this parameter more than 3. default: 3.0sparse_delta_fraction: 0.25  # delta fraction for connection distance. This value represents the visibility range of sparse samples. default: 0.25dense_delta_fraction: 0.001  # delta fraction for interface detection. default: 0.001max_failures: 1000  # maximum consecutive failure limit. default: 1000SPARStwokConfigDefault:type: geometric::SPARStwostretch_factor: 3.0  # roadmap spanner stretch factor. multiplicative upper bound on path quality. It does not make sense to make this parameter more than 3. default: 3.0sparse_delta_fraction: 0.25  # delta fraction for connection distance. This value represents the visibility range of sparse samples. default: 0.25dense_delta_fraction: 0.001  # delta fraction for interface detection. default: 0.001max_failures: 5000  # maximum consecutive failure limit. default: 5000TrajOptDefault:type: geometric::TrajOptrm_group:planner_configs:- SBLkConfigDefault- ESTkConfigDefault- LBKPIECEkConfigDefault- BKPIECEkConfigDefault- KPIECEkConfigDefault- RRTkConfigDefault- RRTConnectkConfigDefault- RRTstarkConfigDefault- TRRTkConfigDefault- PRMkConfigDefault- PRMstarkConfigDefault- FMTkConfigDefault- BFMTkConfigDefault- PDSTkConfigDefault- STRIDEkConfigDefault- BiTRRTkConfigDefault- LBTRRTkConfigDefault- BiESTkConfigDefault- ProjESTkConfigDefault- LazyPRMkConfigDefault- LazyPRMstarkConfigDefault- SPARSkConfigDefault- SPARStwokConfigDefault- TrajOptDefault

moveL运动:
ros2 launch rm_driver rm_65_driver.launch.py
ros2 launch rm_example rm_6dof_movej.launch.py

四、moveit2的使用

https://github.com/moveit/moveit2_tutorials/tree/humble
https://blog.csdn.net/forever0007/article/details/143745333
https://zhuanlan.zhihu.com/p/616101551

五、moveit2和睿尔曼机器人的联合使用

ros2 pkg create \--build-type ament_cmake \--dependencies moveit_ros_planning_interface rclcpp \--node-name hello_moveit hello_moveit

source install/setup.bash
ros2 run hello_moveit hello_moveit

最简单使用

#include <memory>#include <rclcpp/rclcpp.hpp>
#include <moveit/move_group_interface/move_group_interface.h>int main(int argc, char* argv[])
{// Initialize ROS and create the Noderclcpp::init(argc, argv);auto const node = std::make_shared<rclcpp::Node>("hello_moveit", rclcpp::NodeOptions().automatically_declare_parameters_from_overrides(true));// Create a ROS loggerauto const logger = rclcpp::get_logger("hello_moveit");// Create the MoveIt MoveGroup Interfaceusing moveit::planning_interface::MoveGroupInterface;auto move_group_interface = MoveGroupInterface(node, "rm_group");// Set a target Poseauto const target_pose = [] {geometry_msgs::msg::Pose msg;msg.orientation.w = 1.0;msg.position.x = 0.28;msg.position.y = -0.2;msg.position.z = 0.5;return msg;}();move_group_interface.setPoseTarget(target_pose);// Create a plan to that target poseauto const [success, plan] = [&move_group_interface] {moveit::planning_interface::MoveGroupInterface::Plan msg;auto const ok = static_cast<bool>(move_group_interface.plan(msg));return std::make_pair(ok, msg);}();// Execute the planif (success){move_group_interface.execute(plan);}else{RCLCPP_ERROR(logger, "Planing failed!");}// Shutdown ROSrclcpp::shutdown();return 0;
}

六、ROS2 PCL的使用

https://github.com/adrian-soch/pcl_tutorial?utm_source=chatgpt.com