rskill-moveit-look-at

Aim a robot-mounted camera at a 3-D point (ADR-0044 Phase 3; renamed from openral/rskill-look-at under ADR-0052).

A kind: ros_action rSkill wrapping moveit_msgs/action/MoveGroup — like rskill-moveit-joints, but the goal is a look_at block instead of raw constraints, and ros_integration.goal_builder: look_at selects the LookAtRskill adapter that lowers it into MoveGroup pose constraints.

What this skill does

Given a target point and a camera name, plans a collision-aware arm motion that points the named camera's optical axis at the target — so a later perception query (locate_in_view) or manipulation skill sees the object framed. It is the "look" rung of the ADR-0044 recall → navigate → look → verify → manipulate ladder.

How it works

ros_integration.goal_builder: look_at selects LookAtRskill (openral_rskill.look_at_rskill), which lowers the look_at block at dispatch time:

Resolve the camera named by look_at.camera (default "wrist") from the host RobotDescription.sensors. No such sensor → ROSConfigError listing the robot's available sensors — never a silent guess. A sensor whose frame_id is itself a robot link (franka's LIBERO eye-in-hand on panda_hand) is constrained directly; a sensor with parent_frame + static_transform_xyz_rpy (so101-style mount) constrains the parent link through the declared offset.
Read the camera's current pose over TF2 (the only source of frames) in the goal frame.
Place the camera goal — in place (pure re-aim, the default) or at look_at.standoff_m from the target along the current line of approach.
Orient it with compute_gaze_pose (ROS optical convention: camera +Z hits look_at.target_xyz; roll about the optical axis left free at tolerance π for planner reachability) and submit MoveGroup position_constraints + orientation_constraints.

plan_only: true, deliberately: OpenRAL's actuation path is the per-waypoint replay through /openral/candidate_action (chunk_size: 1, so the safety supervisor's per-joint envelope check sees every aiming step, and the HAL actuates). Letting move_group also execute on its own controllers would bypass the kernel and double-drive the arm.

Observation → action contract

Input is the ADR-0026 goal_params_json look_at block; output is a joint trajectory replayed one waypoint per step() as a 1-row JOINT_POSITION Action chunk.

{"look_at": {"target_xyz": [0.5, 0.0, 0.2], "camera": "wrist"}}

Planner settings (request.group_name, scaling, attempts) are inherited from default_goal_json. Omit standoff_m to re-aim in place; set it to also move the camera to that distance from the target.

How it was trained / Upstream provenance

Nothing is trained — this rSkill wraps the upstream MoveIt motion planner and computes the gaze pose analytically.

Field	Value
Upstream	`moveit_msgs/action/MoveGroup` (BSD-3-Clause)
Planner library	MoveIt 2 (BSD-3-Clause)
Gaze geometry	`openral_world_state.geometry.compute_gaze_pose` (Apache-2.0, in-tree)
Collision check	FCL via MoveIt's `PlanningScene` (run during planning)
Wrapped artefact	rSkill manifest + README — no weights, no preprocessor JSONs

Supported robots / embodiments

Robot	Embodiment tag	Status	Notes
Franka Panda	`franka_panda`	validated	default goal targets `panda_arm`; camera `wrist` = LIBERO eye-in-hand on `panda_hand`
Universal Robots UR5e	`ur5e`	experimental	needs `request.group_name` + `look_at.camera`/`frame_id` overrides
Universal Robots UR10e	`ur10e`	experimental	same as UR5e
SO-100 follower	`so100_follower`	experimental	needs a MoveIt config; `wrist` cam declares a static mount
OpenArm	`openarm`	experimental	bi-manual — choose `left_arm`/`right_arm` group
Flexiv Rizon4	`rizon4`	experimental	upstream MoveIt config exists; manifest override needed
Rethink Sawyer	`sawyer`	experimental	upstream MoveIt config exists
Trossen WidowX	`widowx`	experimental	upstream MoveIt config exists

A robot must declare the named camera (default wrist) in its robot.yaml sensors, or the skill fails at configure with the available-sensor list. Listed embodiment_tags only gate palette visibility; actual resolution needs move_group up for that robot.

Sensors required / Observation contract

This skill consumes no camera frames through OpenRAL's sensor pipeline — it aims a camera, it doesn't read one. It needs the named camera's frame to exist in TF (declared in the robot manifest + published by robot_state_publisher), plus MoveIt's own subscriptions:

Source	Topic / contract	Why it's needed
Camera frame	TF `frame_id` of `look_at.camera` (default `wrist`)	Read the camera's current pose; target the gaze
Joint state	`/joint_states`	Plan from the live start state
Planning scene	`/planning_scene`	Collision check during planning
TF	`/tf`, `/tf_static`	Resolve camera + link frames

Manifest summary

Field	Value
`name`	`OpenRAL/rskill-moveit-look-at`
`version`	`0.1.0`
`license`	`apache-2.0`
`kind`	`ros_action`
`role`	`s1`
`actions`	`[look]`
`chunk_size`	`1` (schema-enforced for `kind: ros_action`)
`latency_budget.per_chunk_ms`	`2000` (planning latency)
`ros_integration.package`	`moveit_msgs`
`ros_integration.interface_type`	`MoveGroup`
`ros_integration.interface_name`	`/move_action`
`ros_integration.goal_builder`	`look_at` (selects `LookAtRskill`)
`ros_integration.result_trajectory_field`	`planned_trajectory.joint_trajectory`
`commercial_use_allowed`	`true` (apache-2.0)

Full schema: openral_core.schemas.RSkillManifest.

Quick start

from openral_rskill.loader import rSkill
pkg = rSkill.from_yaml("rskills/rskill-moveit-look-at/rskill.yaml")
print(pkg.manifest.name, pkg.manifest.ros_integration.goal_builder)

End-to-end, with a real MoveIt launch up:

# 1. Bring up MoveIt for your robot (example: Panda)
ros2 launch moveit_resources_panda_moveit_config demo.launch.py

# 2. Dispatch a look-at goal (aim the wrist camera at a tabletop point):
ros2 action send_goal /openral/execute_rskill openral_msgs/action/ExecuteRskill \
    "{rskill_id: 'OpenRAL/rskill-moveit-look-at', deadline_s: 30.0, prompt: 'look at the mug',
      goal_params_json: '{\"look_at\": {\"target_xyz\": [0.5, 0.0, 0.2], \"camera\": \"wrist\"}}'}"

Limitations / Roadmap

Reachability is the planner's call. Roll about the optical axis is left free, but a target outside the arm's dexterous workspace simply fails to plan — there's no base-repositioning fallback here (that's the navigate rung of the ladder, ADR-0044 Phase 4).
Single-camera aim. One camera per dispatch; multi-camera coverage is a reasoner-level concern.
No velocity / jerk bound at the supervisor. Same posture as rskill-moveit-joints: the per-joint position envelope runs per waypoint; richer bounds are tracked separately.

License

The rSkill package itself (this manifest + README) is Apache-2.0. The wrapped MoveIt code (moveit_msgs IDL, moveit2 planners) is BSD-3-Clause and is installed via ros-${ROS_DISTRO}-moveit, outside this repository. Per ADR-0012 both postures are commercial-use-permissive.

OpenRAL
/

rskill-moveit-look-at