Dynamic Message Type Loading
This chapter explains one of the most important and sophisticated features in roc
: dynamic runtime loading of ROS2 message type support. This technique enables roc
to work with any ROS2 message type without requiring compile-time knowledge or static linking against specific message packages.
Table of Contents
- Overview
- The Problem
- The Solution: Runtime Dynamic Loading
- How Dynamic Loading Works
- Implementation Architecture
- Code Walkthrough
- Generic Type Support Resolution
- Benefits and Trade-offs
- Future Enhancements
Overview
Dynamic message type loading is a technique that allows roc
to:
- Load ROS2 message type support libraries at runtime (not compile time)
- Resolve type support functions dynamically using symbol lookup
- Create real RCL publishers/subscribers for any message type
- Support custom message types without code changes
- Work with any ROS2 package that provides proper typesupport libraries
This is what enables commands like:
# Works with any installed ROS2 message type!
roc topic pub /test geometry_msgs/msg/Twist '{linear: {x: 0.5}}'
roc topic pub /custom custom_msgs/msg/MyMessage '{field: value}'
The Problem
Traditional ROS2 tools face a fundamental challenge:
Static Linking Approach (Traditional)
#![allow(unused)] fn main() { // Traditional approach requires compile-time knowledge use geometry_msgs::msg::Twist; use std_msgs::msg::String; // ... must import every message type you want to use fn create_publisher() { // Must know the exact type at compile time let twist_publisher = node.create_publisher::<Twist>("topic", qos); let string_publisher = node.create_publisher::<String>("topic", qos); } }
Problems with static linking:
- ❌ Limited to pre-compiled message types
- ❌ Huge binary size (includes all message libraries)
- ❌ Cannot work with custom/unknown message types
- ❌ Requires recompilation for new message types
- ❌ Complex dependency management
The ROS2 Type Support Challenge
ROS2's architecture requires type support pointers to create publishers:
// This is what RCL requires internally
rcl_ret_t rcl_publisher_init(
rcl_publisher_t * publisher,
const rcl_node_t * node,
const rosidl_message_type_support_t * type_support, // ← This is the key!
const char * topic_name,
const rcl_publisher_options_t * options
);
The type_support
pointer contains:
- Message structure layout
- Serialization/deserialization functions
- Field metadata and types
- Memory management functions
Without valid type support, you cannot create RCL publishers, and topics won't appear in the ROS graph!
The Solution: Runtime Dynamic Loading
roc
solves this through dynamic library loading - a powerful systems programming technique:
Key Insight: ROS2 Type Support Libraries
ROS2 installations contain pre-compiled type support libraries:
/opt/ros/jazzy/lib/
├── libgeometry_msgs__rosidl_typesupport_c.so # Geometry messages
├── libstd_msgs__rosidl_typesupport_c.so # Standard messages
├── libsensor_msgs__rosidl_typesupport_c.so # Sensor messages
├── libcustom_msgs__rosidl_typesupport_c.so # Your custom messages
└── ...
Each library exports type support functions:
$ nm -D libgeometry_msgs__rosidl_typesupport_c.so | grep Twist
rosidl_typesupport_c__get_message_type_support_handle__geometry_msgs__msg__Twist
Dynamic Loading Strategy
Instead of static linking, roc
uses runtime dynamic loading:
- Construct library path from message type:
geometry_msgs/msg/Twist
→/opt/ros/jazzy/lib/libgeometry_msgs__rosidl_typesupport_c.so
- Load library dynamically using
dlopen()
(viarcutils_load_shared_library
) - Resolve type support symbol using
dlsym()
(viarcutils_get_symbol
) - Call the function to get the type support pointer
- Create real RCL publishers with valid type support
How Dynamic Loading Works
Step-by-Step Process
1. Message Type Parsing
#![allow(unused)] fn main() { // Input: "geometry_msgs/msg/Twist" let (package, message) = parse_message_type("geometry_msgs/msg/Twist")?; // package = "geometry_msgs", message = "Twist" }
2. Library Path Construction
#![allow(unused)] fn main() { // Construct library path using naming convention let library_path = format!( "/opt/ros/jazzy/lib/lib{}__rosidl_typesupport_c.so", package ); // Result: "/opt/ros/jazzy/lib/libgeometry_msgs__rosidl_typesupport_c.so" }
3. Symbol Name Construction
#![allow(unused)] fn main() { // Construct symbol name using ROS2 naming convention let symbol_name = format!( "rosidl_typesupport_c__get_message_type_support_handle__{}__msg__{}", package, message ); // Result: "rosidl_typesupport_c__get_message_type_support_handle__geometry_msgs__msg__Twist" }
4. Dynamic Library Loading
#![allow(unused)] fn main() { unsafe { // Initialize library handle let mut shared_lib = rcutils_get_zero_initialized_shared_library(); // Load the shared library let ret = rcutils_load_shared_library( &mut shared_lib, library_path_c.as_ptr(), allocator, ); if ret != 0 { return Err(anyhow!("Failed to load library")); } } }
5. Symbol Resolution
#![allow(unused)] fn main() { unsafe { // Get the symbol from the loaded library let symbol_ptr = rcutils_get_symbol(&shared_lib, symbol_name_c.as_ptr()); if symbol_ptr.is_null() { return Err(anyhow!("Symbol not found")); } // Cast to function pointer and call it type TypeSupportGetterFn = unsafe extern "C" fn() -> *const rosidl_message_type_support_t; let type_support_fn: TypeSupportGetterFn = std::mem::transmute(symbol_ptr); let type_support = type_support_fn(); } }
6. RCL Publisher Creation
#![allow(unused)] fn main() { unsafe { // Now we can create a real RCL publisher! let ret = rcl_publisher_init( &mut publisher, node, type_support, // ← Valid type support from dynamic loading topic_name_c.as_ptr(), &options, ); // Publisher is registered in ROS graph and appears in topic lists! } }
Implementation Architecture
Core Components
1. DynamicMessageRegistry
File: src/shared/dynamic_messages.rs
Central registry for loading and caching message types:
#![allow(unused)] fn main() { pub struct DynamicMessageRegistry { loaded_types: HashMap<String, DynamicMessageType>, } impl DynamicMessageRegistry { pub fn load_message_type(&mut self, type_name: &str) -> Result<DynamicMessageType> { // 1. Parse message type // 2. Load type support dynamically // 3. Cache result // 4. Return type info with valid type support pointer } } }
2. Generic Type Support Loading
#![allow(unused)] fn main() { fn try_get_generic_type_support( &self, package_name: &str, message_name: &str, ) -> Result<*const rosidl_message_type_support_t> { // Automatic library path construction let library_path = format!("/opt/ros/jazzy/lib/lib{}__rosidl_typesupport_c.so", package_name); // Automatic symbol name construction let symbol_name = format!( "rosidl_typesupport_c__get_message_type_support_handle__{}__msg__{}", package_name, message_name ); // Dynamic loading self.load_type_support_from_library(&library_path, &symbol_name) } }
3. Bindgen Integration
File: rclrs/build.rs
Exposes dynamic loading functions to Rust:
#![allow(unused)] fn main() { let bindings = bindgen::Builder::default() .header("wrapper.h") // Dynamic loading functions .allowlist_function("rcutils_load_shared_library") .allowlist_function("rcutils_get_symbol") .allowlist_function("rcutils_unload_shared_library") .allowlist_function("rcutils_get_zero_initialized_shared_library") // Type support types .allowlist_type("rosidl_message_type_support_t") .allowlist_type("rcutils_shared_library_t") .generate()?; }
Data Flow
User Command: roc topic pub /test geometry_msgs/msg/Twist '{linear: {x: 0.5}}'
↓
1. Parse message type: "geometry_msgs/msg/Twist"
↓
2. Construct library path and symbol name
↓
3. Load: /opt/ros/jazzy/lib/libgeometry_msgs__rosidl_typesupport_c.so
↓
4. Resolve: rosidl_typesupport_c__get_message_type_support_handle__geometry_msgs__msg__Twist
↓
5. Call function → Get type_support pointer
↓
6. Create RCL publisher with valid type_support
↓
7. Topic appears in ROS graph! ✅
Code Walkthrough
Complete Type Support Loading Function
#![allow(unused)] fn main() { fn load_type_support_from_library( &self, library_name: &str, symbol_name: &str, ) -> Result<*const rosidl_message_type_support_t> { use std::ffi::CString; unsafe { // Step 1: Initialize shared library handle let mut shared_lib = rcutils_get_zero_initialized_shared_library(); // Step 2: Convert library name to C string let lib_name_c = CString::new(library_name) .map_err(|e| anyhow!("Invalid library name '{}': {}", library_name, e))?; // Step 3: Load the shared library let allocator = rcutils_get_default_allocator(); let ret = rcutils_load_shared_library( &mut shared_lib, lib_name_c.as_ptr(), allocator, ); if ret != 0 { // RCUTILS_RET_OK is 0 return Err(anyhow!("Failed to load library '{}': return code {}", library_name, ret)); } // Step 4: Convert symbol name to C string let symbol_name_c = CString::new(symbol_name) .map_err(|e| anyhow!("Invalid symbol name '{}': {}", symbol_name, e))?; // Step 5: Get the symbol from the library let symbol_ptr = rcutils_get_symbol(&shared_lib, symbol_name_c.as_ptr()); if symbol_ptr.is_null() { rcutils_unload_shared_library(&mut shared_lib); return Err(anyhow!("Symbol '{}' not found in library '{}'", symbol_name, library_name)); } // Step 6: Cast the symbol to a function pointer and call it type TypeSupportGetterFn = unsafe extern "C" fn() -> *const rosidl_message_type_support_t; let type_support_fn: TypeSupportGetterFn = std::mem::transmute(symbol_ptr); let type_support = type_support_fn(); // Step 7: Validate the result if type_support.is_null() { return Err(anyhow!("Type support function returned null pointer")); } println!("Successfully loaded type support for symbol: {}", symbol_name); Ok(type_support) } } }
Publisher Creation with Dynamic Type Support
#![allow(unused)] fn main() { fn create_dynamic_publisher( context: &RclGraphContext, topic_name: &str, message_type: &str, ) -> Result<rcl_publisher_t> { // Load type support dynamically let mut registry = DynamicMessageRegistry::new(); let message_type_info = registry.load_message_type(message_type)?; let type_support = message_type_info.type_support .ok_or_else(|| anyhow!("Could not load type support for {}", message_type))?; unsafe { let mut publisher = rcl_get_zero_initialized_publisher(); let options = rcl_publisher_get_default_options(); let topic_name_c = CString::new(topic_name)?; // Create publisher with dynamically loaded type support! let ret = rcl_publisher_init( &mut publisher, context.node(), type_support, // ← This comes from dynamic loading topic_name_c.as_ptr(), &options, ); if ret != 0 { return Err(anyhow!("Failed to create publisher: {}", ret)); } Ok(publisher) } } }
Generic Type Support Resolution
Fallback Hierarchy
roc
uses a smart fallback strategy:
#![allow(unused)] fn main() { fn try_get_type_support(&self, package_name: &str, message_name: &str) -> Result<TypeSupport> { let full_type = format!("{}/msg/{}", package_name, message_name); match full_type.as_str() { // 1. Optimized paths for common types "geometry_msgs/msg/Twist" => self.try_get_twist_type_support(), "std_msgs/msg/String" => self.try_get_string_type_support(), "std_msgs/msg/Int32" => self.try_get_int32_type_support(), "std_msgs/msg/Float64" => self.try_get_float64_type_support(), // 2. Generic fallback for ANY message type _ => self.try_get_generic_type_support(package_name, message_name), } } }
Automatic Library Discovery
The generic loader automatically constructs paths:
Message Type | Library Path | Symbol Name |
---|---|---|
geometry_msgs/msg/Twist | /opt/ros/jazzy/lib/libgeometry_msgs__rosidl_typesupport_c.so | rosidl_typesupport_c__get_message_type_support_handle__geometry_msgs__msg__Twist |
custom_msgs/msg/MyType | /opt/ros/jazzy/lib/libcustom_msgs__rosidl_typesupport_c.so | rosidl_typesupport_c__get_message_type_support_handle__custom_msgs__msg__MyType |
sensor_msgs/msg/Image | /opt/ros/jazzy/lib/libsensor_msgs__rosidl_typesupport_c.so | rosidl_typesupport_c__get_message_type_support_handle__sensor_msgs__msg__Image |
Testing the Generic Loader
# These all work automatically:
roc topic pub /test1 geometry_msgs/msg/Twist '{linear: {x: 1.0}}' # Known type
roc topic pub /test2 geometry_msgs/msg/Point '{x: 1.0, y: 2.0, z: 3.0}' # Generic loading
roc topic pub /test3 sensor_msgs/msg/Image '{header: {frame_id: "camera"}}' # Generic loading
roc topic pub /test4 custom_msgs/msg/MyType '{my_field: "value"}' # Your custom types!
Output shows the dynamic loading in action:
Attempting generic type support loading:
Library: /opt/ros/jazzy/lib/libgeometry_msgs__rosidl_typesupport_c.so
Symbol: rosidl_typesupport_c__get_message_type_support_handle__geometry_msgs__msg__Point
Successfully loaded type support for symbol: ...
Successfully created RCL publisher with real type support!
Benefits and Trade-offs
Benefits ✅
-
Universal Message Support
- Works with any ROS2 message type
- Supports custom packages automatically
- No compilation required for new types
-
Small Binary Size
- No static linking of message libraries
- Only loads what's actually used
- Minimal memory footprint
-
Runtime Flexibility
- Discover available message types at runtime
- Work with packages installed after compilation
- Perfect for generic tools like
roc
-
Performance
- Type support loaded once and cached
- No runtime overhead after initial load
- Real RCL integration (not simulation)
-
Maintainability
- No manual type definitions required
- Automatic support for new ROS2 versions
- Self-discovering architecture
Trade-offs ⚖️
-
Runtime Dependencies
- Requires ROS2 installation with typesupport libraries
- Fails gracefully if libraries are missing
- Error messages help diagnose missing packages
-
Platform Assumptions
- Assumes standard ROS2 installation paths
- Library naming conventions must match
- Works with standard ROS2 distributions
-
Error Handling Complexity
- Must handle dynamic loading failures
- Symbol resolution errors need clear messages
- Graceful degradation for partial installations
Future Enhancements
1. Introspection-Based Generic Serialization
The next evolution is fully generic serialization using ROS2's introspection API:
#![allow(unused)] fn main() { // Future: No manual serialization needed! pub fn serialize_any_message( yaml_value: &YamlValue, type_support: *const rosidl_message_type_support_t, ) -> Result<Vec<u8>> { // 1. Get introspection data from type_support let introspection = get_message_introspection(type_support)?; // 2. Walk the message structure automatically let message_ptr = allocate_message_memory(introspection.size_of); serialize_fields_recursively(yaml_value, introspection.members, message_ptr)?; // 3. Use RMW to serialize to CDR format let serialized = rmw_serialize(message_ptr, type_support)?; Ok(serialized) } }
2. Automatic Package Discovery
#![allow(unused)] fn main() { // Future: Scan filesystem for available message types pub fn discover_available_message_types() -> Vec<String> { let lib_dir = "/opt/ros/jazzy/lib"; let pattern = "lib*__rosidl_typesupport_c.so"; // Scan libraries and extract symbols scan_libraries_for_message_types(lib_dir, pattern) } }
3. Message Definition Introspection
#![allow(unused)] fn main() { // Future: Runtime message structure inspection pub fn get_message_definition(message_type: &str) -> Result<MessageDefinition> { let type_support = load_type_support(message_type)?; let introspection = get_introspection_data(type_support)?; // Return complete message structure info Ok(MessageDefinition { fields: extract_field_definitions(introspection), dependencies: find_nested_types(introspection), size: introspection.size_of, }) } }
4. Performance Optimizations
- Library preloading for common types
- Symbol caching across multiple calls
- Memory pool for message allocation
- Batch operations for multiple message types
Conclusion
Dynamic message type loading is a sophisticated technique that gives roc
universal ROS2 message support without the limitations of static linking. By leveraging:
- Runtime dynamic library loading (
dlopen
/dlsym
) - ROS2 type support architecture
- Automatic path and symbol construction
- Graceful fallback strategies
roc
can work with any ROS2 message type - including custom packages you create! This makes it a truly generic and powerful tool for ROS2 development.
The implementation demonstrates advanced systems programming concepts while remaining maintainable and extensible. It's a great example of how understanding the underlying architecture (ROS2 type support system) enables building more flexible and powerful tools.
Key takeaway: Dynamic loading isn't just a neat trick - it's a fundamental technique that enables building truly generic and extensible systems that can adapt to runtime conditions and work with code that didn't exist at compile time.