Connectivity Drop with More Than 5 Devices in Micro-ROS Agent on Docker (Raspberry Pi 5)

[SpaceBod]

Description:
I am running a micro-ROS agent in a Docker container using the prebuilt image (humble) on a Raspberry Pi 5. I have several ESP32 devices with micro-ROS (Humble). When I attempt to connect more than five devices to the agent, I experience connectivity dropouts, where some devices fail to stay connected. With 5 devices, they are successful in subscribing to topics.

I suspect this issue might be related to the configuration flags in the micro-ROS build, particularly those related to client and resource limits. I came across the following parameters:

• RMW_UXRCE_MAX_CLIENTS
• RMW_UXRCE_MAX_SUBSCRIPTIONS
• RMW_UXRCE_MAX_SESSIONS
• RMW_UXRCE_MAX_NODES
• RMW_UXRCE_MAX_PUBLISHERS
• RMW_UXRCE_MAX_SERVICES
• RMW_UXRCE_MAX_WAIT_SETS
• RMW_UXRCE_MAX_GUARD_CONDITION
• RMW_UXRCE_MAX_TOPICS

These parameters seem critical to configuring the agent, especially as I'm dealing with multiple devices where each subscribes to three topics. I suspect increasing the RMW_UXRCE_MAX_CLIENTS value might help, but I'm not entirely sure.

What I've Tried:

• I attempted to experiment by manually setting RMW_UXRCE_MAX_CLIENTS to 1 (as a test), but it did not yield the expected results, as multiple clients could still connect.
• I've tried searching through the documentation, including Vulcanexus for guidance, but I haven't been able to solve the issue.

Question:
In order to adjust these flags and rebuild the micro-ROS agent from source, do I need to modify the build configuration in the following file?

• src/micro_ros_setup/config/host/generic/client-host-colcon.meta

Or should the changes be made in this file instead?

• src/uros/rmw_microxrcedds/rmw_microxredds_c/CMakeLists.txt

I believe the latter might be where the flags should be set, but I'm uncertain. Has anyone else experienced similar connectivity issues when connecting more than five devices to a micro-ROS agent on Docker? If so, how did you resolve it?

(I've also noticed a colcon.meta file in the /src folder of my workspace, should the be used?)

Any guidance on where exactly to set the flags for a manual source installation would be greatly appreciated - Or even the correct approach to setting up an agent to allow for more than 20 clients!

Thanks in advance for your help!

System Details:

• Hardware: Raspberry Pi 5
• Docker Image: Prebuilt micro-ROS agent
• Micro-ROS Version: Humble on ESP32 devices
• Number of Devices: 5+ ESP32 devices
• Topics per Device: 3 Topics per device

Links to Documentation:

• Vulcanexus Documentation on Memory Management
• micro_ros_setup Repository
• rmw_microxrcedds Repository

Thank you!

[simonbogh]

Do you connect your devices over serial or udp/tcp?

[SpaceBod]

We are using UDP/TCP.

[SpaceBod]

Update, we have tried to use serial running on one agent via multi-serial:

• When implementing a reconnection mechanism, issues emerged:
  • Devices pinged the Agent to verify liveliness before initiating reconnection.
• Latency scaled poorly with device count:
  • At low device counts (≤3), latency was stable (e.g., 9 ms).
  • As device count increased (≥4), ping responses fluctuated dramatically (up to 600+ ms).
• Even with a 3-second timeout window, devices incorrectly assumed disconnection, triggering unnecessary reconnects.
• Reconnect attempts would conflict, breaking otherwise healthy connections and destabilising the system.