Skip to content

feat(rtps, cdr): Add cdr and rtps components#643

Open
finger563 wants to merge 20 commits into
mainfrom
feat/rtps
Open

feat(rtps, cdr): Add cdr and rtps components#643
finger563 wants to merge 20 commits into
mainfrom
feat/rtps

Conversation

@finger563

@finger563 finger563 commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds the initial cdr and rtps components and wires them into the repo, examples, docs, CI, and host-side test tooling.

Highlights:

  • add a standalone cdr component for reusable CDR / PL_CDR payload handling
  • add an rtps component built on socket / task with:
    • RTPS message/header/submessage helpers
    • SPDP participant discovery
    • SEDP publication/subscription discovery
    • temporary UInt32 user-data publish/receive support
  • update rtps to use the shared cdr component for parameter-body and payload CDR handling
  • add ESP-IDF examples for both cdr and rtps
  • update the RTPS example to support real two-node Wi-Fi testing with initiator / responder roles via menuconfig
  • add a pure-stdlib host RTPS harness in python/rtps_host.py
  • update the Python RTPS harness defaults so it works out of the box with the default ESP RTPS example
  • add / expand docs for cdr, rtps, and rtsp, including protocol explanations, Mermaid diagrams, compatibility/status tables, and spec references
  • wire the new components into Doxygen, Sphinx docs, CI build matrices, and component upload workflow

Motivation and Context

This change starts the DDS / ROS 2 interoperability foundation in ESPP by introducing:

  1. a reusable CDR serialization layer that can stand alone from DDS/RTPS, and
  2. an RTPS transport/discovery layer that is cross-platform because it is built on the existing socket component.

The immediate goal is to support standards-shaped RTPS discovery and a simple host/device validation path while keeping the architecture manageable:

  • cdr owns byte layout, encapsulation, alignment, strings, arrays, and sequences
  • rtps owns discovery, transport, GUIDs, locators, and temporary user-data exchange
  • future DDS matching / higher-level type support can build on top of these layers

This PR also improves the docs/test story so the new components are understandable and usable from both ESP targets and a PC host.

How has this been tested?

  • Built the new CDR example:
    • source ~/esp/esp-idf/export.sh && cd components/cdr/example && idf.py build
  • Built the RTPS example:
    • source ~/esp/esp-idf/export.sh && cd components/rtps/example && rm -f sdkconfig && idf.py build
  • Verified rtps builds after being updated to depend on and use cdr
  • Verified cdr and rtps public header documentation changes still compile cleanly via the example builds
  • Ran Python syntax validation:
    • python3 -m py_compile python/rtps_host.py
  • Ran a host-side RTPS loopback smoke test with two rtps_host.py processes:
    • one default responder instance
    • one initiator instance publishing espp/rtps_example/request
    • confirmed discovery plus echoed espp/rtps_example/response samples
  • Previously validated the docs pipeline updates on this branch with the repo docs build flow, including the RTPS / RTSP / CDR pages

Screenshots (if appropriate, e.g. schematic, board, console logs, lab pictures):

Showing

  • one example initiator on qtpy esp32s3
  • one example responder on qtpy esp32s3
  • python test script
CleanShot 2026-06-15 at 14 04 27@2x

Responder:
CleanShot 2026-06-15 at 14 12 10@2x

Initiator:
CleanShot 2026-06-15 at 14 12 56@2x

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update
  • Hardware (schematic, board, system design) change
  • Software change

Checklist:

  • My change requires a change to the documentation.
  • I have added / updated the documentation related to this change via either README or WIKI

Software

  • I have added tests to cover my changes.
  • I have updated the .github/workflows/build.yml file to add my new test to the automated cloud build github action.
  • All new and existing tests passed.
  • My code follows the code style of this project.

Copilot AI review requested due to automatic review settings June 15, 2026 19:06
@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown

✅Static analysis result - no issues found! ✅

Comment thread python/rtps_host.py Fixed
Comment thread python/rtps_host.py Fixed
Comment thread python/rtps_host.py Fixed
Comment thread python/rtps_host.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces two new ESPP components—cdr (Common Data Representation helpers) and rtps (an RTPS discovery + temporary UInt32 data scaffold)—and wires them into documentation and CI, including a pure-stdlib Python host harness for discovery and smoke testing.

Changes:

  • Add components/cdr with CDR/PL_CDR reader/writer utilities plus an example and docs pages.
  • Add components/rtps with SPDP/SEDP discovery framing, participant/endpoint tracking, and a temporary UInt32 user-data path, plus an ESP example and docs pages.
  • Extend documentation tooling (Mermaid support) and CI matrices to build/upload the new components.

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/rtps_host.py New pure-stdlib RTPS discovery + temporary UInt32 host harness.
python/README.md Document the new RTPS host harness and example invocations.
docker_build_docs.sh Install doc Python requirements inside the docs build container before building.
doc/requirements.txt Add sphinxcontrib-mermaid for Mermaid diagram support.
doc/en/rtsp.rst Add “How RTSP Works” explanation and Mermaid diagrams/spec links.
doc/en/rtps.rst New RTPS component documentation page (scope, flow, specs, example link).
doc/en/rtps_example.md Include RTPS example README into docs.
doc/en/index.rst Add cdr and rtps pages to the docs toctree.
doc/en/cdr.rst New CDR component documentation page.
doc/en/cdr_example.md Include CDR example README into docs.
doc/Doxyfile Add CDR/RTPS headers + examples to Doxygen inputs/example paths.
doc/conf_common.py Enable Mermaid Sphinx extension and set Mermaid configuration.
components/rtps/src/rtps.cpp Implement RTPS message framing/parsing, discovery, and temporary UInt32 data path.
components/rtps/README.md RTPS component overview, expected compatibility, and feature status.
components/rtps/include/rtps.hpp Public RTPS API: types, config, discovery, and temp UInt32 helpers.
components/rtps/idf_component.yml RTPS component manager manifest + dependencies.
components/rtps/example/sdkconfig.defaults RTPS example default config (partition table, task stacks).
components/rtps/example/README.md RTPS two-board initiator/responder smoke-test instructions.
components/rtps/example/partitions.csv Custom partition table for RTPS example.
components/rtps/example/main/rtps_example.cpp RTPS example app: Wi-Fi STA + discovery + request/response loop.
components/rtps/example/main/Kconfig.projbuild Menuconfig options for role/domain/participant/topics/Wi-Fi.
components/rtps/example/main/CMakeLists.txt Example main component registration + dependencies.
components/rtps/example/CMakeLists.txt RTPS example project setup and component list.
components/rtps/CMakeLists.txt RTPS component build registration and requirements.
components/cdr/src/cdr.cpp CDR translation unit (header-only-ish implementation anchor).
components/cdr/README.md CDR component overview and example description.
components/cdr/include/cdr.hpp CDR/PL_CDR encapsulation + reader/writer APIs and helpers.
components/cdr/idf_component.yml CDR component manager manifest.
components/cdr/example/sdkconfig.defaults Minimal defaults for the CDR example.
components/cdr/example/README.md CDR example usage and expected output.
components/cdr/example/main/CMakeLists.txt CDR example main component registration.
components/cdr/example/main/cdr_example.cpp CDR example app: serialize/deserialize primitives, strings, sequences.
components/cdr/example/CMakeLists.txt CDR example project setup and component list.
components/cdr/CMakeLists.txt CDR component build registration.
.github/workflows/upload_components.yml Add cdr and rtps to the upload list.
.github/workflows/build.yml Add CI builds for cdr/example and rtps/example.

Comment thread doc/conf_common.py Outdated
Comment thread python/rtps_host.py
Comment thread python/rtps_host.py
Comment thread components/rtps/src/rtps.cpp Outdated
@finger563 finger563 self-assigned this Jun 15, 2026
@finger563 finger563 added enhancement New feature or request rtps real time publish subscribe cdr common data representation serialization labels Jun 15, 2026
@finger563 finger563 requested a review from Copilot June 16, 2026 18:10
Comment thread python/rtps_host.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 6 comments.

Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/example/main/Kconfig.projbuild
finger563 and others added 8 commits June 16, 2026 13:30
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@finger563 finger563 requested a review from Copilot June 16, 2026 19:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

python/rtps_host.py:1

  • RTPS submessages can legally use a length of 0 to mean “extends to end of message”. With the current loop, length == 0 causes offset not to advance (after the header), leading to an infinite loop. Additionally, the code unconditionally interprets the length as little-endian without checking the submessage endianness flag. Fix by treating length == 0 as len(packet) - offset (after reading the submessage header), and either reject non-little-endian submessages or decode fields according to the endianness bit.

Comment thread components/cdr/include/cdr.hpp
Comment thread components/cdr/include/cdr.hpp
Comment thread components/rtps/src/rtps.cpp
Comment thread components/rtps/src/rtps.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cdr common data representation serialization enhancement New feature or request rtps real time publish subscribe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants