Replies: 4 comments 4 replies
-
|
I do like the idea of a more dependable transport! As flexible as raw repl is, yeah it's not always reliable enough for all applications. It'll need a few more layers than just the packet structure though; the actual logic of what to do when a corrupted packet is detected is where the real complication lies. Personally I've been starting to lean into using a network connection more and more for this kind of purpose, though I've been sticking to hardware with a real network port and working on a USB-Net impl. A growing number of ports support having PPP enabled; this is the traditional gold standard of reliable serial protocols that has this stuff figured out. I've thought about maybe trying to add PPP support directly into mpremote to avoid needing to set up modem handling in the OS. I haven't system this though, and for seamless operation it'd need a RAW-PPP mode switch sequence to switch the repl transport into PPP mode and maybe provide a native REPL-over-socket handler to keep mpremote working. Other types of data could be sent over separate sockets then. If that still feels too much, I'd strongly prefer to find a library / protocol that's already proven trustworthy to provides handshaking, sync, corruption detection, retransmission etc. if that framing and reliable transport layer is proven and efficient, then your payload structure to provide a more structured way of sending different kinds of data could certainly prove useful! |
Beta Was this translation helpful? Give feedback.
-
|
Hi, PPP is a good option if we enable IP stack on all devices and have REPL over TCP. I am bit worried about security, but we can bind these services to the dedicated interface. Not sure how to consider the additional interface address, maybe we can have a dedicated loopback interface, aka 127.0.0.x. This is larger change than I had in mind... it will provide a great flexibility. Is lwip dependency acceptable for core? I guess some changes are required to support parallel ppp instances, one will be allocated to the REPL. Thanks, |
Beta Was this translation helpful? Give feedback.
-
|
I might be wrong, but I see the REPL as a tool for development, not for regularly transferring files from/to the device. In development mode, you have the device on the desk with some short (USB-?)cables. Besides that, inventing some new framing/protocol comes with drawbacks (backward compatibility, unreliable until development is real finished, tooling support). OK, using ftp (or another TCP-based protocol) over SLIP or PPP to make the link reliable seems to be ruled out for devices with limited memory. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @karfas, The REPL is not only used during application development, it also plays an important role in manufacturing, software updates, system bring-up, diagnostics, and customer support. In many devices, USB is not available (either due to hardware constraints or because it is already allocated to the functional firmware). In such cases, the debug UART is the only available communication channel, and this is exactly how we operate today. Because of that, the robustness of REPL transport becomes critical. While ZModem is indeed a strong option for file transfer, it doesn’t fully address the broader requirements of the REPL environment. The REPL provides:
ZModem only solves the file transfer portion, not the multiplexed program-and-control flow the REPL already supports. This is why a ZModem-only solution is not a complete match. The transport we really need is conceptually similar to what SSH provides: As noted in my earlier proposal, a lightweight framed protocol would provide:
Regarding backward compatibility: As for adding a full TCP/IP stack just to achieve reliable console behavior, this feels quite heavy for the problem at hand. If that ends up being the preferred direction, I’m happy to help implement it. Thanks, |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Extended-Raw REPL Mode Proposal
Summary
MicroPython currently provides two special REPL protocols: paste mode and raw mode. Paste mode is optimized for human input, while raw mode is optimized for machine-driven execution.
However, when the REPL runs over unreliable transports—particularly serial links with possible transmission or receive errors—neither mode provides any mechanism for framing, error detection, or recovery. Higher-level tools built on top of REPL (e.g., file transfer utilities such as #18420) are therefore vulnerable to corruption and desynchronization.
This proposal introduces a new, optional extended-raw mode that adds structured framing, integrity checks, and support for reliable multi-channel communication.
Motivation
Machine communication with MicroPython devices commonly occurs over serial interfaces that may experience:
In the current REPL design:
As a result:
A modern, structured transport mode would enable tooling and automation to be significantly more robust.
Proposed Solution: Extended-Raw Mode
Add a new REPL mode that encapsulates messages in well-defined packets with framing, length, sequence numbering, and CRC.
Packet Structure
Payload Structure
Example command types
Key Benefits
1. Reliable Communication
CRC and sequence numbers allow detection and recovery from errors, missing packets, and corruption.
2. Tooling Support
File transfer, remote execution, and debugging tools can operate reliably even on noisy links.
3. Multi-channel Communication
Enables clean separation of stdout, stderr, stdin, control signals, and metadata.
4. Automatic Resynchronization
The fixed signature allows clients to re-align with the stream after corruption.
Backward Compatibility
Conclusion
Adding extended-raw mode provides a robust, fault-tolerant transport layer for REPL-based communication. This significantly improves reliability for embedded development and automation workflows.
Beta Was this translation helpful? Give feedback.
All reactions