mpremote: do not list bogus devices #14374
Conversation
|
I should add that this problem only started from kernel 6.7+. Works fine (i.e. bogus devices are not listed) on Linux kernel 6.6 and earlier. |
bulletmark
force-pushed
the
mpremote-fix
branch
from
7110e10
to
b532105
Compare
|
Code size report: |
bulletmark
force-pushed
the
mpremote-fix
branch
from
b532105
to
d02d30a
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #14374 +/- ##
=======================================
Coverage 98.39% 98.39%
=======================================
Files 161 161
Lines 21204 21204
=======================================
Hits 20864 20864
Misses 340 340 ☔ View full report in Codecov by Sentry. |
|
Strange, running Kernel 6.8 on a Pi 5 here and do not see this problem. I also don't have a dozen Devices prefixed Ironically I think your issue is some bug in your device tree config that's enabling 32, non-existent hardware serial ports... |
|
I'm on a stock Arch x86_64 system with stock kernel etc. I have used and these look exactly the same regardless of which kernel I run. I run a few other Arch 64 servers, e.g. on a N100 SFF box, and on a DigitalOcean VM, and when I check them I see they also show 32 ports same as above. Regarding your comment I don't think it's valid to exclude them here- since it's entirely possible (and more so plausible with the behaviour of the new USB functionality) to be connecting to a MicroPython device via an onboard hardware serial port. Of course I am certainly not suggesting to exclude any potentially valid serial ports. The PR only filters out those which are clearly bogus (i.e. reported above as |
No it doesn't. It filters out any with a Here's what a perfectly valid hardware UART looks like on an embedded system: |
|
So when a device is connected to that port how is it reported then? |
The same. UART does nothing until you open and use it. It's just a couple of pins you can choose (or not choose) to wiggle. There's no VID/PID/serial or other identifier, since those all come from the USB->UART bridge (be it hardware or software) that's typically used in USB CDC configurations. Try running the following on your system. This reports the import serial.tools.list_ports
for p in serial.tools.list_ports.comports():
print("{} {} {} {:04x}:{:04x} {} {}".format(
p.device,
p.subsystem,
p.serial_number,
p.vid if isinstance(p.vid, int) else 0,
p.pid if isinstance(p.pid, int) else 0,
p.manufacturer,
p.product,
)
)I wonder if the |
|
On kernel 6.6 that prints out nothing. On kernel 6.8: |
|
~Your ports didn't get renamed across Kernels from
I've no idea whether this is actually correct since their only source is 23 years out of date and I wont pretend to know the idiosyncrasies of UNIX, Linux and POSIX- https://web.archive.org/web/20010205041900/http://www.easysw.com/~mike/serial/serial.html~ Edit: Sorry I re-read your reply above, tty* vs ttyS* is me barking up the wrong tree. I've confirmed with some other Arch users and slightly updated the code below to reflect this- I've plucked out the code for import glob
import serial.tools.list_ports_common
from serial.tools.list_ports_linux import SysFS
def comports(include_links=False):
devices = set()
devices.update(glob.glob('/dev/ttyS*')) # built-in serial ports
devices.update(glob.glob('/dev/ttyUSB*')) # usb-serial with own driver
devices.update(glob.glob('/dev/ttyXRUSB*')) # xr-usb-serial port exar (DELL Edge 3001)
devices.update(glob.glob('/dev/ttyACM*')) # usb-serial with CDC-ACM profile
devices.update(glob.glob('/dev/ttyAMA*')) # ARM internal port (raspi)
devices.update(glob.glob('/dev/rfcomm*')) # BT serial devices
devices.update(glob.glob('/dev/ttyAP*')) # Advantech multi-port serial controllers
devices.update(glob.glob('/dev/ttyGS*')) # https://www.kernel.org/doc/Documentation/usb/gadget_serial.txt
if include_links:
devices.update(list_ports_common.list_links(devices))
return [SysFS(d) for d in devices]
for p in comports():
print("{} {} {} {:04x}:{:04x} {} {}".format(
p.device,
p.subsystem,
p.serial_number,
p.vid if isinstance(p.vid, int) else 0,
p.pid if isinstance(p.pid, int) else 0,
p.manufacturer,
p.product,
)
) |
|
Note: I've got independent confirmation that the 6.6 -> 6.8 Kernel transition does indeed change the subsystem of these nodes from "platform" to "serial-base" which, afaik, is a bug in the Arch kernel/system and not this tool. I haven't got the first clue where this change and how to report this as a bug, though... |
|
The output from that second script on 6.8 is exactly the same as the previous output I pasted here. On 6.6, it is also the same except Note Arch is a distro and merely packages up the stock Linux kernel so this is not a bug in Arch. Report it to Linus :). This is clearly a bug somewhere (possibly pyserial needs an update for kernels >= 6.7?) but regardless I don't see why |
This change needed on systems with Linux kernel >= 6.7. This is likely a temporary change until the issue discussed in micropython/micropython#14374 is addressed.
So Arch has made a change to cause this. Sounds an awful lot like an Arch problem and not a the-way-serial-ports-have-been-enumerated-for-25-years-needs-to-change problem... 😆
Then why does my 6.8 Raspberry Pi kernel not have the same issue? I guess the short answer to this is that Debian uses /dev/tty* rather than /dev/ttyS* so the ttys don't leak into UART enumeration at all... but it's notable that they also a platform of In Arch, it could easily be a configuration issue that's surfaced by a Kernel change- but again, I've no earthly clue where to locate what actually changed and why. Since "platform" vs "serial-base" is, afaict, the canonical way to tell (I realise my tone probably comes across as "some guy being awkward for the sake of awkward" here, and maybe in some way I am- but I promise I'm just trying to get to the actual root issue because this change might affect me, or our production product provisioning process. P.S. I don't have anything against Arch) Anyway I've stated my case, we've got to the root cause, and - indeed - we probably need some guidance from actual maintainers on this 😆 |
|
@Gadgetoid what kernel version are you running on your RPi? And what output do you get from |
|
I see the same behaviour on kernel |
|
@projectgus I think you and Gadgetoid are misunderstanding this PR. It does not prohibit Yep, I can see that I have 32 serial ports. Possibly there is an MP device connected to one of them but the above output is pointless. |
|
It looks like Arch changed from 4 runtime UARTs to 32 a while back in their kernel configuration: https://bbs.archlinux.org/viewtopic.php?id=271561 As noted in that thread, there's a kernel command line parameter that lets you turn this value down. So I think this is both an x86_64 thing, and probably an Arch-specific thing. I can't explain why the 32 UARTs apparently didn't show on the arch-lts kernel, as the config suggests it's the same there too: Maybe on some generic x86 systems there wasn't an 8250 driver being loaded until recently, that might be the root cause there. |
That is a worthwhile distinction, thanks @bulletmark . Perhaps a more specific description of what this PR does is:
The additional metadata is all USB related, so that's effectively what this change does. There is still a downside from this, for example if you're on a system where you don't remember what the hardware serial ports are named (i.e. /dev/ttyAMAx on ARM) then you might reasonably expect More generally, I'd suggest the principle of least surprise here is that "mpremote connect list" lists the allowed values that can be supplied to "mpremote connect". That's symmetrical, at least. So it should include any hardware ports that the operating system says are there. It's unfortunate and kind of weird that there doesn't seem to be a way to tell if a hardware serial port is "bogus" without opening it (which immediately fails, at least for all the ttyS devices on my serial-port-free laptop!) |
|
I'm also seeing this issue on Arch Linux. It's annoying, but... I also see the same problem now with Chromium's available list of serial devices when using WebSerial (on Arch Linux, with Chromium). When you attempt to open a WebSerial connection, the Chromium pop-up dialog presents you with 32 "bogus" tty devices. So that makes me think it's really an Arch Linux "bug". |
|
@dpgeorge it is only a problem on Arch because Arch uses bleeding edge Linux kernel versions and it only happened on the update of the kernel to 6.7 (6.6 and earlier are fine). Thus it will likely be a problem of all other distros when they catch up. At the end of the day this PR is a trivial one-line if test and just skips clearly bogus devices from being output in the |
Signed-off-by: Mark Blakeney <mark.blakeney@bullet-systems.net>
On Linux, since kernel 6.7 (and now 6.8) we get the following:
With this PR applied: