Re: [Dorset] Python Problem in MQTT subscriber program
On Fri, 27 Aug 2021 14:02:19 +0100, PeterMerchant wrote: > Patrick, I am most grateful for your comments. You obviously know a > lot more about MQTT than I. I wouldn't say I know a lot about MQTT, and I did not mean to imply that I do. I have a fairly basic high-level understanding of it, but I've forgotten most of the lower-level details I previously had a light grasp of! I haven't looked at paho-mqtt or Arduino MQTT before, so for my own understanding I had to go back over the basics that you are probably used to dealing with. I may well have learnt more from trying to find the problem in your code than you will have learnt from me! I didn't want to make any assumptions about what kind of data your system was sending and how it was packaging it up and unpackaging it, because there is potentially more than one MQTT client library for Python, and there is certainly more than one way to encode an MQTT payload. For example, some people might use Google's Protocol Buffers or another serialisation scheme to encode program objects into bytes and then stuff those into an MQTT payload. Others might define a scheme of their own. In your case, it is ASCII text (a perfectly valid option). Just in case it was unclear to anyone: when I said the payload is bytes, rather than a text string, what I meant was that it is not a Python text string object, and MQTT does not require it to encode a text string. However, in this particular program it does encode an ASCII text string, so in that sense it IS very much a text string, and Python makes it easy to treat it as one. Patrick -- Next meeting: Online, Jitsi, Tuesday, 2021-09-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Python Problem in MQTT subscriber program
Patrick, I am most grateful for your comments. You obviously know a lot more about MQTT than I. I think that I'll have a little play with the software that I have working, and then start cleaning it up using some of your suggestions. And perhaps read a bit more of the MQTT v5 spec. Much thanks, Peter On 27/08/2021 12:23, Patrick Wigmore wrote: Hmm. I don't think I'm able to solve this. Referring to the full MQTT subscriber/car control program you sent me off-list, this is some of the context I was missing: It is using paho-mqtt, f.k.a. Mosquitto, and message is a MQTTMessage object, with message.payload being arbitrary bytes. It looks like this payload is indeed the unadulterated MQTT payload, as sent over the network. So, the payload is bytes, rather than a text string, and there isn't any intervening code that's not shown and does additional encoding of the payload. on_message is run by paho-mqtt as an on_message callback function when an MQTT PUBLISH message is received in a subscribed topic, which I believe is what you intend. The publisher is using the Arduino MQTT Client, which wraps lwmqtt. This much is probably obvious if you've been immersed in it for a while. So, regardless of the payload contents: * Given that message.payload is a Python Bytes object, which it should be, print(message.payload) should always work. Bytes that don't encode printable characters print as hexadecimal within an escape sequence. * Testing whether the payload equals some specific bytes should at least not cause a crash, even if the test doesn't evaluate to True/ False as expected. * The if/elif/else structure looks fine. So, unless I'm missing a syntax error or typo somewhere, I don't think there's anything wrong with your non-working on_message! Just to confirm to myself how Python's print handles unprintable bytes: >>> print(b'\x00\x00hello\xff\x68\x65\x6c\x6c\x6f') b'\x00\x00hello\xffhello' I'll assume that all the functions that do the GPIO (e.g. motorleft) are correct. That could be a wrong assumption, but I'll assume it anyway because it seems likely. Given that you were interpreting the same bytes (b'Pot-left' and b'Pot-rt') successfully before, all I can think is that the code at the publishing end must have changed too, to get it to send different bytes. Since the subscriber crashes, there must be something about the way the publishing code was changed that causes it to send messages that paho-mqtt can't handle. Because paho-mqtt shouldn't care about the contents of the payload, this would seem to suggest that there is a mistake in the way the MQTT packets are being constructed at the publisher. I'm having a hard time imagining that you would have intentionally significantly altered the code at the publishing end, though, so it is probably something quite subtle; perhaps an issue that was already there but is only tripped up by something else that changed. But I feel like I'm probably missing something obvious here. Normally when I try to make these kinds of deductions on the list, someone comes along and points out a glaring error in my logic, so no doubt I am making a similar error right now! Patrick -- Next meeting: Online, Jitsi, Tuesday, 2021-09-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Python Problem in MQTT subscriber program
Hmm. I don't think I'm able to solve this. Referring to the full MQTT subscriber/car control program you sent me off-list, this is some of the context I was missing: It is using paho-mqtt, f.k.a. Mosquitto, and message is a MQTTMessage object, with message.payload being arbitrary bytes. It looks like this payload is indeed the unadulterated MQTT payload, as sent over the network. So, the payload is bytes, rather than a text string, and there isn't any intervening code that's not shown and does additional encoding of the payload. on_message is run by paho-mqtt as an on_message callback function when an MQTT PUBLISH message is received in a subscribed topic, which I believe is what you intend. The publisher is using the Arduino MQTT Client, which wraps lwmqtt. This much is probably obvious if you've been immersed in it for a while. So, regardless of the payload contents: * Given that message.payload is a Python Bytes object, which it should be, print(message.payload) should always work. Bytes that don't encode printable characters print as hexadecimal within an escape sequence. * Testing whether the payload equals some specific bytes should at least not cause a crash, even if the test doesn't evaluate to True/ False as expected. * The if/elif/else structure looks fine. So, unless I'm missing a syntax error or typo somewhere, I don't think there's anything wrong with your non-working on_message! Just to confirm to myself how Python's print handles unprintable bytes: >>> print(b'\x00\x00hello\xff\x68\x65\x6c\x6c\x6f') b'\x00\x00hello\xffhello' I'll assume that all the functions that do the GPIO (e.g. motorleft) are correct. That could be a wrong assumption, but I'll assume it anyway because it seems likely. Given that you were interpreting the same bytes (b'Pot-left' and b'Pot-rt') successfully before, all I can think is that the code at the publishing end must have changed too, to get it to send different bytes. Since the subscriber crashes, there must be something about the way the publishing code was changed that causes it to send messages that paho-mqtt can't handle. Because paho-mqtt shouldn't care about the contents of the payload, this would seem to suggest that there is a mistake in the way the MQTT packets are being constructed at the publisher. I'm having a hard time imagining that you would have intentionally significantly altered the code at the publishing end, though, so it is probably something quite subtle; perhaps an issue that was already there but is only tripped up by something else that changed. But I feel like I'm probably missing something obvious here. Normally when I try to make these kinds of deductions on the list, someone comes along and points out a glaring error in my logic, so no doubt I am making a similar error right now! Patrick -- Next meeting: Online, Jitsi, Tuesday, 2021-09-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
