Yup, each observation is in its own loop. See the following log messages 4/1/25 17:46:44 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174643 EDT (1743544003) 'dateTime' '1743544003.599552', 'lightning_distance' '22.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.06267', 'lightning_strike_count' '0.0', 'usUnits' '17' 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.066883', 'lightning_strike_count' '0.0', 'usUnits' '17' 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.0723326', 'lightning_strike_count' '0.0', 'usUnits' '17' 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.063789', 'lightning_distance' '22.0', 'usUnits' '17' 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.0683196', 'lightning_distance' '22.0', 'usUnits' '17' 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) data-> final loop packet is rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 EDT (1743544014) 'dateTime' '1743544014.0791476', 'lightning_distance' '22.0', 'usUnits' '17' If subscribing to the event topic is not possible, perhaps you could apply the correction on the archive record? See, https://www.weewx.com/docs/5.1/reference/weewx-options/stdcalibrate/#corrections rich On Tuesday, 1 April 2025 at 19:35:02 UTC-4 Andrew McGinnis wrote: > *What is emitting mqtt ? * > rtl_433 runs as a service on the same system as weewx is running on, set > to output MQTT topics in SI units, via rtl_433 -C si -F mqtt. > > *Are there actual duplicate mqtt messages with the same timestamps ?* > Yes and no. The majority of all of my sensors output multiple packets, for > redundancy. Many- but not all- have a topic that is something to the > effect of "sequence", but but don't otherwise differ in the payloads of the > subscribed topics (notwithstanding Acurite's use of multiple message types > with mixed topics across them). Here's a message type 39 with wind avg, > uv, lux, and lightning topics included. The packet was broadcast in > triplicate, three copies broadcast and received, with sequence_num 1, 2, > and 3: > > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg c23be7810084cf5696a4 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 1 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg c63be7810084cf5696a8 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 2 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0 > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg ca3be7810084cf5696ac > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK > > *How is your mqtt subscribe configured ?* > What I pasted above was pulled straight from "mosquitto_sub -v -h > localhost -t "rtl_433/pi4b8/devices/Acurite-Atlas/A/571/#"". The > weewx.conf section looks like this: > [MQTTSubscribeDriver] > driver = user.MQTTSubscribe > host = 10.19.76.14 > port = 1883 > keepalive = 20 > username = None > password = None > [[message_callback]] > type = individual > [[topics]] > unit_system = METRICWX > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/humidity]]] > name = outHumidity > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/rain_mm]]] > name = rain > contains_total = true > [[[br-weewx/Acurite-Atlas/A/571/rain_mm]]] > **this one comes from a relay broker so I can capture the same topic > twice and handle it differently for QC purposes** > name = snow > contains_total = false > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/temperature_C]]] > name = outTemp > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h]]] > name = windSpeed > units = km_per_hour > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_dir_deg]]] > name = windDir > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok]]] > name = outTempBatteryStatus > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux]]] > name = luminosity > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv]]] > name = UV > > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count]]] > name = lightning_strike_count > contains_total = true > [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance]]] > name = lightning_distance > units = km > *Which mqtt subscribe variant and version are you using ?* > pi@pi4b8:/etc/systemd/system $ weectl extension list > Using configuration file /etc/weewx/weewx.conf > Extension Name Version Description > MQTTSubscribe 2.3.0 Augment WeeWX records or packets with MQTT > data. > > *What do a few received mqtt messages look like ?* > Native mosquitto_sub gets what I pasted above. MQTTSubscribe doesn't put > (pretty much) anything in the log unless debug is enabled. > > *What do a few weewx loop messages with debug=1 look like ?* > with debug enabled, the weewx log looks something like this (filtered to > just lightning topics): > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count > 'lightning_strike_count' '0.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count > 'lightning_strike_count' '0.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count > 'lightning_strike_count' '0.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance > 'lightning_distance' '22.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance > 'lightning_distance' '22.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> incoming > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance > 'lightning_distance' '22.0' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime' > '1743544003.5789976', 'lightning_strike_count' '0.0', 'usUnits' '17' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime' > '1743544003.5918882', 'lightning_strike_count' '0.0', 'usUnits' '17' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime' > '1743544003.5977087', 'lightning_strike_count' '0.0', 'usUnits' '17' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime' > '1743544003.5803194', 'lightning_distance' '22.0', 'usUnits' '17' > 4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime' > '1743544003.5927773', 'lightning_distance' '22.0', 'usUnits' '17' > 4/1/25 17:46:44 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174643 > EDT (1743544003) 'dateTime' '1743544003.599552', 'lightning_distance' > '22.0', 'usUnits' '17' > 4/1/25 17:46:44 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > TopicManager data-> outgoing > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime' > '1743544003.599552', 'lightning_distance' '22.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.06267', 'lightning_strike_count' > '0.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.066883', 'lightning_strike_count' > '0.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.0723326', 'lightning_strike_count' > '0.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.063789', 'lightning_distance' > '22.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.0683196', 'lightning_distance' > '22.0', 'usUnits' '17' > 4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver) > data-> final loop packet is > rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654 > EDT (1743544014) 'dateTime' '1743544014.0791476', 'lightning_distance' > '22.0', 'usUnits' '17' > > *What do you see ?* > MQTTSubscribe looks to be passing the topic payload as the observation. > This is expected, as there's no logic applied to a non-cumulative value. > > *What do you expect/hope to see ?* > StdCalibrate should be applying the correction formula to loop packets, or > at least that's my understanding. All of my WX sensors are configured > through MQTT. When weewx receives an observation datapoint, regardless of > the driver producing it, it should apply the correction before any record > is made. In this case, > "lightning_distance = lightning_distance if lightning_strike_count > 0 > else None" should be applied akin to =IF(lightning_strike_count > 0, > lightning_distance, "None"), but instead of evaluating to the False > condition, it's evaluating to True. > > > On Tue, Apr 1, 2025 at 6:14 PM vince <[email protected]> wrote: > >> Andrew - I think you need to more clearly state your setup and which >> problem(s) you are asking for help with because I certainly got very lost >> in your posts….. >> >> What is emitting mqtt ? >> Are there actual duplicate mqtt messages with the same timestamps ? >> How is your mqtt subscribe configured ? >> Which mqtt subscribe variant and version are you using ? >> >> What do a few received mqtt messages look like ? >> What do a few weewx loop messages with debug=1 look like ? >> What do you see ? >> What do you expect/hope to see ? >> >> On Tuesday, April 1, 2025 at 1:43:56 PM UTC-7 gjr80 wrote: >> >>> A few comments. >>> >>> I don't see multiple copies of a loop packet arriving causing a problem. >>> Archive record obs that are averaged (eg outTemp) will be the same >>> irrespective of how many time the loop packet is 'accumulated'. Per-period >>> obs such as rain and, in this case, lightning_strike_count will >>> similarly be unaffected as the difference for each such obs between >>> successive duplicate loop packets will be zero. Of course cumulative obs >>> will be nonsense, but as WeeWX does not use cumulative obs that should not >>> be a problem. >>> >>> Also, keep in mind how the [StdCalibrate] [[Corrections]] operate. Each >>> [StdCalibrate] >>> [[Corrections]] option value is used as the argument to a python eval() >>> statement; if the eval() statement causes a python exception to be >>> raised the correction is silently ignored (and if it exists the destination >>> obs is left unchanged, if it does not exist it is not created). I would >>> suggest your main issue is your lightning_distance correction not >>> working as intended (are you sure lightning_strike_count is indeed 0 or >>> is it None?). Also, the 2nd of your three lightning_distance >>> corrections is the one you use; WeeWX does not use 'nan' and use of Null >>> will raise a python exception. >>> >>> I would suggest you set some appropriate debug settings (I am not >>> familiar with the MQTTSubscribe driver to know what debug settings are >>> required) in place such that loop packets emitted by the MQTTSubscribe >>> driver >>> are logged. Then you will know exactly what is being fed to WeeWX and how >>> to ensure it is processed/recorded correctly. >>> >>> Gary >>> On Wednesday, 2 April 2025 at 04:52:32 UTC+10 Andrew McGinnis wrote: >>> >>>> ETA- 5000+ strike count/min comes from the sensor broadcasting packets >>>> in triplicate, and every 10 seconds for lightning. So up to 18 messages in >>>> each 1-min interval, with what was the true cumulative count of 303, >>>> resulted in 5454 strikes "recorded". >>>> >>>> If anyone has a way to suppress duplicate MQTT messages, I'm all ears. >>>> I get why sensors will send a packet multiple times to maximize the >>>> chances >>>> of reception at a station, but weewx (or, at least MQTTSubscribeDriver) >>>> sees each one and records it as a separate datapoint to aggregate in the >>>> archive period. Mostly harmless, but not entirely accurate. >>>> On Tuesday, April 1, 2025 at 2:28:50 PM UTC-4 Andrew McGinnis wrote: >>>> >>>>> I just set up an Acurite Atlas with the lightning detector sensor >>>>> add-on. The strike_count that it outputs is cumulative (after I found I >>>>> had 5000+ lightning strikes per 1min archive interval). I have it, and >>>>> the >>>>> strike_distance outputs mapped to weewx as below. rtl_433 is converting >>>>> strike_distance from the imperial 'miles' that is natively output, to >>>>> metric 'km', hence the additional units switch: >>>>> >>>>> [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count]]] >>>>> name = lightning_strike_count >>>>> contains_total = true >>>>> [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance]]] >>>>> name = lightning_distance >>>>> units = km >>>>> >>>>> The sensor output always has a strike count and distance, but since >>>>> the strike count is cumulative, there should be no recorded distance if >>>>> the >>>>> delta is 0. >>>>> >>>>> In StdCalibrate/Corrections, I have these: >>>>> radiation = luminosity / 126.7 if luminosity is not None else None >>>>> (<----this is working as expected) >>>>> lightning_distance = lightning_distance if lightning_strike_count > 0 >>>>> else float('nan') (<----does not work) >>>>> lightning_distance = lightning_distance if lightning_strike_count > 0 >>>>> else None (<----does not work) >>>>> lightning_distance = lightning_distance if lightning_strike_count > 0 >>>>> else Null (<----does not work) >>>>> >>>>> The *radiation *observation is being recorded as 0 when *luminosity *is >>>>> recorded as 0. But, since 0 is a valid possible output, if 0 != None, >>>>> then >>>>> 0/126.7 = 0. Fair enough. >>>>> The *lightning_distance* should be evaluating as 0 or None, but >>>>> instead is being recorded as the sensor's output value, while >>>>> *lightning_strike_count* is recorded as 0. I went into the db and ran >>>>> the following, then rebuilt the affected dailies: >>>>> >>>>> - update archive set lightning_strike_count = 0 where >>>>> lightning_strike_count > 5; >>>>> - update archive set lightning_distance = 0 where >>>>> lightning_strike_count = 0; >>>>> >>>>> strike_count > 5 was because I caught my contains_total omission mid >>>>> day yesterday, and there was thunder last night. Strikes before the storm >>>>> were largely invalid, but distance wasn't necessarily. >>>>> >>>>> [image: Screenshot 2025-04-01 141609.png] >>>>> Straight away, *lightning_distance* is being recorded as the sensor's >>>>> value, instead of 0, or ideally None/null. >>>>> 0 is a valid possible distance, assuming I get a direct strike. >>>>> Without a strike, it's not 0 distance- there *isn't* a distance. >>>>> >>>>> Rather than continue trying random changes to the Corrections logic, >>>>> can anyone point to where the issue or mistake may lie? Is it something >>>>> to >>>>> do with the referenced *lightning_strike_count* being a cumulative >>>>> number? Do I just need to yield and make the logic evaluate to 0? >>>>> >>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "weewx-user" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/weewx-user/b3O1IzVjiCc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/weewx-user/2e0d1f8c-8531-401e-9bfd-f93123d8ec79n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/2e0d1f8c-8531-401e-9bfd-f93123d8ec79n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/bd56fa81-303a-44dc-9a44-c7d62937ed74n%40googlegroups.com.
