Bingo, fixed by installing v2.1.0-RC MQTTSubscribeDriver. Adding collect_observations = True and single_queue = True to [[topics], I get desired behavior. Null lightning distance is recorded in absence of strikes and with an test simulated lightning strike by battery charger shorting, I get lightning strike recording and a lightning distance. Loop shows both lightning distance and strikes in same loop:
LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: 5636.94172822, dateTime: 1628708167.42, dewpoint: 70.6358563958, heatindex: 93.4196227576, humidex: 103.698544022, lightning_distance: None, lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, outTemp: 87.8, rainRate: 0, usUnits: 1 LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: 5636.94172822, dateTime: 1628708167.43, dewpoint: 70.6358563958, heatindex: 93.4196227576, humidex: 103.698544022, lightning_distance: None, lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, outTemp: 87.8, rainRate: 0, usUnits: 1 Supporting the idea that lightning strikes and distances were being interpreted at different times I caught two lightning strikes this morning (before installing v2.1.0-RC) which shows the 8 am lightning strike before lightning distance change and the 11 am lightning strike after lightning distance change. By not being synchronized, the correction (lightning_distance = lightning_distance if lightning_strike_count > 0 else None) would never work. Thanks [image: Screenshot (14).png] On Wednesday, August 11, 2021 at 11:41:31 AM UTC-4 bell...@gmail.com wrote: > I fear the root cause is that MQTTSubscribeDriver takes each MQTT message > and creates a loop packet. This works ok for 'aggregated' messages like > json or keyword (the original implementations). But for 'individual' > payloads that are dependent on each other, it is problematic.I've already > had to deal with this for wind data. MQTTSubscribe collects speed and > direction data into a single loop packet. > > Since then I've been experimenting with collecting data and creating the > loop packet from the collection. Looking at your loop packets, I think it > might help you. > > The code is a pre-release which can be found here, > https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v2.1.0-rc02 > > You will need to add two undocumented configuration options. > > [MQTTSubscribeDriver] > . > . > . > > [[topics] > # With the exception of wind data, by default a packet is created > for every MQTT message received. > # When this is true, MQTTSubscribe attempts to collect > observations across messages into a packet. > # Default is False. > # This is experimental and may be removed. > collect_observations = True > > # With the exception of wind data, by default a queue is created > for every MQTT topic. > # When this is true, MQTTSubsribe uses a single queue for all non > wind data. > # This is useful when 'collect_observations = True'. > # Default is False. > # This is experimental and may be removed. > single_queue = True > > . > . > . > > With this, the incoming data should be collected and a loop packet only > created when the arriving observation is already in the collection. Since, > hopefully, the loop packet now has both strike and distance data, your > [[Corrections]] should work. > > If this doesn't work. I'll have to go back to the drawing board and look > for a different solution. If it comes to this, I'd need a debug level log > for at least one archive period so that I can analyze the data. > > This is beta code, so the usual caveats apply. > rich > > ps. > If you are using rtl_433 to publish to MQTT, another option might be to > use the 'events' topic that it publishes to. This is a json formatted > message so the loop packet created for each MQTT message should have both > the strike and distance data. > > On Wednesday, 11 August 2021 at 09:52:39 UTC-4 gle...@gmail.com wrote: > >> I am using weewx-MQTTSubscribe to bring data into Weewx from sensors, so >> i don't use SDR sensor map, see config for mqttsubscribe here >> https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Configuring A >> number of unsuccessful tries to modify mqttsubscribe are commented out. >> [MQTTSubscribeDriver] >> [[topics]] >> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]] >> >> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]]] >> name = lightning_strike_count #strikes_total >> #strike_count >> contains_total = true >> # conversion_type = int >> # units = count >> # expires_after = 0 >> >> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]] >> >> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]]] >> name = lightning_distance #strike_dist #LS_1_distance >> # units = mile >> # conversion_type = int >> # expires_after = 0 >> >> >> >> [[Corrections]] >> # For each type, an arbitrary calibration expression can be given. >> # It should be in the units defined in the StdConvert section. >> # Example: >> lightning_distance = lightning_distance if >> lightning_strike_count > 0 else None >> >> [Accumulator] >> [[lightning_strike_count]] >> extractor = sum >> [[lightning_distance]] >> extractor = min >> # merger = minmax >> >> >> On Wednesday, August 11, 2021 at 7:58:38 AM UTC-4 tarob...@gmail.com >> wrote: >> >>> I think going back to your original issue something seems to be off. Can >>> you post you weewx.conf for lightning data? If you're using sdr, should >>> contain the following in each section: >>> >>> [SDR] >>> [[sensor_map]] >>> ...... >>> lightning_distance = distance.XXXX.AcuriteLightningPacket >>> strikes_total = strikes_total.XXXX.AcuriteLightningPacket >>> [[deltas]] >>> rain = rain_total >>> lightning_strike_count = strikes_total >>> >>> [StdCalibrate] >>> >>> [[Corrections]] >>> lightning_distance = lightning_distance if >>> lightning_strike_count > 0 else None >>> >>> [Accumulator] >>> [[lightning_strike_count]] >>> extractor = sum >>> [[lightning_distance]] >>> extractor = min >>> >>> On Tuesday, August 10, 2021 at 2:46:44 PM UTC-4 gle...@gmail.com wrote: >>> >>>> Initially I thought Rich's solution was going to work, adding his >>>> suggested correction <lightning_distance = lightning_distance if >>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0 else >>>> None> eliminated my fixed lightning_distance data in the absence of >>>> lightning_counts, but it created a new problem, now I don't get >>>> lightning_distance values with a lightning strike. I triggered the >>>> lightning detector with a spark by transiently shorting a battery charger >>>> at time 14:22:16 , which produced 2 detected lightnings and 14:22:24 which >>>> produced one, but the lightning_distance remains at None. My mqtt >>>> explorer >>>> reports the lightning detector is sending a constant 12-mile lightning >>>> distant >>>> >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.81, >>>> lightning_strike_count: 2.0, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>> >>>> <-------------------------------------------------------- >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>> rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.83, >>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>> rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.81, >>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.83, >>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.67, >>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.67, >>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>> 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>> lightning_strike_count: 1.0, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>> >>>> <----------------------------------------------------------- >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>> rainRate: 0.32, usUnits: 1 >>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>> rainRate: 0.32, usUnits: 1 >>>> >>>> >>>> On Tuesday, August 10, 2021 at 9:20:17 AM UTC-4 bell...@gmail.com >>>> wrote: >>>> >>>>> I'm no python expert (probably know just enough to be dangerous), but >>>>> something like this might get you want you want. >>>>> >>>>> # ligtning_strike_count must exist and have a count > 0 for >>>>> lightning_distance to have a valid value >>>>> lightning_distance = lightning_distance if 'lightning_strike_count'in >>>>> locals() and lightning_strike_count> 0 else None >>>>> >>>>> rich >>>>> On Tuesday, 10 August 2021 at 03:03:01 UTC-4 gjr80 wrote: >>>>> >>>>>> On Tuesday, 10 August 2021 at 08:37:42 UTC+10 gle...@gmail.com wrote: >>>>>> >>>>>>> Here is my output running weewxd directly, I see three lines which >>>>>>> show lightning_distance to be None, which is expected from corrections, >>>>>>> but >>>>>>> the following three lines show lightning_distance to be 11.999... It >>>>>>> appears the correction " lightning_distance = lightning_distance if >>>>>>> lightning_strike_count > 0 else None" is working but being ignored in >>>>>>> final >>>>>>> output to graph and database. Any additional thoughts? >>>>>>> >>>>>> >>>>>> The reason you see no change to lightning_distance for some packets >>>>>> is that for those packets lightning_strike_count does not exist so >>>>>> your calibration expression fails and no change is made, in other words >>>>>> lightning_distance remains as it was 11.9999999954. Why does this >>>>>> happen, in layman's terms when lightning_strike_count is not in the >>>>>> loop packet the calibration expression in effect has an unknown variable >>>>>> ( >>>>>> lightning_strike_count) and the calibration expression raises an >>>>>> error. The StdCalibrate service which handles the calibration >>>>>> expressions catches that error and discards that calibration expression. >>>>>> Given the limitations of the StdCalibrate service I am not aware of >>>>>> any calibration expression that would do as you want, of course a WeeWX >>>>>> and >>>>>> python wizard might come along an prove me wrong! >>>>>> >>>>>> My thoughts on a solution, unless there is a simple solution in your >>>>>> WeeWX driver or elsewhere upstream of WeeWX I would be writing your own >>>>>> WeeWX service to look at the packet and make the necessary correction. >>>>>> This >>>>>> way you can use a bit of python code to do exactly what you want and you >>>>>> won't be limited by the single line expressions as used by >>>>>> StdCalibrate. All up it should take no more than 10 lines of code. >>>>>> The service could be a data_service or process_service (refer >>>>>> weewx.conf [Engine] [[Services]]) but would need to appear before >>>>>> StdArchive is called. The advantage of your own service is that >>>>>> there is no need to change any one else's code (eg WeeWX driver, >>>>>> upstream >>>>>> code) so you will not have your changes lost during an upgrade (WeeWX or >>>>>> otherwise). >>>>>> >>>>>> Gary >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ca810af0-2673-4278-b12a-93f65e15ff58n%40googlegroups.com.
