[weewx-user] Re: Fixing values in rainRate table

[Thomas Carlin]

Is there any case where rainRate will be > 0, but rain = 0 for a time 
period?

What do  sum, count, wsum, sumtime account for in the rainRate table?

I suspect that if these values are off in the archive table, is a safe to 
assume that they are off in the archive_day_rain table as well?  Should i 
rebuild the daily summaries as part of this cleanup also?

Thanks,

On Thursday, August 15, 2024 at 9:32:44 PM UTC-6 vince wrote:

> Each will be so fast running a couple commands won’t take long..
>
> On Thursday, August 15, 2024 at 7:58:36 PM UTC-7 Thomas Carlin wrote:
>
>> I also ran the queries below.  Obviously all the archive records that 
>> relate to the records in my original post need to be updated, but is it 
>> safe to assume that all of the records that are in these queries will need 
>> to be updated as well, or is there a case where rainRate > 0 but rain=0.0?
>>
>> sqlite> select count(*) from archive where rainRate > 0 and rain=0.0;
>> count(*)
>> 4464
>> sqlite> select count(*) from archive where rainRate > 0 and rain is NULL;
>> count(*)
>> 185
>> sqlite> select count(*) from archive where rain is NULL;
>> count(*)
>> 317
>>
>> These are easy, I expect, to fix, i expect.  Simply:
>> update archive set rain=0.0 where rain is NULL;
>> update archive set rainRate = 0 where rainRate > 0 AND rain=0.0;
>>
>> Is there a better way to fix all these things in one shot?
>>
>> Thanks again!
>>
>> On Thursday, August 15, 2024 at 8:39:06 PM UTC-6 Thomas Carlin wrote:
>>
>>> Hi All! I've been playing with some skin modifications on my weather 
>>> station again, and found some strange values in a few tables.  Most of them 
>>> i have been able to clean up without any issue, but I feel like there is 
>>> more to the rainRate table.  
>>>
>>> I found the following table of values, obviously the min and max are 
>>> false.  I can't however reason out how, if at all the sum, count, wsum 
>>> relate to the other values.  
>>>
>>> What is the best way to clear out these bad values?  I am guessing on 
>>> some of the min values, I already ran an update command before pulling this 
>>> full list, and because i live dangerously, didn't pull a backup first, but 
>>> i do know that the first 10 entries are 100 percent accurate.
>>>
>>> sqlite> select * from archive_day_rainRate where max >10;
>>> dateTime|min|mintime|max|maxtime|sum|count|wsum|sumtime
>>>
>>> 1605164400|0.0|1605164402|655.3501|1605188100|2136.40082144|287|640920.246432|86100
>>>
>>> 1618898400|0.0|1618898403|655.3501|1618930800|26711.4767995|288|8013443.03985|86400
>>>
>>> 1643439600|0.0|1643439601|655.3501|1643465400|2551.68191489362|244|765504.574468086|73200
>>>
>>> 1645513200|0.0|1645513203|655.3501|1645584600|31106.9906622517|287|9332097.19867551|86100
>>>
>>> 1645599600|0.0|1645599603|655.3501|1645680600|8779.61978259878|288|2633885.93477963|86400
>>> 1662703200|0.0|1662703202|655.3501|1662706200|40020.4846440398 
>>> <(484)%20644-0398>|288|12006145.3932119|86400
>>>
>>> 166815|0.0|1668150004|655.3501|1668153600|8030.40075523906|254|2409120.22657172|76200
>>> 1668754800|655.35|1668817498|655.35|1668817498|655.35|1|196605.0|300
>>> 1668841200|655.35|1668856799|655.35|1668856799|1966.05|3|589815.0|900
>>> 1668927600|655.35|1668940798|655.35|1668940798|2621.4|4|786420.0|1200
>>> 1669014000|655.35|1669023898|655.35|1669023898|3932.1|6|1179630.0|1800
>>> 1669100400|655.35|1669109698|655.35|1669109698|4587.45|7|1376235.0|2100
>>> 1669186800|655.35|1669187699|655.35|1669187699|6553.5|10|1966050.0|3000
>>> 1669273200|655.35|1669278900|655.35|1669278900|7208.85|11|2162655.0|3300
>>> 1669359600|655.35|1669365300|655.35|1669365300|7864.2|12|2359260.0|3600
>>> 1669446000|655.35|1669451697|655.35|1669451697|7208.85|11|2162655.0|3300
>>> 1669532400|655.35|1669536899|655.35|1669536899|9830.25|15|2949075.0|4500
>>> 1669618800|655.35|166962|655.35|166962|9174.9|14|2752470.0|4200
>>> 1669705200|655.35|1669719897|655.35|1669719897|9174.9|14|2752470.0|4200
>>> 1669791600|655.35|1669793098|655.35|1669793098|11140.95|17|3342285.0|5100
>>> 1669878000|655.35|1669879500|655.35|1669879500|13107.0|20|3932100.0|6000
>>> 1669964400|655.35|1669966800|655.35|1669966800|10485.6|16|3145680.0|4800
>>> 1670050800|655.35|1670052899|655.35|1670052899|7864.2|12|2359260.0|3600
>>> 

[weewx-user] Re: Fixing values in rainRate table

[Thomas Carlin]

I also ran the queries below.  Obviously all the archive records that 
relate to the records in my original post need to be updated, but is it 
safe to assume that all of the records that are in these queries will need 
to be updated as well, or is there a case where rainRate > 0 but rain=0.0?

sqlite> select count(*) from archive where rainRate > 0 and rain=0.0;
count(*)
4464
sqlite> select count(*) from archive where rainRate > 0 and rain is NULL;
count(*)
185
sqlite> select count(*) from archive where rain is NULL;
count(*)
317

These are easy, I expect, to fix, i expect.  Simply:
update archive set rain=0.0 where rain is NULL;
update archive set rainRate = 0 where rainRate > 0 AND rain=0.0;

Is there a better way to fix all these things in one shot?

Thanks again!

On Thursday, August 15, 2024 at 8:39:06 PM UTC-6 Thomas Carlin wrote:

> Hi All! I've been playing with some skin modifications on my weather 
> station again, and found some strange values in a few tables.  Most of them 
> i have been able to clean up without any issue, but I feel like there is 
> more to the rainRate table.  
>
> I found the following table of values, obviously the min and max are 
> false.  I can't however reason out how, if at all the sum, count, wsum 
> relate to the other values.  
>
> What is the best way to clear out these bad values?  I am guessing on some 
> of the min values, I already ran an update command before pulling this full 
> list, and because i live dangerously, didn't pull a backup first, but i do 
> know that the first 10 entries are 100 percent accurate.
>
> sqlite> select * from archive_day_rainRate where max >10;
> dateTime|min|mintime|max|maxtime|sum|count|wsum|sumtime
>
> 1605164400|0.0|1605164402|655.3501|1605188100|2136.40082144|287|640920.246432|86100
>
> 1618898400|0.0|1618898403|655.3501|1618930800|26711.4767995|288|8013443.03985|86400
>
> 1643439600|0.0|1643439601|655.3501|1643465400|2551.68191489362|244|765504.574468086|73200
>
> 1645513200|0.0|1645513203|655.3501|1645584600|31106.9906622517|287|9332097.19867551|86100
>
> 1645599600|0.0|1645599603|655.3501|1645680600|8779.61978259878|288|2633885.93477963|86400
> 1662703200|0.0|1662703202|655.3501|1662706200|40020.4846440398 
> <(484)%20644-0398>|288|12006145.3932119|86400
>
> 166815|0.0|1668150004|655.3501|1668153600|8030.40075523906|254|2409120.22657172|76200
> 1668754800|655.35|1668817498|655.35|1668817498|655.35|1|196605.0|300
> 1668841200|655.35|1668856799|655.35|1668856799|1966.05|3|589815.0|900
> 1668927600|655.35|1668940798|655.35|1668940798|2621.4|4|786420.0|1200
> 1669014000|655.35|1669023898|655.35|1669023898|3932.1|6|1179630.0|1800
> 1669100400|655.35|1669109698|655.35|1669109698|4587.45|7|1376235.0|2100
> 1669186800|655.35|1669187699|655.35|1669187699|6553.5|10|1966050.0|3000
> 1669273200|655.35|1669278900|655.35|1669278900|7208.85|11|2162655.0|3300
> 1669359600|655.35|1669365300|655.35|1669365300|7864.2|12|2359260.0|3600
> 1669446000|655.35|1669451697|655.35|1669451697|7208.85|11|2162655.0|3300
> 1669532400|655.35|1669536899|655.35|1669536899|9830.25|15|2949075.0|4500
> 1669618800|655.35|166962|655.35|166962|9174.9|14|2752470.0|4200
> 1669705200|655.35|1669719897|655.35|1669719897|9174.9|14|2752470.0|4200
> 1669791600|655.35|1669793098|655.35|1669793098|11140.95|17|3342285.0|5100
> 1669878000|655.35|1669879500|655.35|1669879500|13107.0|20|3932100.0|6000
> 1669964400|655.35|1669966800|655.35|1669966800|10485.6|16|3145680.0|4800
> 1670050800|655.35|1670052899|655.35|1670052899|7864.2|12|2359260.0|3600
> 1670137200|655.35|1670194711|655.35|1670138996|7864.2|109|2359260.0|32700
>
> 1670396400|0.0|1670396403|655.3501|1670407500|37676.1049695061|287|11302831.4908518|86100
>
> 1672297200|0.0|1672297201|655.3501|1672323300|22253.0669985626|287|6675920.09956877|86100
>
> 1676098800|0.0|1676098802|655.3501|1676113800|28772.4393132542|288|8631731.79397624|86400
>
> 1676185200|0.0|1676185202|655.3501|1676209500|13341.363576159|268|4002409.07284769|80400
>
> 1676703600|0.0|1676703602|655.3501|1676716500|8775.40582191782|287|2632621.74657535|86100
>
> 1698991200|0.0|1698991202|655.3501|1699011600|19847.6247601028|288|5954287.42803082|86400
>
> 1699686000|0.0|1699686004|655.3501|1699711200|17135.218|287|5140565.4001|86100
>
> 1701586800|0.0|1701586804|655.35|1701627947|15.2406976744186|287|4572.20930232558|86100
>
> Your thoughts, expertise and software is appreciated!
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/e760b237-819f-4173-8005-ab87d77705e2n%40googlegroups.com.

[weewx-user] Fixing values in rainRate table

[Thomas Carlin]

Hi All! I've been playing with some skin modifications on my weather 
station again, and found some strange values in a few tables.  Most of them 
i have been able to clean up without any issue, but I feel like there is 
more to the rainRate table.  

I found the following table of values, obviously the min and max are 
false.  I can't however reason out how, if at all the sum, count, wsum 
relate to the other values.  

What is the best way to clear out these bad values?  I am guessing on some 
of the min values, I already ran an update command before pulling this full 
list, and because i live dangerously, didn't pull a backup first, but i do 
know that the first 10 entries are 100 percent accurate.

sqlite> select * from archive_day_rainRate where max >10;
dateTime|min|mintime|max|maxtime|sum|count|wsum|sumtime
1605164400|0.0|1605164402|655.3501|1605188100|2136.40082144|287|640920.246432|86100
1618898400|0.0|1618898403|655.3501|1618930800|26711.4767995|288|8013443.03985|86400
1643439600|0.0|1643439601|655.3501|1643465400|2551.68191489362|244|765504.574468086|73200
1645513200|0.0|1645513203|655.3501|1645584600|31106.9906622517|287|9332097.19867551|86100
1645599600|0.0|1645599603|655.3501|1645680600|8779.61978259878|288|2633885.93477963|86400
1662703200|0.0|1662703202|655.3501|1662706200|40020.4846440398|288|12006145.3932119|86400
166815|0.0|1668150004|655.3501|1668153600|8030.40075523906|254|2409120.22657172|76200
1668754800|655.35|1668817498|655.35|1668817498|655.35|1|196605.0|300
1668841200|655.35|1668856799|655.35|1668856799|1966.05|3|589815.0|900
1668927600|655.35|1668940798|655.35|1668940798|2621.4|4|786420.0|1200
1669014000|655.35|1669023898|655.35|1669023898|3932.1|6|1179630.0|1800
1669100400|655.35|1669109698|655.35|1669109698|4587.45|7|1376235.0|2100
1669186800|655.35|1669187699|655.35|1669187699|6553.5|10|1966050.0|3000
1669273200|655.35|1669278900|655.35|1669278900|7208.85|11|2162655.0|3300
1669359600|655.35|1669365300|655.35|1669365300|7864.2|12|2359260.0|3600
1669446000|655.35|1669451697|655.35|1669451697|7208.85|11|2162655.0|3300
1669532400|655.35|1669536899|655.35|1669536899|9830.25|15|2949075.0|4500
1669618800|655.35|166962|655.35|166962|9174.9|14|2752470.0|4200
1669705200|655.35|1669719897|655.35|1669719897|9174.9|14|2752470.0|4200
1669791600|655.35|1669793098|655.35|1669793098|11140.95|17|3342285.0|5100
1669878000|655.35|1669879500|655.35|1669879500|13107.0|20|3932100.0|6000
1669964400|655.35|1669966800|655.35|1669966800|10485.6|16|3145680.0|4800
1670050800|655.35|1670052899|655.35|1670052899|7864.2|12|2359260.0|3600
1670137200|655.35|1670194711|655.35|1670138996|7864.2|109|2359260.0|32700
1670396400|0.0|1670396403|655.3501|1670407500|37676.1049695061|287|11302831.4908518|86100
1672297200|0.0|1672297201|655.3501|1672323300|22253.0669985626|287|6675920.09956877|86100
1676098800|0.0|1676098802|655.3501|1676113800|28772.4393132542|288|8631731.79397624|86400
1676185200|0.0|1676185202|655.3501|1676209500|13341.363576159|268|4002409.07284769|80400
1676703600|0.0|1676703602|655.3501|1676716500|8775.40582191782|287|2632621.74657535|86100
1698991200|0.0|1698991202|655.3501|1699011600|19847.6247601028|288|5954287.42803082|86400
1699686000|0.0|1699686004|655.3501|1699711200|17135.218|287|5140565.4001|86100
1701586800|0.0|1701586804|655.35|1701627947|15.2406976744186|287|4572.20930232558|86100

Your thoughts, expertise and software is appreciated!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/66e01e40-f99d-4d5c-b8b6-61f07ec4e502n%40googlegroups.com.

[weewx-user] Re: set average readings to use different unit group than instantaneous readings

[Thomas Carlin]

Thanks Andrew.  I'll start working on updating my templates.  So there is 
no simple or easy way to modify it or assign it to a different group in the 
config as to effect each template without having to touch everything?

On Friday, February 7, 2020 at 11:12:02 PM UTC-7, Andrew Milner wrote:
>
> see here in customisation guide
> http://weewx.com/docs/customizing.htm#Formatting_examples
>
>
>
> On Saturday, 8 February 2020 08:08:05 UTC+2, Andrew Milner wrote:
>>
>> the simple way is to change the formatting in the template(s) where 
>> appropriate.  You are not changing units per se eg from m/s to kph to mph 
>> etc but merely the displayed fprmat - the underlying unit remains the same.
>>
>>
>>
>> On Saturday, 8 February 2020 06:17:06 UTC+2, Thomas Carlin wrote:
>>>
>>> Good evening again, 
>>>
>>> The title pretty well says it all, I would like to set the average wind 
>>> speeds to use a different unit groups.  My vantage Pro2 reads wind speed in 
>>> 1mph increments, so there is no need to output decimal points on those 
>>> values, but I would like to be able to display more accurate average 
>>> values.  
>>>
>>> Is there a way to do this in the weewx report engine?
>>>
>>> Thank you in advance, as always!
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/6e797254-3ca7-4288-8b24-6bd622a2440f%40googlegroups.com.

[weewx-user] set average readings to use different unit group than instantaneous readings

[Thomas Carlin]

Good evening again, 

The title pretty well says it all, I would like to set the average wind 
speeds to use a different unit groups.  My vantage Pro2 reads wind speed in 
1mph increments, so there is no need to output decimal points on those 
values, but I would like to be able to display more accurate average 
values.  

Is there a way to do this in the weewx report engine?

Thank you in advance, as always!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/a03cadf4-662c-414c-b646-2c643c349728%40googlegroups.com.

Re: [weewx-user] Vantage Pro2 Wind average and RMS integers

[Thomas Carlin]

Never mind, John, you were absolutely right, weewx.conf had mile_per_hour = 
%.0f.  corrected, restarted, and I'm good to go.

Thanks for pointing out the obvious for me!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/76e3a77a-1ad3-47b8-85c9-2dc65fc556dd%40googlegroups.com.

Re: [weewx-user] Vantage Pro2 Wind average and RMS integers

[Thomas Carlin]

Thanks all,
John: I did check and I don't have any string formats in weewx.conf, but
I'll add them and see if it makes any difference, it will be tomorrow
before I have a chance.

tk: I know I am using software record generation, as I have a home brew
interface that doesn't support hardware generation, and my graphs look
normal and continuous, unlike the one in the linked post.  The average and
RMS values are calculated on the fly based on the archive record in the
database, are they not?

For reference, my station is visible at Carlincomputing.duckdns.org/weewx
if you can glean any useful information from there.



On Sun, Jan 19, 2020, 8:46 PM John Kline  wrote:

> weewx.conf has the highest precedence and *might* be overriding your
> choice in skin.conf.  Another possibility is that you don’t have the
> correct number of brackets in your skin.conf entry.
>
> Perhaps you could try the following and report back.
>
> In weewx.conf, under the report in question (labeled XxxReport here), add
> the 3 lines below.  Or change them if they already exist.
>
> [[XxxReport]]
>
> [[[Units]]]
> StringFormats
> mile_per_hour = %.2f
>     mile_per_hour2 = %.2f
>
>
> On Jan 19, 2020, at 7:35 PM, Thomas Carlin 
> wrote:
>
> 
> Good evening,
>
> I know that the Vantage Pro2 reports wind speed in integer values, but the
> average should calculate out to a decimal value, however in my reports I
> only get the integer.  I have checked my skin config file, and both
> mile_per_hour and mile_per_hour2 are both %0.2f, (2 decimal points, right?)
> but my reports don't reflect this.  How can get a 2 decimal value in my
> reports for these averages?
>
> Thank you all for the great software you have made available to us!
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/cdd99d8a-96ee-4bd8-b3bc-191c08f08eeb%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/cdd99d8a-96ee-4bd8-b3bc-191c08f08eeb%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/m1BJlv8GOU4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/DF4C29E4-FB82-4157-AB3A-C48335CF6DB8%40johnkline.com
> <https://groups.google.com/d/msgid/weewx-user/DF4C29E4-FB82-4157-AB3A-C48335CF6DB8%40johnkline.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CALvrRRd%2B6k-i%3DQWvM3mzhrA0gymcG-44q6dqrr6yVaZMaacSKA%40mail.gmail.com.

[weewx-user] Vantage Pro2 Wind average and RMS integers

[Thomas Carlin]

Good evening,

I know that the Vantage Pro2 reports wind speed in integer values, but the 
average should calculate out to a decimal value, however in my reports I 
only get the integer.  I have checked my skin config file, and both 
mile_per_hour and mile_per_hour2 are both %0.2f, (2 decimal points, right?) 
but my reports don't reflect this.  How can get a 2 decimal value in my 
reports for these averages?

Thank you all for the great software you have made available to us!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/cdd99d8a-96ee-4bd8-b3bc-191c08f08eeb%40googlegroups.com.

Re: [weewx-user] Re: How do i change the type of aggregation record that is created for a custom data source?

[Thomas Carlin]

Just wanted to confirm for any reading this later, After fixing my sensor 
issues, this change did exactly what I needed to do, and I am now seeing 
SUM values instead of average in the archive packets.

Thanks again,

On Sunday, September 1, 2019 at 5:27:27 PM UTC-6, Thomas Carlin wrote:
>
> I think you may be correct... It looks like only half or so of the data 
> was actually pulling from the sensors.  I'll have to figure that issue out, 
> but it doesn't look like the problem was in the config changes we made.  
>
> Thanks again Tom!
>
> On Sunday, September 1, 2019 at 5:04:24 PM UTC-6, Thomas Keffer wrote:
>>
>> Hmmm.
>>
>> If you run weewxd directly from the console command line, what values do 
>> you see go by on the console?
>>
>> On Sun, Sep 1, 2019 at 3:18 PM Thomas Carlin  wrote:
>>
>>> Thanks Tom,
>>>
>>> Thanks for the fast response!  I created the section that you said in 
>>> weewx.conf, see below, and now I don't get any data in the archive table, 
>>> also included below, with previous AVG values for comparison sake.  
>>> Suggestions?
>>>
>>> [Accumulator]
>>> [[kwh]]
>>> extractor = sum
>>> [[IrrigationFlow]]
>>> extractor = sum
>>> sqlite> select datetime, kwh, IrrigationFlow from archive where datetime 
>>> >=1567374900;
>>> dateTime|kwh|IrrigationFlow
>>> 1567374900|0.009349|0.0
>>> 1567375200|0.0100596551724138|0.0
>>> 1567375800||
>>> 1567376100||
>>> sqlite>
>>>
>>>
>>> ?
>>>
>>>
>>>
>>> On Sunday, September 1, 2019 at 3:20:00 PM UTC-6, Thomas Carlin wrote:
>>>>
>>>> Good afternoon, I hope everyone is enjoying their labor day weekend!
>>>>
>>>> I have a custom data source that I wrote a driver for, and have 
>>>> recently changed it from pulling data when the archive packet is created, 
>>>> to when the loop packet is received.  This is working well, and everything 
>>>> is pulling correctly, but now I have an issue with the creation of the 
>>>> archive packet.  Previously, the device that I received the data from 
>>>> performed the aggregation, but now it is producing much more frequent 
>>>> information, and the driver/weewx service needs to handle the aggregation. 
>>>>  
>>>>
>>>> Several of my data sources are similar to 'rain' where I want to count 
>>>> the total of something used or collected, and it looks like the archive 
>>>> packet is created using the average.  Based on the documentation, I need 
>>>> to 
>>>> implement genArchiveRecords() for these to specify how the data is 
>>>> aggregated, but I can't find any example code of how this is done.  Could 
>>>> you please point me in the correct direction for modifying this, or to a 
>>>> driver with some example code that I can modify for my needs?
>>>>
>>>> If this is more fitting for the development group, please move it there.
>>>>
>>>> Many thanks,
>>>>
>>>> Thomas
>>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "weewx-user" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to weewx...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/weewx-user/c323ce23-ff42-42fe-988c-14df56e7deb6%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/weewx-user/c323ce23-ff42-42fe-988c-14df56e7deb6%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/d015522c-5f7e-4742-8b24-f307adfe7796%40googlegroups.com.

Re: [weewx-user] Re: How do i change the type of aggregation record that is created for a custom data source?

[Thomas Carlin]

I think you may be correct... It looks like only half or so of the data was 
actually pulling from the sensors.  I'll have to figure that issue out, but 
it doesn't look like the problem was in the config changes we made.  

Thanks again Tom!

On Sunday, September 1, 2019 at 5:04:24 PM UTC-6, Thomas Keffer wrote:
>
> Hmmm.
>
> If you run weewxd directly from the console command line, what values do 
> you see go by on the console?
>
> On Sun, Sep 1, 2019 at 3:18 PM Thomas Carlin  > wrote:
>
>> Thanks Tom,
>>
>> Thanks for the fast response!  I created the section that you said in 
>> weewx.conf, see below, and now I don't get any data in the archive table, 
>> also included below, with previous AVG values for comparison sake.  
>> Suggestions?
>>
>> [Accumulator]
>> [[kwh]]
>> extractor = sum
>> [[IrrigationFlow]]
>> extractor = sum
>> sqlite> select datetime, kwh, IrrigationFlow from archive where datetime 
>> >=1567374900;
>> dateTime|kwh|IrrigationFlow
>> 1567374900|0.009349|0.0
>> 1567375200|0.0100596551724138|0.0
>> 1567375800||
>> 1567376100||
>> sqlite>
>>
>>
>> ?
>>
>>
>>
>> On Sunday, September 1, 2019 at 3:20:00 PM UTC-6, Thomas Carlin wrote:
>>>
>>> Good afternoon, I hope everyone is enjoying their labor day weekend!
>>>
>>> I have a custom data source that I wrote a driver for, and have recently 
>>> changed it from pulling data when the archive packet is created, to when 
>>> the loop packet is received.  This is working well, and everything is 
>>> pulling correctly, but now I have an issue with the creation of the archive 
>>> packet.  Previously, the device that I received the data from performed the 
>>> aggregation, but now it is producing much more frequent information, and 
>>> the driver/weewx service needs to handle the aggregation.  
>>>
>>> Several of my data sources are similar to 'rain' where I want to count 
>>> the total of something used or collected, and it looks like the archive 
>>> packet is created using the average.  Based on the documentation, I need to 
>>> implement genArchiveRecords() for these to specify how the data is 
>>> aggregated, but I can't find any example code of how this is done.  Could 
>>> you please point me in the correct direction for modifying this, or to a 
>>> driver with some example code that I can modify for my needs?
>>>
>>> If this is more fitting for the development group, please move it there.
>>>
>>> Many thanks,
>>>
>>> Thomas
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/weewx-user/c323ce23-ff42-42fe-988c-14df56e7deb6%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/weewx-user/c323ce23-ff42-42fe-988c-14df56e7deb6%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/8a488185-ae91-4d71-bf02-a98b5c319ec0%40googlegroups.com.

[weewx-user] Re: How do i change the type of aggregation record that is created for a custom data source?

[Thomas Carlin]

Thanks Tom,

Thanks for the fast response!  I created the section that you said in 
weewx.conf, see below, and now I don't get any data in the archive table, 
also included below, with previous AVG values for comparison sake.  
Suggestions?

[Accumulator]
[[kwh]]
extractor = sum
[[IrrigationFlow]]
extractor = sum
sqlite> select datetime, kwh, IrrigationFlow from archive where datetime 
>=1567374900;
dateTime|kwh|IrrigationFlow
1567374900|0.009349|0.0
1567375200|0.0100596551724138|0.0
1567375800||
1567376100||
sqlite>


?



On Sunday, September 1, 2019 at 3:20:00 PM UTC-6, Thomas Carlin wrote:
>
> Good afternoon, I hope everyone is enjoying their labor day weekend!
>
> I have a custom data source that I wrote a driver for, and have recently 
> changed it from pulling data when the archive packet is created, to when 
> the loop packet is received.  This is working well, and everything is 
> pulling correctly, but now I have an issue with the creation of the archive 
> packet.  Previously, the device that I received the data from performed the 
> aggregation, but now it is producing much more frequent information, and 
> the driver/weewx service needs to handle the aggregation.  
>
> Several of my data sources are similar to 'rain' where I want to count the 
> total of something used or collected, and it looks like the archive packet 
> is created using the average.  Based on the documentation, I need to 
> implement genArchiveRecords() for these to specify how the data is 
> aggregated, but I can't find any example code of how this is done.  Could 
> you please point me in the correct direction for modifying this, or to a 
> driver with some example code that I can modify for my needs?
>
> If this is more fitting for the development group, please move it there.
>
> Many thanks,
>
> Thomas
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/c323ce23-ff42-42fe-988c-14df56e7deb6%40googlegroups.com.

[weewx-user] How do i change the type of aggregation record that is created for a custom data source?

[Thomas Carlin]

Good afternoon, I hope everyone is enjoying their labor day weekend!

I have a custom data source that I wrote a driver for, and have recently 
changed it from pulling data when the archive packet is created, to when 
the loop packet is received.  This is working well, and everything is 
pulling correctly, but now I have an issue with the creation of the archive 
packet.  Previously, the device that I received the data from performed the 
aggregation, but now it is producing much more frequent information, and 
the driver/weewx service needs to handle the aggregation.  

Several of my data sources are similar to 'rain' where I want to count the 
total of something used or collected, and it looks like the archive packet 
is created using the average.  Based on the documentation, I need to 
implement genArchiveRecords() for these to specify how the data is 
aggregated, but I can't find any example code of how this is done.  Could 
you please point me in the correct direction for modifying this, or to a 
driver with some example code that I can modify for my needs?

If this is more fitting for the development group, please move it there.

Many thanks,

Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/bf7d69e5-7c93-438d-846d-279667f0299d%40googlegroups.com.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

I am augmenting the VP2 data, and looking for an idea what the maximum time 
my code could run to without interrupting or missing data from my VP2.  
Right now, on my 'production' system I expect additional code to run about 
3/4 second, with most of it spent connecting to MQTT, but I know that I can 
optimize that by moving a couple things to the init function rather than 
being part of the loop.  Do you thing that 3/4 or even 1 second will cause 
issues?  I'm planning on setting my sensors up to report data every 10 
seconds, so every 4th cycle will be slightly longer.

On Thursday, October 4, 2018 at 8:20:27 PM UTC-6, gjr80 wrote:
>
> On Friday, 5 October 2018 11:58:44 UTC+10, Thomas Carlin wrote:
>>
>> Perfect!  thank you Gary!  I knew it had to be something simple like 
>> that.  Everything is up and running now, and I'll just need to catch the 
>> necessary exceptions, and optimize the collection as much as possible.  Is 
>> there a target time I should shoot for, or just try to make it as fast as 
>> possible?
>>
>
> Not quite sure what you mean by a target time. If you are still using the 
> Vantage as the heart of your system then the Vantage loop interval will set 
> the rate at which NEW_LOOP_PACKETS events occur, ie every 2.5 odd seconds, 
> so that will be the fastest rate at which WeeWX will receive data. If you 
> are now subscribing to an MQTT topic to get your data that you augment the 
> Vantage loop packets with then I guess how your service works with MQTT 
> will depend on how you are using MQTT; are you subscribing to a topic, 
> caching values etc. Perhaps if you show us your service as it stands now.
>
> Gary
>
>  
>
>>
>> On Thursday, October 4, 2018 at 7:52:37 PM UTC-6, gjr80 wrote:
>>>
>>> Hi Thomas,
>>>
>>> It sounds like you should be using event.packet rather than event.record. 
>>> Archive records are found in event.record, loop packets are found in 
>>> event.packet. It's a subtle but important distinction. You have 
>>> probably moved on from the code you posted above, and I probably should 
>>> have mentioned it earlier, but I would avoid method names such as def 
>>> new_archive_packet(). There is nothing wrong with using that name per 
>>> se, but since we have loop packets and archive records it does tend to 
>>> create a little confusion, something like  def new_archive_record() and def 
>>> new_loop_packet() would be better.
>>>
>>> Gary
>>>
>>> On Friday, 5 October 2018 10:30:15 UTC+10, Thomas Carlin wrote:
>>>>
>>>> Hi Gary, 
>>>>
>>>> I've started working on this service, and I have run into an issue.  
>>>>
>>>> A quick refresher so you don't have to read through the whole thread:  
>>>> I am writing an MQTT service to add data to the loop packets.  You pointed 
>>>> me in the right direction with the NEW_LOOP_PACKET, and I'm 99% of the way 
>>>> there.  My service is pulling data with the loop packets, processing and 
>>>> normalizing it, and I'm now to the point that I am ready to stuff it into 
>>>> the loop packet.  
>>>>
>>>> Since I am modifying an existing service that I wrote, I am trying to 
>>>> use the line event.record[key] = value. that I found in the 
>>>> documentation.  When I try this, I get the following trackback.  I'm sure 
>>>> that this is because event.record is for archive packets, and I'm now 
>>>> working with loop packets.
>>>>
>>>> Traceback (most recent call last):
>>>>   File "/usr/bin/weewxd", line 64, in 
>>>> weewx.engine.main(options, args)
>>>>   File "/usr/share/weewx/weewx/engine.py", line 877, in main
>>>> engine.run()
>>>>   File "/usr/share/weewx/weewx/engine.py", line 191, in run
>>>> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, 
>>>> packet=packet))
>>>>   File "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>>>> callback(event)
>>>>   File "/usr/share/weewx/user/esp8266.py", line 105, in new_loop_packet
>>>> event.record[self.subscriptions[topic]] = value
>>>> AttributeError: 'Event' object has no attribute 'record'
>>>>
>>>>
>>>>
>>>> Any thoughts on what I can do to correct this?  I have verified that 
>>>> the variable self.subscriptions[topic] provides the expected value of 
>>>> extraTemp1, and value contains 74.30.
>>>>
>>>> Also, related to this discussion, I would like to provide a simplified 
>>>> version of code, and an explanatory blurb to be added to the documentation 
>>>> to hopefully help others in a similar situation.  Is that something that 
>>>> the development team would be open to?
>>>>
>>>>
>>>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Re: the new_archive_packet() recommendation, I didn't think about it, but I 
agree.  It is posted written in the "second data source" documentation with 
the function new_archive_packet().

On Thursday, October 4, 2018 at 7:52:37 PM UTC-6, gjr80 wrote:
>
> Hi Thomas,
>
> It sounds like you should be using event.packet rather than event.record. 
> Archive records are found in event.record, loop packets are found in 
> event.packet. It's a subtle but important distinction. You have probably 
> moved on from the code you posted above, and I probably should have 
> mentioned it earlier, but I would avoid method names such as def 
> new_archive_packet(). There is nothing wrong with using that name per se, 
> but since we have loop packets and archive records it does tend to create a 
> little confusion, something like  def new_archive_record() and def 
> new_loop_packet() would be better.
>
> Gary
>
> On Friday, 5 October 2018 10:30:15 UTC+10, Thomas Carlin wrote:
>>
>> Hi Gary, 
>>
>> I've started working on this service, and I have run into an issue.  
>>
>> A quick refresher so you don't have to read through the whole thread:  I 
>> am writing an MQTT service to add data to the loop packets.  You pointed me 
>> in the right direction with the NEW_LOOP_PACKET, and I'm 99% of the way 
>> there.  My service is pulling data with the loop packets, processing and 
>> normalizing it, and I'm now to the point that I am ready to stuff it into 
>> the loop packet.  
>>
>> Since I am modifying an existing service that I wrote, I am trying to use 
>> the line event.record[key] = value. that I found in the documentation.  
>> When I try this, I get the following trackback.  I'm sure that this is 
>> because event.record is for archive packets, and I'm now working with 
>> loop packets.
>>
>> Traceback (most recent call last):
>>   File "/usr/bin/weewxd", line 64, in 
>> weewx.engine.main(options, args)
>>   File "/usr/share/weewx/weewx/engine.py", line 877, in main
>> engine.run()
>>   File "/usr/share/weewx/weewx/engine.py", line 191, in run
>> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))
>>   File "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>> callback(event)
>>   File "/usr/share/weewx/user/esp8266.py", line 105, in new_loop_packet
>> event.record[self.subscriptions[topic]] = value
>> AttributeError: 'Event' object has no attribute 'record'
>>
>>
>>
>> Any thoughts on what I can do to correct this?  I have verified that the 
>> variable self.subscriptions[topic] provides the expected value of 
>> extraTemp1, and value contains 74.30.
>>
>> Also, related to this discussion, I would like to provide a simplified 
>> version of code, and an explanatory blurb to be added to the documentation 
>> to hopefully help others in a similar situation.  Is that something that 
>> the development team would be open to?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Perfect!  thank you Gary!  I knew it had to be something simple like that.  
Everything is up and running now, and I'll just need to catch the necessary 
exceptions, and optimize the collection as much as possible.  Is there a 
target time I should shoot for, or just try to make it as fast as possible?

On Thursday, October 4, 2018 at 7:52:37 PM UTC-6, gjr80 wrote:
>
> Hi Thomas,
>
> It sounds like you should be using event.packet rather than event.record. 
> Archive records are found in event.record, loop packets are found in 
> event.packet. It's a subtle but important distinction. You have probably 
> moved on from the code you posted above, and I probably should have 
> mentioned it earlier, but I would avoid method names such as def 
> new_archive_packet(). There is nothing wrong with using that name per se, 
> but since we have loop packets and archive records it does tend to create a 
> little confusion, something like  def new_archive_record() and def 
> new_loop_packet() would be better.
>
> Gary
>
> On Friday, 5 October 2018 10:30:15 UTC+10, Thomas Carlin wrote:
>>
>> Hi Gary, 
>>
>> I've started working on this service, and I have run into an issue.  
>>
>> A quick refresher so you don't have to read through the whole thread:  I 
>> am writing an MQTT service to add data to the loop packets.  You pointed me 
>> in the right direction with the NEW_LOOP_PACKET, and I'm 99% of the way 
>> there.  My service is pulling data with the loop packets, processing and 
>> normalizing it, and I'm now to the point that I am ready to stuff it into 
>> the loop packet.  
>>
>> Since I am modifying an existing service that I wrote, I am trying to use 
>> the line event.record[key] = value. that I found in the documentation.  
>> When I try this, I get the following trackback.  I'm sure that this is 
>> because event.record is for archive packets, and I'm now working with 
>> loop packets.
>>
>> Traceback (most recent call last):
>>   File "/usr/bin/weewxd", line 64, in 
>> weewx.engine.main(options, args)
>>   File "/usr/share/weewx/weewx/engine.py", line 877, in main
>> engine.run()
>>   File "/usr/share/weewx/weewx/engine.py", line 191, in run
>> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))
>>   File "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>> callback(event)
>>   File "/usr/share/weewx/user/esp8266.py", line 105, in new_loop_packet
>> event.record[self.subscriptions[topic]] = value
>> AttributeError: 'Event' object has no attribute 'record'
>>
>>
>>
>> Any thoughts on what I can do to correct this?  I have verified that the 
>> variable self.subscriptions[topic] provides the expected value of 
>> extraTemp1, and value contains 74.30.
>>
>> Also, related to this discussion, I would like to provide a simplified 
>> version of code, and an explanatory blurb to be added to the documentation 
>> to hopefully help others in a similar situation.  Is that something that 
>> the development team would be open to?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Hi Gary, 

I've started working on this service, and I have run into an issue.  

A quick refresher so you don't have to read through the whole thread:  I am 
writing an MQTT service to add data to the loop packets.  You pointed me in 
the right direction with the NEW_LOOP_PACKET, and I'm 99% of the way 
there.  My service is pulling data with the loop packets, processing and 
normalizing it, and I'm now to the point that I am ready to stuff it into 
the loop packet.  

Since I am modifying an existing service that I wrote, I am trying to use 
the line event.record[key] = value. that I found in the documentation.  
When I try this, I get the following trackback.  I'm sure that this is 
because event.record is for archive packets, and I'm now working with loop 
packets.

Traceback (most recent call last):
  File "/usr/bin/weewxd", line 64, in 
weewx.engine.main(options, args)
  File "/usr/share/weewx/weewx/engine.py", line 877, in main
engine.run()
  File "/usr/share/weewx/weewx/engine.py", line 191, in run
self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))
  File "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
callback(event)
  File "/usr/share/weewx/user/esp8266.py", line 105, in new_loop_packet
event.record[self.subscriptions[topic]] = value
AttributeError: 'Event' object has no attribute 'record'



Any thoughts on what I can do to correct this?  I have verified that the 
variable self.subscriptions[topic] provides the expected value of extraTemp1, 
and value contains 74.30.

Also, related to this discussion, I would like to provide a simplified 
version of code, and an explanatory blurb to be added to the documentation 
to hopefully help others in a similar situation.  Is that something that 
the development team would be open to?


-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Perfect Thank you Gary!  That gets me what I need.  Now I just need to 
reprogram all my sensors and my driver, or create another service to 
collect and queue data.  I'm leaning towards MQTT, for its simplicity, 
speed, and scalability.  This sounds more like a winter project to me!

A couple quick questions before we wrap this up:
I have seen rumblings about MQTT drivers in the forums, but some of them 
look like they are so Weewx provides MQTT data for something else.  Is 
there anything that you are aware of that could act as a second service 
with MQTT being the data source?  Has anything been blessed by the weewx 
development crew? I would hate to reinvent the wheel if someone else 
figured it out for me!

Second, just for my piece of mind, did i miss the NEW_LOOP_PACKET in the 
documentation, or was it not mentioned?  If it is the latter, I think this 
would be a valuable addition.

Thank you all for your help, excellent as always!
Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Hi Gary,
All that makes sense, thank you.  My service is coded directly from the 
documentation linked above, with the addition of a function to "somehow 
downloads the data", and I have included it below.  One thing to note, I am 
not a programmer, I dabble as a hobby, so if there is something done poorly 
or improperly, I am open to suggestions, recomendations and things that can 
be done better.  I have also included the relevant bit of code from 
weewx.config about how I am referencing my service, and the configuration 
to help explain things.  Currently my service is adding data to the archive 
record, and in order to augment the loop packets, I would need to change 
the structure to speed up the process of getting data.  Currently, with 4 
separate sensors, it takes a measurable amount of time to get the data from 
all of them, and I would want to convert to a queue structure, maybe a 
local MQTT instance.  (I have already toyed with this idea, and this would 
give me a good excuse.) Let me know if there is anything else that I can 
answer or provide that would help.  2 second resolution would be great, but 
I would be happy with 30 or 60 seconds.  Currently, my archive interval is 
set to 5 minutes.

THank you,

Custom Service: (esp8266.py)

#!/usr/bin/env python   
   

import weewx
import syslog
from weewx.engine import StdService
import schemas.wview
import requests

class add_esp_records(StdService):

def __init__(self, engine, config_dict):

# Initialize my superclass first:   

   super(add_esp_records, self).__init__(engine, config_dict)

# Bind to any new archive record events:   

self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_packet)

try:
# Dig the needed options out of the configuration dictionary.   
   
# If a critical option is missing, an exception will be raised 
and 
# the alarm will not be set.   

self.devices= config_dict['ESP8266']

#print "Devices: " 

#print self.devices 
   


syslog.syslog(syslog.LOG_INFO, "ESP8266: Setup for devices")

except KeyError as e:
syslog.syslog(syslog.LOG_INFO, "ESP8266: Not configured. 
 Missing parameter: %s" % e)


def new_archive_packet(self, event):

for device, value in self.devices.iteritems():
self.url = device
#print "URL: " 

#print self.url 
   
self.sensors = value
#print "Sensors: " 

#print self.sensors 
   

#Pass the URL and the expected values to the read function. 
   
esp_records = read_esp_json(self.url);

if esp_records != 0:


#print self.sensors['mac']  
   
#print esp_records['UID']   
   

if self.sensors['mac'] == esp_records['UID']:
#print "Sensor ID matches, continue"
   

for sensor in self.sensors:
if sensor != 'mac':
print "DB Entry: " + self.sensors[sensor]
try:
value = esp_records[sensor]

except KeyError:
 value = None

print "Value: " + str(value)
event.record[self.sensors[sensor]] = value

#Reads the da

[weewx-user] Re: Manually add min and max data

[Thomas Carlin]

Thank you for your replies. 


A couple points of clarification: The main datasource for my station is a 
Vantage Pro2, and it is using weewx as designed, LOOP packets, etc.  I have 
added a second data source as described here 
http://www.weewx.com/docs/customizing.htm#Adding_2nd_source, which specifically 
discusses adding data to the record dictionary.

Weewx is currently configured to get the hilo data from the loop packets as 
described, and this is working for the Pro2.  My question is specifically about 
the additional custom service that has been added to the data_services section 
as described in the doc linked above.


Does that clarify anything?


Based on the comments it sounds like there is no way to add the hilo data 
manually as part of the additional service, correct?

Is there a way to have my custom service run as part of a loop, rather than the 
record without reinventing the wheel, or disrupting the Pro2 and its data?


Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Manually add min and max data

[Thomas Carlin]

Good evening!

I have extended my weewx instance with several sensors build on the ESP8266 
platform with help from many in this community.  I have built a driver 
based on the custom service documentation provided by TK and crew.  This 
has been working wonderfully, and I am really enjoying the extra 
temperature sensors, as well as things like electricity monitors, 
irrigation system monitors etc.  The only issue that I have with this 
system is that I am only adding a single piece of data into the archive 
each time the archive interval is reached.  I would like to know if there 
is a way to as part of the additional service, add into the records the 
minimum and maximum values recorded by the sensors, and the dates and 
times.  Obviously, the sensors will have to record and transmit this data 
on the archive interval, and the driver will need to interpret this data.  
I just need to know how to insert the min and max data into the database.  
Is this possible with the current architecture?

For Example, the contents of the an archive for outTemp and one of my 
custom sensors.  Notice the precision of the time of the outTemp, and the 
general time of the l2_volt, all on the 5 minute archive interval of my 
system.  

sqlite> select * from archive_day_outTemp order by dateTime desc limit 1;
dateTime  |min |mintime   |max |maxtime   |sum 
|count|wsum|sumtime
1536040800|55.1|1536065400|87.5|1536097806|18741.1237726958|263  
|5622337.13180874|78900


sqlite> select * from archive_day_l2_volt order by dateTime desc limit 10;
dateTime  |min   |mintime   |max   |maxtime   |sum |count|wsum |sumtime
1536040800|115.15|1536117900|125.74|1536079200|32513.28|263  |9753984.0|78900


Hopefully this request makes sense, and thank you for your help!


http://carlincomputing.duckdns.org/weewx/index.html

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

Hey Martin,
Thank you for your interest!  I don't know if I mentioned it here, but even
cooler than that IMHO, the color scale on all the charts is dynamic, and
will adjust as the range does. The font colors also change dynamically for
better visibility.

For the record, I am not a python programmer by any means, everything I
know is self inflicted, and I am nearly positive that there is a better way
to do everything that I have done here, which is why I haven't submitted
these changes to GitHub for peer analysis, and integration.  Working with
someone else's code and modifying it to work for your needs is difficult,
and not always effective.  When I have time in the next few days, I will
try to pull the code off my machine and post it, with some basic direction,
but the support that I provide will be limited.  If anyone from FuzzyArcher
sees this, I would be happy to submit it to GitHub.  I would hesitate to
say that this is a 'finished' product, but it works for my needs.  Thoughts
and recommendations are welcome!

On Fri, Apr 27, 2018 at 7:25 AM RothMa  wrote:

> Hello Thomas,
> I want to create my own skin for Weewx. My search on how to display
> historical data on the websites led me to
> https://github.com/brewster76/fuzzy-archer and
> https://github.com/weatherstorm/Sofaskin-CW9009. I also found this thread. 
> Thanks
> for the detailed explanations.
> Now I took some code from both Github projects and was able to integrate
> the historical tables in my site using the "historygenerator.py". I'm
> able to change HTML and CSS configurations, but I'm not good at programming
> Python.
> On your wonderful page you have a monthly summary on the right side of the
> tables and an annual overview below each table. Can you please show me
> how you've extended the historygenerator.py to create the Monthly Summary
> column and the Annual Summary row?
>
> thank you very much
> Best regards
> Martin
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/FHuFLPrg_DE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Thomas Carlin
970-401-3805

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

Hi Chris,
Sorry it took so long, I started a new job a few weeks ago, and hobbies 
have taken a back seat!  Let me start by saying that all the information, 
and better descriptions of what is happening can be found in the Weewx 
Customization Guide that TK and the guys have put together.  
http://weewx.com/docs/customizing.htm

All of the back end is contained in one file, On my system 
\usr\share\weewx\user\ArchiveSearch.py  Look at Defining New Tags 
 in the docs, the 
similarities should be immediately apparent.  This then get's included in 
the search list extensions in skin.conf.  You can then include your new 
tags in your *.tmpl files, to be generated each time the generator runs. 

First the ArchiveSearch.py:  I removed most of the comments that are in the 
documentation for the sake of brevity.  It is worth noting that you may 
need to install python relativedelta on your system for this to work.

*import datetime*
*import time*
*from dateutil.relativedelta import relativedelta*

*from weewx.cheetahgenerator import SearchList*
*from weewx.tags import TimespanBinder*
*from weeutil.weeutil import TimeSpan*
*from weeutil.weeutil import archiveDaySpan*
*from weeutil.weeutil import archiveWeekSpan*
*from weeutil.weeutil import archiveMonthSpan*
*from weeutil.weeutil import archiveYearSpan*

*class ArchiveSearch(SearchList):  
   # 1*
*"""My search list extension"""*

*def __init__(self, generator):
   # 2*
*SearchList.__init__(self, generator)*

*def get_extension_list(self, timespan, db_lookup):
   # 3*
*"""Returns a search list extension with two additions.*

*Parameters:*
*  timespan: An instance of weeutil.weeutil.TimeSpan. This will*
*hold the start and stop times of the domain of *
*valid times.*

*  db_lookup: This is a function that, given a data binding*
* as its only parameter, will return a database manager*
* object.*


*"""*
*# First, create TimespanBinder object for all time. This one is 
easy*
*# because the object timespan already holds all valid times to be*
*# used in the report.*
*all_time = TimespanBinder(timespan, *
*   db_lookup,*
*   formatter=self.generator.formatter,*
*   converter=self.generator.converter,*
*  context="month")   # 4*

*# Now get a TimespanBinder object for the last seven days. This 
one we*
*# will have to calculate. First, calculate the time at midnight, 
seven*
*# days ago. The variable week_dt will be an instance of 
datetime.date.*
*today_last_year_dt = datetime.date.fromtimestamp(timespan.stop) - 
relativedelta(years=1)*

*# Convert it to unix epoch time:*
*today_last_year_ts = time.mktime(today_last_year_dt.timetuple())  
 # 6*



*today_last_year_time_span = archiveDaySpan(today_last_year_ts)*
*# Form a TimespanBinder object, using the time span we just*
*# calculated:*
*today_last_year= TimespanBinder(today_last_year_time_span,*
* db_lookup,*
*
 formatter=self.generator.formatter,*
*
 converter=self.generator.converter,*
* context="month") # 7*
*#Assembles this week last year*
*week_last_year_time_span = archiveWeekSpan(today_last_year_ts)*
*# Form a TimespanBinder object, using the time span we just*
*# calculated:*
*week_last_year= TimespanBinder(week_last_year_time_span,*
* db_lookup,*
*
 formatter=self.generator.formatter,*
*
 converter=self.generator.converter,*
* context="month") # 7*


*#Assembles this month last year.*
*month_last_year_time_span = archiveMonthSpan(today_last_year_ts)*
*# Form a TimespanBinder object, using the time span we just*
*# calculated:*
*month_last_year= TimespanBinder(month_last_year_time_span,*
* db_lookup,*
*
 formatter=self.generator.formatter,*
*
 converter=self.generator.converter,*
* context="month") # 7*

*#Assembles last year.*
*last_year_time_span = archiveYearSpan(today_last_year_ts)*
*# Form a TimespanBinder object, using the time span we just*
*# calculated:*
*last_year= TimespanBind

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

Hi Chris!

Thank you for the kind words.  I did not make this available anywhere, but 
I would be happy to post it here if you are interested.  Keep in mind that 
I am not a programer by any stretch, but I haven't had any issues with it.  
The really cool extension on the history tab I didn't write, but I 
extensively modified it to auto-define the color scheme, and color the 
fonts accurately so they don't get lost on the background.  The original 
unmodified version is available here:  
https://github.com/brewster76/fuzzy-archer/blob/master/bin/user/historygenerator.py,
 
and I am less confident in the modifications I made to it than the 
historical summary cards, but I have debated submitting it to the author 
for review anyway.  

Let me know if you would like the History Cards, and I'll post some code.


On Sunday, March 4, 2018 at 3:53:05 PM UTC-7, Chris Alemany wrote:
>
> Hi Thomas! 
>
> This is a great addition, love how you presented it on your website too.
>
> Do you have the extensions you made available to download anywhere?
>
> Thanks!
> Chris
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Air quality sensors?

[Thomas Carlin]

Yes you can access the purple air data locally, I posted a link to their
hidden document above. Something like /json. It will still send
data to their cloud, but you won't be dependent on it to get your data, and
you will be able to access it quickly.

On Jan 7, 2018 6:39 PM, "Thomas Keffer"  wrote:

> That AdaFruit sensor looks sweet at a great price. Have you thought about
> how to mount it outside?
>
> -tk
>
> On Sun, Jan 7, 2018 at 5:17 PM, Greg Troxel  wrote:
>
>> Daniel Rich  writes:
>>
>> > How goes your work on a PurpleAir plugin? Mine finally showed up last
>> week
>> > (they had some supplier issues) and I just got it up on the outside of
>> the
>> > house a few minutes ago. If you have something that works I'd be glad to
>> > give it a try and let you know how it goes for me.
>>
>> Can you set up the purpleair to give you data locally and not upload to
>> their cloud?  That was not at all clear from their website.
>>
>> Also, I recently came across a new sensor at adafruit:
>>
>>   https://www.adafruit.com/product/3686
>>
>> (I have only read the ad page, not actually played with one.)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to weewx-user+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/weewx-user/hzN9K3QH7kU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Air quality sensors?

[Thomas Carlin]

That's great!  You'll have to keep us updated.  Are you accessing it using 
you LAN (192.168.whatever) or pulling the data from their servers?

On Tuesday, December 5, 2017 at 9:09:36 PM UTC-7, Kenneth Baker wrote:
>
>
> Daniel & Thomas, 
>
> I started in on a 2nd data source collector today.  The code is based on 
> Matthew's 
> cmon plugin.  I plan to do some more testing and then I'll post it here 
> for 
> comments. 
>
> Thanks! 
>
> -- Ken 
>
>
> > A friend of mine pointed me at the PurpleAir back when we had a valley 
> full 
> > of smoke from the Napa/Sonoma fires a few months ago. This thread just 
> > convinced me to buy one. :-)  If no one else has tackled it by the time 
> it 
> > shows up and I have time to install it, I'll look into adding a data 
> source 
> > for it. It shouldn't be too hard to pull the JSON and convert it to a 
> 2nd 
> > database in weewx (and in my InfluxDB as well). 
> > 
> > On Monday, December 4, 2017 at 8:51:03 AM UTC-8, Thomas Carlin wrote: 
> >> 
> >> I have not.  It should be pretty easy with the google doc link above, 
> and 
> >> the documentation provided by TK and the crew here: 
> >> http://www.weewx.com/docs/customizing.htm#Adding_2nd_source.  You 
> would 
> >> just need to parse and sanitize the data coming in from the json 
> output, 
> >> and stuff it into the database.  Once it is there, it's just a matter 
> of 
> >> modifying the weewx UI to display as desired. 
> >> 
> >> I have a handful of other projects going right now, but this is 
> definitely 
> >> on the list.  When it happens is anyone's guess. 
> >> 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "weewx-user" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to 
> > weewx-user+...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
> > 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Air quality sensors?

[Thomas Carlin]

I have not.  It should be pretty easy with the google doc link above, and 
the documentation provided by TK and the crew here: 
http://www.weewx.com/docs/customizing.htm#Adding_2nd_source.  You would 
just need to parse and sanitize the data coming in from the json output, 
and stuff it into the database.  Once it is there, it's just a matter of 
modifying the weewx UI to display as desired.

I have a handful of other projects going right now, but this is definitely 
on the list.  When it happens is anyone's guess.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Air quality sensors?

[Thomas Carlin]

For anyone that is interested, I did finally get a response from the folks 
at purple air, and you can remotely access the data from the sensors in 
JSON format by going to http:///json.  This sensor system seems 
to be an excellent compromise between system quality, price, and 
*hopefully* longevity and reliability.  

I don't know if the document is publically readable or not, but their 
documentation can be found Here: 
https://docs.google.com/document/d/15ijz94dXJ-YAZLi9iZ_RaBwrZ4KtYeCy08goGBwnbCU/edit#heading=h.l6r1b1hpfhod

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx crashing

[Thomas Carlin]

That should do the trick.  Thanks Tom!

Enjoy Thanksgiving tomorrow all!


On Tuesday, November 21, 2017 at 5:29:12 AM UTC-7, Tom Keffer wrote:
>
> Good point. I went with
>
> val != val
>
>
> Commit  0bce71e 
> <https://github.com/weewx/weewx/commit/0bce71e16587b017b474434032919a0310a0d6bd>
>  
>
> -tk
>
> On Mon, Nov 20, 2017 at 9:47 PM, Thomas Carlin  > wrote:
>
>> True, didn't think about that.  It's dirty, but 
>>
>> str(val) == 'nan'
>>
>> would work also.  Aside from isnan(), that is the most readable solution 
>> that I am aware of.
>>
>>
>> On Monday, November 20, 2017 at 8:33:28 PM UTC-7, gjr80 wrote:
>>>
>>> Might have to be careful we don't break python 2.5 compatibility, I 
>>> think isnan() was introduced in python 2.6.
>>>
>>> Gary
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx crashing

[Thomas Carlin]

True, didn't think about that.  It's dirty, but 

str(val) == 'nan'

would work also.  Aside from isnan(), that is the most readable solution 
that I am aware of.

On Monday, November 20, 2017 at 8:33:28 PM UTC-7, gjr80 wrote:
>
> Might have to be careful we don't break python 2.5 compatibility, I think 
> isnan() was introduced in python 2.6.
>
> Gary
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Air quality sensors?

[Thomas Carlin]

I have been looking for air quality sensors also, and the one that peaked 
my interest is the PurpleAir:  https://www.purpleair.com/  Unfortunately, 
none of their documentation says anything about local access, but I have 
found documentation for pulling the data out of thingspeak.  The downside 
being that is reliant on many things, someone else's servers, your network, 
Internet, their network, etc.  Another that looked interesting is the uRAD 
project: https://www.uradmonitor.com/.  

It should be trivial to write a driver for these, TK and the crew have 
great documentation in the customizing guide.  I'll have to research the 
Ebay one, If only it was networked, and weather shielded to live outside!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx crashing

[Thomas Carlin]

I don't think this will catch float('nan'), which is how I was bit this 
time.  

val = float('nan')
>>> isinstance(val, (float, int))
True
>>>
>>> type(val)



May I recommend as an alternative, slightly more robust solution:

def addSum(self, val, weight=1):
 """Add a scalar value to my running sum and count."""
 if val is not None:
+if (not isinstance(val, (float, int))) or (math.isnan(val)):
+raise ValueError("accum: Expected float or int, got %s" % 
val)
 self.sum += val
 self.count   += 1
 self.wsum+= val * weight


On Monday, November 20, 2017 at 4:16:56 PM UTC-7, Tom Keffer wrote:
>
> Implemented at commit 40d7a2c 
> <https://github.com/weewx/weewx/commit/40d7a2c4cd921489664e3110c29f6256e26c7ae0>
> .
>
> -tk
>
> On Mon, Nov 20, 2017 at 4:45 PM, gjr80 > 
> wrote:
>
>> Well I am embarrased to say that to say I did not know about nan/NaN and 
>> python, today's lesson I guess.
>>
>> Gary
>>
>> On Tuesday, 21 November 2017 01:30:57 UTC+10, Thomas Carlin wrote:
>>>
>>> Tom,
>>> Your interpretation is exactly correct, and yes it is a coding error on 
>>> my side.  My thought was just a single sentence there about the Weewx 
>>> engine not doing any sanitation, and that being the responsibility of the 
>>> driver.  If not, no worries, just a thought.
>>>
>>> Gary,
>>> No worries on the delay.  You got me going in the right direction, and 
>>> we got it figured out! It's not like you get paid for doing this.  Until 
>>> yesterday, the extent of the sanitation that I was doing on the incoming 
>>> data was running float(data), and catching any ValueError exception.  Now, 
>>> because nan is a special 'string', when you run float('nan'), it returns 
>>> NaN, but now it is the Python 'number' NaN.  When you add a number to NaN, 
>>> your result is always NaN.  When this is stuffed into the database, 
>>> apparently it is interpreted as SQL NULL.  Next time the loop runs, Python 
>>> queries the database, and interprets SQL NULL as None, and then we get the 
>>> type mismatch.  
>>>
>>> I have added some more sanitation for this, and we are back up and 
>>> running, and it *should* conform to expected behavior.  
>>>
>>> Does that make sense?
>>>
>>> Thank you for the input on the driver.  I have a laundry list of things 
>>> that I would like to add to make it more robust, and I'll add the loop 
>>> output to that.  I may be converting to a queue as well, simply due to the 
>>> number of sensors that are being polled, and the "couple of seconds" note 
>>> in the Second Datasource section of the docs.  But that is a project for 
>>> another day!
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx crashing

[Thomas Carlin]

Tom,
Your interpretation is exactly correct, and yes it is a coding error on my
side.  My thought was just a single sentence there about the Weewx engine
not doing any sanitation, and that being the responsibility of the driver.
If not, no worries, just a thought.

Gary,
No worries on the delay.  You got me going in the right direction, and we
got it figured out! It's not like you get paid for doing this.  Until
yesterday, the extent of the sanitation that I was doing on the incoming
data was running float(data), and catching any ValueError exception.  Now,
because nan is a special 'string', when you run float('nan'), it returns
NaN, but now it is the Python 'number' NaN.  When you add a number to NaN,
your result is always NaN.  When this is stuffed into the database,
apparently it is interpreted as SQL NULL.  Next time the loop runs, Python
queries the database, and interprets SQL NULL as None, and then we get the
type mismatch.

I have added some more sanitation for this, and we are back up and running,
and it *should* conform to expected behavior.

Does that make sense?

Thank you for the input on the driver.  I have a laundry list of things
that I would like to add to make it more robust, and I'll add the loop
output to that.  I may be converting to a queue as well, simply due to the
number of sensors that are being polled, and the "couple of seconds" note
in the Second Datasource section of the docs.  But that is a project for
another day!

On Mon, Nov 20, 2017 at 12:53 AM, gjr80  wrote:

> Thomas C,
>
> Sorry I did not respond to your earlier posts after mine, the extract you
> provided does indeed show a daily summary record with sum and wsum fields
> as None (well the db fields contain nothing but they will end up as
> python None values within weeWX). At the moment I have still not been
> able to fathom the circumstances that would cause this.
>
> One thing about your driver though. If you are using the default weeWX
> schema, or a modified version of the default schema to which you have only
> added numeric (read SQL type REAL) fields, then your driver must only emit
> numeric values or the python None value. If you emit strings it will
> cause an exception to be thrown within weeWX. The default weeWX behaviour
> is if the sensor is present and emitting values then the driver should emit
> those values as numeric values in the approriate loop packet/archive record
> field, if the sensor is present but offline then the driver emits the
> python None value in the field concerned and if the sensor does not exist
> then the driver should not emit anything for that sensor ie the field is
> excluded from the loop packet/archive record.
>
> Note. Technically there is no reason that you cannot add a string to a
> loop packet or an archive record but if it is placed in a field that weeWX
> tries to 'accumulate' it will cause an exception.
>
> I still don't see how your driver can cause the exception you first posted
> (emitting a string will cause a similar but clearly different exception),
> but it may be worthwhile reworking your driver to conform with the weeWX
> expected behaviour. It might also be worthwhile instrumenting your driver
> to drop the emitted loop packet to the log so we can see exactly what is
> coming out of the driver in the leadup to the exception.
>
> Gary
>
>
> On Monday, 20 November 2017 08:33:28 UTC+10, Thomas Carlin wrote:
>>
>> Hey Tom
>>
>> At this point in the code 'nan' is not equivalent to 'Not a Number', it
>> is simply a string that is pulled out of an http page, not generated by
>> python.  Originally, it is generated on my Arduino when data cannot be
>> pulled from the temperature sensor.  Unfortunately in this case when you
>> run float("nan") in python, it returns nan, and that was the extent of my
>> sanitization.  I'll write a condition into my driver to catch that, and
>> should be good to go.
>>
>> On Sunday, November 19, 2017 at 2:14:59 PM UTC-7, Tom Keffer wrote:
>>>
>>> But, how did you create a NaN? They are not returned from any of the
>>> Python mathematical functions.
>>>
>>> -tk
>>>
>>> On Sun, Nov 19, 2017 at 2:04 PM, Thomas Carlin 
>>> wrote:
>>>
>>>> I'm pretty sure this is what is happening.  Take a look at the output
>>>> below:
>>>>
>>>> ...
>>>> DB Entry: extraTemp3
>>>> Value: nan
>>>> DB Entry: extraHumid3
>>>> Value: nan
>>>> ...
>>>> Unable to access http://192.168.2.105
>>>> REC:2017-11-19 12:38:00 MST (1511120280) ..., *extraHumid3: nan*,
>&

Re: [weewx-user] Re: Weewx crashing

[Thomas Carlin]

Hey Tom

At this point in the code 'nan' is not equivalent to 'Not a Number', it is 
simply a string that is pulled out of an http page, not generated by 
python.  Originally, it is generated on my Arduino when data cannot be 
pulled from the temperature sensor.  Unfortunately in this case when you 
run float("nan") in python, it returns nan, and that was the extent of my 
sanitization.  I'll write a condition into my driver to catch that, and 
should be good to go.

On Sunday, November 19, 2017 at 2:14:59 PM UTC-7, Tom Keffer wrote:
>
> But, how did you create a NaN? They are not returned from any of the 
> Python mathematical functions.
>
> -tk
>
> On Sun, Nov 19, 2017 at 2:04 PM, Thomas Carlin  > wrote:
>
>> I'm pretty sure this is what is happening.  Take a look at the output 
>> below:
>>
>> ...
>> DB Entry: extraTemp3
>> Value: nan
>> DB Entry: extraHumid3
>> Value: nan
>> ...
>> Unable to access http://192.168.2.105
>> REC:2017-11-19 12:38:00 MST (1511120280) ..., *extraHumid3: nan*, 
>> extraTemp1: 55.58, *extraTemp3: nan*, heatindex: 31.7143461695...
>>
>> The next attempt to add data to the database resulted in the program 
>> crashing due to a type mismatch.  
>>
>> May I request that a blurb be added to the documentation here: 
>> http://www.weewx.com/docs/customizing.htm#Adding_2nd_source about 
>> sanitizing the data before it is fed to event.record()?
>>
>>
>> On Sunday, November 19, 2017 at 10:59:29 AM UTC-7, Thomas Carlin wrote:
>>>
>>> The plot thickens.  Right before I went to bed this morning, I checked 
>>> the current daily summary record for extraTemp3, and it looked as you would 
>>> expect it to.  It had data in it for both sum and wsum, unlike the one 
>>> posted above.  Roughly an hour later, Weewx crashed with the same error, 
>>> and this morning when I went to look into it, the current record, same one 
>>> as last night, no longer had data in it for the same fields.  
>>>
>>> After looking a little closer, I noticed that the extraHumid3, which I 
>>> created, did exactly the same thing as extraTemp3.  Since both these pieces 
>>> of data come from the same DHT11 sensor, my current theory is that under 
>>> the wrong conditions, my sensor is spitting out garbage that is not being 
>>> properly sanitized, which somehow empties those fields.  Then next time the 
>>> loop runs, everything falls apart.  I'm setting up a sandbox right now so I 
>>> can play with it without screwing with my 'production' system.  Just hoping 
>>> the sandbox crashes like the current system does!
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

I'm pretty sure this is what is happening.  Take a look at the output below:

...
DB Entry: extraTemp3
Value: nan
DB Entry: extraHumid3
Value: nan
...
Unable to access http://192.168.2.105
REC:2017-11-19 12:38:00 MST (1511120280) ..., *extraHumid3: nan*, 
extraTemp1: 55.58, *extraTemp3: nan*, heatindex: 31.7143461695...

The next attempt to add data to the database resulted in the program 
crashing due to a type mismatch.  

May I request that a blurb be added to the documentation here: 
http://www.weewx.com/docs/customizing.htm#Adding_2nd_source about 
sanitizing the data before it is fed to event.record()?

On Sunday, November 19, 2017 at 10:59:29 AM UTC-7, Thomas Carlin wrote:
>
> The plot thickens.  Right before I went to bed this morning, I checked the 
> current daily summary record for extraTemp3, and it looked as you would 
> expect it to.  It had data in it for both sum and wsum, unlike the one 
> posted above.  Roughly an hour later, Weewx crashed with the same error, 
> and this morning when I went to look into it, the current record, same one 
> as last night, no longer had data in it for the same fields.  
>
> After looking a little closer, I noticed that the extraHumid3, which I 
> created, did exactly the same thing as extraTemp3.  Since both these pieces 
> of data come from the same DHT11 sensor, my current theory is that under 
> the wrong conditions, my sensor is spitting out garbage that is not being 
> properly sanitized, which somehow empties those fields.  Then next time the 
> loop runs, everything falls apart.  I'm setting up a sandbox right now so I 
> can play with it without screwing with my 'production' system.  Just hoping 
> the sandbox crashes like the current system does!
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

The plot thickens.  Right before I went to bed this morning, I checked the 
current daily summary record for extraTemp3, and it looked as you would 
expect it to.  It had data in it for both sum and wsum, unlike the one 
posted above.  Roughly an hour later, Weewx crashed with the same error, 
and this morning when I went to look into it, the current record, same one 
as last night, no longer had data in it for the same fields.  

After looking a little closer, I noticed that the extraHumid3, which I 
created, did exactly the same thing as extraTemp3.  Since both these pieces 
of data come from the same DHT11 sensor, my current theory is that under 
the wrong conditions, my sensor is spitting out garbage that is not being 
properly sanitized, which somehow empties those fields.  Then next time the 
loop runs, everything falls apart.  I'm setting up a sandbox right now so I 
can play with it without screwing with my 'production' system.  Just hoping 
the sandbox crashes like the current system does!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

Well, so much for that theory.   About 7:35p it crashed again.  But 
unplugging the device that generates that data has allowed it to run 
normally again.  However, based on your response the last record in this 
output would be the culprit then correct?  If my count is correct, both sum 
and wsum would be considered None when the accumulator is initialized.  Any 
idea what could cause this behavior?


sqlite> select * from archive_day_extraTemp3 where datetime >= 151064;
dateTime|min|mintime|max|maxtime|sum|count|wsum|sumtime
1510642800|70.52|1510728900|78.26|1510721100|2935.64|40|880692.0|12000
1510729200|65.48|1510746000|78.08|1510779600|16512.56|232|4953768.0|69600
1510815600|67.28|1510842900|71.06|1510842000|2934.08|43|880224.0|12900
1510902000|0.0|0|0.0|0
1510988400|74.48|1511058300|83.12|1511039400||45||13500
sqlite>




sqlite> select dateTime, inTemp, extraTemp3 from archive where datetime >= 
1511057400;
dateTime|inTemp|extraTemp3
1511057400|70.60002|75.02
1511057700|70.39998|74.66
1511058000|70.39998|74.66
1511058300|70.2767123287672|74.48
1511058600|70.2|
1511069400|69.8092307692308|



A f thought:  

In my driver if the correct data is not supplied, None is sent to 
event.record, is this correct, or should it simply not be called in that 
circumstance?
Snippit from my current driver:
for sensor in self.sensors:
if sensor != 'mac':
#print "DB Entry: " + self.sensors[sensor]
try:
value = esp_records[sensor]
except KeyError:
value = None

#print "Value: " + str(value)
event.record[self.sensors[sensor]] = value





I never modify the database without going through weewx, unless I am 
purging old data, so it shouldn't be an issue, but you never know.

Thanks Gary

On Saturday, November 18, 2017 at 8:39:03 PM UTC-7, gjr80 wrote:
>
> Yep, that could explain the symptoms. The accumulator for a given day and 
> for a given observation is at times initialised from that particular day 
> from that particular observation's daily summary table. If the sum field of 
> the day and daily summary concerned was None instead of a numeric then the 
> sum field of that accumulator would be initialised with None instead of 0.0 
> and you would see the symptoms you experienced. Rebuilding the daily 
> summaries would fix such a problem. It will be interesting to see if the 
> same issue comes back, certainly if you were making changes to the database 
> schema there is potential for a hiccup to occur, if you are relying on 
> weeWX to handle all database interraction then such a hiccup should never 
> occur.
>
> Gary
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

I did a little more playing with this today, and determined that the 
problem is with extraTemp3, which I just started using, but is built into 
the system.  That being the case, I decided to drop and re-build the daily 
summaries, just for fun.  My theory being that was done right before the 
issue started occurring to add a battery voltage field, and there could 
have been an issue there somehow.  It hasn't been running for too long yet, 
but so far it seems to be working.  I'll keep an eye on it over the next 
few days, and see how it does, and update the thread.

Thanks,

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] wview migration: serial server compatibility?

[Thomas Carlin]

On Saturday, November 18, 2017 at 1:41:41 PM UTC-7, vince wrote:
>
> I'd be very interested in writeups of these kinds of setups that enable 
> the VP2 console to be situated far away in the house from the actual 
> computer running weewx.
>

It's not really a write up but:

My setup is very simple.  I took the work that Dekay did here:
 
http://madscientistlabs.blogspot.com/2011/10/build-your-own-davis-console-datalogger.html
 
,
 
and addapted it to an ESP8266 module: 
https://www.sparkfun.com/products/13678.  Both systems run 3.3 volt logic, 
so no shifting is necessary.  Simply connect GND to GND RX to TX, and TX to 
RX, and power the module.  I added a 3.3v regulator to the board, and then 
tied off the 5v brick provided for the Vantage to avoid extra wall warts.

Flash the ESP8266 with JeeLabs esp-link, https://github.com/jeelabs/esp-link, 
and configure it for your network. Finally, set the IP and Port in 
weewx.conf, and that's all she wrote!

Obviously, you need good wifi signal, and this comes with all the downsides 
of most other homebrew connections, no local storage, so if Weewx stops 
running, that data is lost, requires software archiving, etc.  But it comes 
with the benefit of no unsightly cables.  It will also only work with the 
consoles that were released before they started baking in the DRM, (green 
dot consoles as they have been referred to)  I'm sure that both of these 
can be addressed with the rest of the work that DeKay has done, but I 
haven't taken the time to do so for my module.


The OP's solution is even easier, although more expensive, but also much 
more robust.  Take the Vantage Serial module from Davis, plug it into a 
Serial Server: https://www.ebay.com/i/292322627039?rt=nc 
, plug in a network cable, and 
get it on the network, again configuring Weewx for IP, with the correct 
address and port.  This *shouldn't* have any of the same limitations that 
my solution does.  Since you are using the 'proper' module, DRM isn't an 
issue, and the storage is there, ready to be accessed.  

A third option is to purchase Davis's Weatherlink IP: 
https://www.davisnet.com/product/weatherlinkip-for-vantage-stations/  which 
does the same as the OP's solution, without any adational hardware.  
Depending on your needs and pocket book, this may be the 'best' solution, 
as there are less peices to fail, and Davis has presumably spent many hours 
and $$ engeniering the solution.

PS. Sorry for the hijack!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] wview migration: serial server compatibility?

[Thomas Carlin]

This should work just fine.  I have my Vantage Pro2 hooked up via IP using 
a home brewed ESP8266 for a WiFi connection.  Essentially, it takes the 
Serial stream, and converts it directly to IP, and the reverse for sent 
commands.  Just make sure your IP and port are correct in your 
configuration, and you should be good to go.  

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

Hey Gary,

The config file is below.  I didn't think about it yesterday, but I am 
using the history generator script (found here 
).
 
I have modified the schema to include a few extra data sources, but if i 
understand correctly, I have not tried to add any accumulators.  What would 
that code like for the initialization you mention?  (Other than = None)?  
Could I add a debug line somewhere to get a little more information out of 
the system to determine more precisely what accumulation is incorrect?  As 
a side note, since it started running again, it has run without hiccup for 
a day, with no changes at all.

Thanks again!

Datasources:
schema_with_esp = schemas.wview.schema + [('DoorOpen1', 'REAL'), (
'DoorClosed1', 'REAL'), ('DoorOpen2', 'REAL'), ('DoorClosed2', 'REAL'), (
'IrrigationFlow', 'REAL'), ('IrrigationPressure', 'REAL'),('esp1Signal', 
'REAL'), ('esp2Signal', 'REAL'),  ('esp3Signal', 'REAL'), ('esp3LIPOVoltage'
, 'REAL'), ('esp3LIPOCurrent', 'REAL'), ('esp3SolarVoltage', 'REAL'), (
'esp3SolarCurrent', 'REAL'), ('esp3OutputVoltage', 'REAL'), (
'esp3OutputCurrent', 'REAL'), ('extraHumid3', 'REAL'), ('esp4Signal', 'REAL'
), ('esp4LIPOVoltage', 'REAL')]


I also have added these to, and created some simple groups for these, 
following the documentation, which works as expected.  


Weewx.conf (Edited for brevity)
# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2015 Tom Keffer
# See the file LICENSE.txt for your rights.

##

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 1

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /

# How long to wait before timing out a socket (FTP, HTTP) connection
socket_timeout = 20

# Do not modify this. It is used when installing and updating weewx.
version = 3.7.1

##

#   This section is for information about the additional ESP8266 Sensors 
that are in use.

[ESP8266]
[[http://1.2.3.4]]
mac = 5C:CF:7F:02:0C:17
Temp = extraTemp1
Humid = extraHumid1
doorOpen = DoorOpen1
doorClosed = DoorClosed1
Signal = esp1Signal

[[http://1.2.3.4]]
mac = 5C:CF:7F:07:5B:90
Temp = extraTemp2
Humid = extraHumid2
doorOpen = DoorOpen2
doorClosed = DoorClosed2
Signal = esp2Signal

#[[http://1.2.3.4]]
#mac = 30:AE:A4:30:93:54
#Temp = extraTemp3
#Humid = extraHumid3
# Signal = esp4Signal
# Voltage = esp4LIPOVoltage
#[[http://1.2.3.4]]
#mac = 5C:CF:7F:12:17:97
#Flow = IrrigationFlow
#Pressure = IrrigationPressure
# Signal = esp3Signal
# LIPO_Voltage = esp3LIPOVoltage
# LIPO_Current = esp3LIPOCurrent
# Solar_Voltage = esp3SolarVoltage
# Solar_Current = esp3SolarCurrent
# Output_Voltage = esp3OutputVoltage
# Output_Current = esp3OutputCurrent

##

#   This section is for information about the station.

[Station]

# Description of the station location
location = y

# Latitude and longitude in decimal degrees
latitude = 
longitude = 

# Altitude of the station, with unit it is in. This is downloaded from
# from the station if the hardware supports it.
altitude = 1234, foot

# Set to type of station hardware. There must be a corresponding stanza
# in this file with a 'driver' parameter indicating the driver to be 
used.
station_type = Vantage

# If you have a website, you may specify an URL
#station_url = http://www.example.com

# The start of the rain year (1=January; 10=October, etc.). This is
# downloaded from the station if the hardware supports it.
rain_year_start = 1

# Start of week (0=Monday, 6=Sunday)
week_start = 6

##

[Vantage]
# This section is for the Davis Vantage series of weather stations.

# Connection type: serial or ethernet 
#  serial (the classic VantagePro)
#  ethernet (the WeatherLinkIP)
type = ethernet

# If the connection type is serial, a port must be specified:
#   Debian, Ubuntu, Redhat, Fedora, and SuSE:
# /dev/ttyUSB0 is a common USB port name
# /dev/ttyS0   is a common serial port name
#   BSD:
# /dev/cuaU0   is a common serial port name
port = /dev/ttyUSB0

# If the connection type is ethernet, an IP Address/hostname is 
required:
host = 1.2.3.4

##
# The rest of this section rarely needs any attention. 
# You can safely leave it "as is."

[weewx-user] Re: Weewx crashing

[Thomas Carlin]

Sorry, Debian 9.2, Stretch, fully updated.  I misread /proc/version


On Thursday, November 16, 2017 at 8:56:28 PM UTC-7, Thomas Carlin wrote:
>
> Hey Tom et al.,
>
> I have an issue with Weewx, which has been running well until yesterday.  
> I am running it on Debian 6.3.0-18, fully up to date with a Vantage Pro 2.  
> Weewx was installed through apt-get, and is fully up to date.  I do have 
> several modifications, including adation sensors that I wrote a small 
> driver for, but they have not caused me any issues in the past, and in the 
> event that a sensor is unavailable, fail gracefully.  I have not modified 
> any 'original' scripts, but have added fields to the database, tags, the 
> mentioned driver, and am using the cmon plugin.  I would be happy to 
> provide more details if necessary.
>
> The error in syslog is below.  Please note that this is the first 
> occurrence of this (that I can find in my logs).
>
> Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
> 19:30:20 MST (1510799420) to database 'cmon.sdb'
> Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
> 19:30:20 MST (1510799420) to daily summary in 'cmon.sdb'
> Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
> 19:30:00 MST (1510799400) to database 'weewx.sdb'
> Nov 15 19:30:20 Ds-1 weewx[27964]: engine: Main loop exiting. Shutting 
> engine down.
> Nov 15 19:30:20 Ds-1 weewx[27964]: engine: Shutting down StdReport thread
> Nov 15 19:30:20 Ds-1 weewx[27964]: engine: StdReport thread has been 
> terminated
> Nov 15 19:30:21 Ds-1 weewx[27964]: engine: Caught unrecoverable exception 
> in engine:
> Nov 15 19:30:21 Ds-1 weewx[27964]:   unsupported operand type(s) 
> for +=: 'NoneType' and 'float'
> Nov 15 19:30:21 Ds-1 weewx[27964]:   Traceback (most recent call 
> last):
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 871, in main
> Nov 15 19:30:21 Ds-1 weewx[27964]:   engine.run()
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 201, in run
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 223, in dispatchEvent
> Nov 15 19:30:21 Ds-1 weewx[27964]:   callback(event)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 561, in post_loop
> Nov 15 19:30:21 Ds-1 weewx[27964]:   self._software_catchup()
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 633, in _software_catchup
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD, 
> record=record, origin='software'))
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 223, in dispatchEvent
> Nov 15 19:30:21 Ds-1 weewx[27964]:   callback(event)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/engine.py", line 588, in new_archive_record
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> dbmanager.addRecord(event.record, accumulator=self.old_accumulator)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/manager.py", line 245, in addRecord
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> self._addSingleRecord(record, cursor, log_level)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/manager.py", line 1183, in _addSingleRecord
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> _day_summary.addRecord(record, weight=_weight)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/accum.py", line 242, in addRecord
> Nov 15 19:30:21 Ds-1 weewx[27964]:   func(self, record, 
> obs_type, add_hilo, weight)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/accum.py", line 301, in add_value
> Nov 15 19:30:21 Ds-1 weewx[27964]:   
> self[obs_type].addSum(val, weight=weight)
> Nov 15 19:30:21 Ds-1 weewx[27964]: File 
> "/usr/share/weewx/weewx/accum.py", line 88, in addSum
> Nov 15 19:30:21 Ds-1 weewx[27964]:   self.sum += val
> Nov 15 19:30:21 Ds-1 weewx[27964]:   TypeError: unsupported 
> operand type(s) for +=: 'NoneType' and 'float'
> Nov 15 19:30:21 Ds-1 weewx[27964]:   E

[weewx-user] Weewx crashing

[Thomas Carlin]

Hey Tom et al.,

I have an issue with Weewx, which has been running well until yesterday.  I 
am running it on Debian 6.3.0-18, fully up to date with a Vantage Pro 2.  
Weewx was installed through apt-get, and is fully up to date.  I do have 
several modifications, including adation sensors that I wrote a small 
driver for, but they have not caused me any issues in the past, and in the 
event that a sensor is unavailable, fail gracefully.  I have not modified 
any 'original' scripts, but have added fields to the database, tags, the 
mentioned driver, and am using the cmon plugin.  I would be happy to 
provide more details if necessary.

The error in syslog is below.  Please note that this is the first 
occurrence of this (that I can find in my logs).

Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
19:30:20 MST (1510799420) to database 'cmon.sdb'
Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
19:30:20 MST (1510799420) to daily summary in 'cmon.sdb'
Nov 15 19:30:20 Ds-1 weewx[27964]: manager: Added record 2017-11-15 
19:30:00 MST (1510799400) to database 'weewx.sdb'
Nov 15 19:30:20 Ds-1 weewx[27964]: engine: Main loop exiting. Shutting 
engine down.
Nov 15 19:30:20 Ds-1 weewx[27964]: engine: Shutting down StdReport thread
Nov 15 19:30:20 Ds-1 weewx[27964]: engine: StdReport thread has been 
terminated
Nov 15 19:30:21 Ds-1 weewx[27964]: engine: Caught unrecoverable exception 
in engine:
Nov 15 19:30:21 Ds-1 weewx[27964]:   unsupported operand type(s) 
for +=: 'NoneType' and 'float'
Nov 15 19:30:21 Ds-1 weewx[27964]:   Traceback (most recent call 
last):
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 871, in main
Nov 15 19:30:21 Ds-1 weewx[27964]:   engine.run()
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 201, in run
Nov 15 19:30:21 Ds-1 weewx[27964]:   
self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 223, in dispatchEvent
Nov 15 19:30:21 Ds-1 weewx[27964]:   callback(event)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 561, in post_loop
Nov 15 19:30:21 Ds-1 weewx[27964]:   self._software_catchup()
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 633, in _software_catchup
Nov 15 19:30:21 Ds-1 weewx[27964]:   
self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD, 
record=record, origin='software'))
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 223, in dispatchEvent
Nov 15 19:30:21 Ds-1 weewx[27964]:   callback(event)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/engine.py", line 588, in new_archive_record
Nov 15 19:30:21 Ds-1 weewx[27964]:   
dbmanager.addRecord(event.record, accumulator=self.old_accumulator)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/manager.py", line 245, in addRecord
Nov 15 19:30:21 Ds-1 weewx[27964]:   
self._addSingleRecord(record, cursor, log_level)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/manager.py", line 1183, in _addSingleRecord
Nov 15 19:30:21 Ds-1 weewx[27964]:   
_day_summary.addRecord(record, weight=_weight)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/accum.py", line 242, in addRecord
Nov 15 19:30:21 Ds-1 weewx[27964]:   func(self, record, 
obs_type, add_hilo, weight)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/accum.py", line 301, in add_value
Nov 15 19:30:21 Ds-1 weewx[27964]:   self[obs_type].addSum(val, 
weight=weight)
Nov 15 19:30:21 Ds-1 weewx[27964]: File 
"/usr/share/weewx/weewx/accum.py", line 88, in addSum
Nov 15 19:30:21 Ds-1 weewx[27964]:   self.sum += val
Nov 15 19:30:21 Ds-1 weewx[27964]:   TypeError: unsupported operand 
type(s) for +=: 'NoneType' and 'float'
Nov 15 19:30:21 Ds-1 weewx[27964]:   Exiting.


I do have a watchdog on the process that tries to re-launch weewx if it 
detects that it is not running.  Most recently it crashed at 11:00 A, and 
the watchdog tried to re-launch every minute until 19:30, when it started 
successfully, and the system threw the same error every time.  

Do you need any more information from me, and/or is there something else 
that I can do to help troubleshoot this issue?

Thank you in advance, and thank you for this fantastic software that you 
have written!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, v

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Is last not always valid? I assumed it would give the last value, whether
it was 10 seconds ago, or 2 hours ago.

On Jun 30, 2017 7:50 PM, "Thomas Keffer"  wrote:

> There must not have been a valid value for outTemp for the given value of
> timespan in the first call. You can check for yourself by printing out
> timespan, then looking in your database for the day starting with the
> printed starting time.
>
> -tk
>
> On Fri, Jun 30, 2017 at 6:45 PM, Thomas Carlin 
> wrote:
>
>> I am specifically wondering why it gets N/A the first time, and I have
>> seen it for up to the first 3 results, and never get a value the first
>> time.  Why does the query not report the first time it is run?
>>
>> On Friday, June 30, 2017 at 7:36:07 PM UTC-6, Tom Keffer wrote:
>>>
>>> I'm not sure what you're asking. Why did it print 44.1°F six times? If
>>> so, this is exactly what Gary and I were discussing earlier: your extension
>>> gets loaded once *for each template*. So, that's six calls to
>>> get_extension_list() and six prints of 44.1°F.
>>>
>>> How often the extension gets used in the template is an entirely
>>> different matter.
>>>
>>> For example, if you take out your line stats.day().outTemp.last, but
>>> put a print statement in, say, TimespanBinder.__getattr__, nothing will
>>> get printed unless the tag appears in your template.
>>>
>>> Does that help?
>>>
>>> -tk
>>>
>>>
>>>
>>> On Fri, Jun 30, 2017 at 5:38 PM, Thomas Carlin 
>>> wrote:
>>>
>>>> Okay, so I distilled the code down to the bare minimum, and still get
>>>> the same results, (returning N/A the first time it runs for each report
>>>> cycle), and have included it below.  I know that this SLE doesn't actually
>>>> add anything, but since it has the same symptoms I am trying to diagnose,
>>>> it should be a bit easier.
>>>>
>>>> from weewx.cheetahgenerator import SearchList
>>>>
>>>> from weewx.tags import TimespanBinder
>>>> from weeutil.weeutil import TimeSpan
>>>> import weeutil.weeutil
>>>> import weewx.tags
>>>>
>>>>
>>>> class GarageDoorAnalyzer(SearchList):
>>>> def __init__(self, generator):
>>>> SearchList.__init__(self, generator)
>>>>
>>>> self.search_list_extension = {}
>>>>
>>>> def get_extension_list(self, timespan, db_lookup):
>>>>
>>>> garagedoor_options = {'outTemp' : 'outTemp'}
>>>>
>>>> stats = weewx.tags.TimeBinder(
>>>> db_lookup,
>>>> timespan.stop,
>>>> formatter=self.generator.formatter,
>>>> converter=self.generator.converter)
>>>>
>>>> latest = stats.day().outTemp.last
>>>>
>>>> print latest
>>>>
>>>> return [self.search_list_extension]
>>>>
>>>>
>>>> This little bit of code produces the output, It is essentially the same
>>>> run though the main engine, just intertwined with loop packet output:
>>>> sudo wee_reports /etc/weewx/weewx.conf
>>>> Using configuration file /etc/weewx/weewx.conf
>>>> Generating for all time
>>>>N/A
>>>> 44.1°F
>>>> 44.1°F
>>>> 44.1°F
>>>> 44.1°F
>>>> 44.1°F
>>>> 44.1°F
>>>>
>>>>
>>>> There is nothing in syslog or messages during this time. This 'server'
>>>> this is running on is a Raspberry Pi 1, could it be lack of resources?
>>>>
>>>> Gary, your getattr code was exactly what I needed, and worked perfectly.
>>>>
>>>> Thank you all!
>>>>
>>>> Thomas
>>>>
>>>>
>>>> On Friday, June 30, 2017 at 2:01:52 PM UTC-6, gjr80 wrote:
>>>>>
>>>>> Seeing the code and the context in which it is used is probably a good
>>>>> starting point.
>>>>>
>>>>> Gary
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "weewx-user" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to weewx-user+...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to weewx-user+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/weewx-user/P0emh6GW40I/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

I am specifically wondering why it gets N/A the first time, and I have seen 
it for up to the first 3 results, and never get a value the first time. 
 Why does the query not report the first time it is run?

On Friday, June 30, 2017 at 7:36:07 PM UTC-6, Tom Keffer wrote:
>
> I'm not sure what you're asking. Why did it print 44.1°F six times? If so, 
> this is exactly what Gary and I were discussing earlier: your extension 
> gets loaded once *for each template*. So, that's six calls to 
> get_extension_list() and six prints of 44.1°F. 
>
> How often the extension gets used in the template is an entirely different 
> matter. 
>
> For example, if you take out your line stats.day().outTemp.last, but put 
> a print statement in, say, TimespanBinder.__getattr__, nothing will get 
> printed unless the tag appears in your template.
>
> Does that help?
>
> -tk
>
>
>
> On Fri, Jun 30, 2017 at 5:38 PM, Thomas Carlin  > wrote:
>
>> Okay, so I distilled the code down to the bare minimum, and still get the 
>> same results, (returning N/A the first time it runs for each report cycle), 
>> and have included it below.  I know that this SLE doesn't actually add 
>> anything, but since it has the same symptoms I am trying to diagnose, it 
>> should be a bit easier.
>>
>> from weewx.cheetahgenerator import SearchList
>>
>> from weewx.tags import TimespanBinder
>> from weeutil.weeutil import TimeSpan
>> import weeutil.weeutil
>> import weewx.tags
>>
>>
>> class GarageDoorAnalyzer(SearchList):
>> def __init__(self, generator):
>> SearchList.__init__(self, generator)
>>
>> self.search_list_extension = {}
>>
>> def get_extension_list(self, timespan, db_lookup):
>>
>> garagedoor_options = {'outTemp' : 'outTemp'}
>>
>> stats = weewx.tags.TimeBinder(
>> db_lookup,
>> timespan.stop,
>> formatter=self.generator.formatter,
>> converter=self.generator.converter)
>>
>> latest = stats.day().outTemp.last
>>
>> print latest
>>
>> return [self.search_list_extension]
>>
>>
>> This little bit of code produces the output, It is essentially the same 
>> run though the main engine, just intertwined with loop packet output:
>> sudo wee_reports /etc/weewx/weewx.conf
>> Using configuration file /etc/weewx/weewx.conf
>> Generating for all time
>>N/A
>> 44.1°F
>> 44.1°F
>> 44.1°F
>> 44.1°F
>> 44.1°F
>> 44.1°F
>>
>>
>> There is nothing in syslog or messages during this time. This 'server' 
>> this is running on is a Raspberry Pi 1, could it be lack of resources?
>>
>> Gary, your getattr code was exactly what I needed, and worked perfectly.
>>
>> Thank you all!
>>
>> Thomas
>>
>>
>> On Friday, June 30, 2017 at 2:01:52 PM UTC-6, gjr80 wrote:
>>>
>>> Seeing the code and the context in which it is used is probably a good 
>>> starting point.
>>>
>>> Gary
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Okay, so I distilled the code down to the bare minimum, and still get the 
same results, (returning N/A the first time it runs for each report cycle), 
and have included it below.  I know that this SLE doesn't actually add 
anything, but since it has the same symptoms I am trying to diagnose, it 
should be a bit easier.

from weewx.cheetahgenerator import SearchList

from weewx.tags import TimespanBinder
from weeutil.weeutil import TimeSpan
import weeutil.weeutil
import weewx.tags


class GarageDoorAnalyzer(SearchList):
def __init__(self, generator):
SearchList.__init__(self, generator)

self.search_list_extension = {}

def get_extension_list(self, timespan, db_lookup):

garagedoor_options = {'outTemp' : 'outTemp'}

stats = weewx.tags.TimeBinder(
db_lookup,
timespan.stop,
formatter=self.generator.formatter,
converter=self.generator.converter)

latest = stats.day().outTemp.last

print latest

return [self.search_list_extension]


This little bit of code produces the output, It is essentially the same run 
though the main engine, just intertwined with loop packet output:
sudo wee_reports /etc/weewx/weewx.conf
Using configuration file /etc/weewx/weewx.conf
Generating for all time
   N/A
44.1°F
44.1°F
44.1°F
44.1°F
44.1°F
44.1°F


There is nothing in syslog or messages during this time. This 'server' this 
is running on is a Raspberry Pi 1, could it be lack of resources?

Gary, your getattr code was exactly what I needed, and worked perfectly.

Thank you all!

Thomas


On Friday, June 30, 2017 at 2:01:52 PM UTC-6, gjr80 wrote:
>
> Seeing the code and the context in which it is used is probably a good 
> starting point.
>
> Gary
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

What other information do you need?

On Fri, Jun 30, 2017 at 8:21 AM, Thomas Keffer  wrote:

> On Fri, Jun 30, 2017 at 7:15 AM, Thomas Carlin 
> wrote:
>
>> Any thoughts on the query needing to be run 3 times before I get any data?
>>
>>
> ​I would need more information before drawing any conclusions.
>
> -tk​
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/weewx-user/P0emh6GW40I/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Thomas Carlin
970-401-3805

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

I have a dictionary that contains 'outTemp' among other things for the
function, that is passed to the function that retrieves the timespan
binder.  I think that Gary's solution will work, I shouldn't ask questions
at the end of the day, the solution always ends up being obvious when I do!

Any thoughts on the query needing to be run 3 times before I get any data?

On Fri, Jun 30, 2017 at 7:37 AM, Thomas Keffer  wrote:

> I'm not understanding the "array['valid_data_source']" syntax either.
>
> Read the section in the Customizing Guide about how these tags work
> <http://weewx.com/docs/customizing.htm#defining_new_tags>. The expression
> stats.day() actually returns a TimespanBinder object. This object has a
> __getattr__ function defined on it. This function gets called when
> evaluating stats.day().outTemp with argument 'outTemp'. There is no array
> involved --- it's all attribute expressions.
>
> -tk
>
> On Thu, Jun 29, 2017 at 8:05 PM, Thomas Carlin 
> wrote:
>
>> Hey Guys,
>>
>> Finally picking this back up again, and have run into a bit of a snag.  I
>> think it is a syntax issue, but I can't figure out how to take
>>
>> latest = stats.day().outTemp.last
>>
>> and turn it into
>>
>> latest = stats.day().array['valid_data_source'].last
>>
>> so I can use it in a function, and pass the necessary parameters.
>>
>> Also, and please excuse me if this was answered, but the conversation
>> went a little over my head, but, the query needs to run 3 times before it
>> returns anything but N/A.
>>
>> Any thoughts?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to weewx-user+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/weewx-user/P0emh6GW40I/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Thomas Carlin
970-401-3805

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Hey Guys,

Finally picking this back up again, and have run into a bit of a snag.  I 
think it is a syntax issue, but I can't figure out how to take 

latest = stats.day().outTemp.last

and turn it into 

latest = stats.day().array['valid_data_source'].last

so I can use it in a function, and pass the necessary parameters.

Also, and please excuse me if this was answered, but the conversation went 
a little over my head, but, the query needs to run 3 times before it 
returns anything but N/A.  

Any thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

TK:  That did the trick, Many thanks!

Gary/TK:  As of right now, it has not been an issue, and my main server has 
plenty of CPU to handle it.  My dev environment is on a Pi1, so it feels 
the pinch a little bit.  Thank you for the additional information, that 
makes sense, and ill see about optimizing the order to reduce the number of 
calls, but I still have a catch built into the code so that it only runs 
once.

One other thing that I have run into both with the existing 'last' option, 
and the '5_min' option i wrote before we got this all squared away is that 
the first couple calls report None or N/A.  Any idea what could cause this? 
 I am running on a Pi1 like i said, so both CPU time and RAM is at a bit of 
a premium, and currently my catch only stops after a successful run, so I 
can easily work around it, but this is mainly for my own education.

On Sunday, June 25, 2017 at 4:14:28 PM UTC-6, Tom Keffer wrote:
>
> Geez, I've been on summer vacation too long. I don't know where that 
> "latest" came from. It should be 'last'. Unless, of course, you want some 
> other aggregation for the day
>
> stats.day().outTemp.last  Last temperature for the day.
> stats.day().outTemp.avgAverage temperature for the day
> stats.day().outTemp.max  Maximum temperature, etc.
>
> These aggregation types (last, avg, etc.) are not functions, so they 
> should not require parentheses. 
>
> -tk
>
>
> On Sun, Jun 25, 2017 at 2:51 PM, Thomas Carlin  > wrote:
>
>> That makes sense.  I feel like we are making progress.  At least the 
>> error keeps changing.  Now I am getting a KeyError: 'latest' on the same 
>> line.  I get the same thing when I use 'current()' or 'latest'  without (). 
>>   Traceback below:
>>
>> Traceback (most recent call last):
>>   File "/usr/share/weewx/weewx/reportengine.py", line 239, in run
>> obj.start()
>>   File "/usr/share/weewx/weewx/reportengine.py", line 273, in start
>> self.run()
>>   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 157, in run
>> ngen = self.generate(gen_dict[section_name], self.gen_ts)
>>   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 231, in 
>> generate
>> ngen += self.generate(section[subsection], gen_ts)
>>   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 231, in 
>> generate
>> ngen += self.generate(section[subsection], gen_ts)
>>   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 319, in 
>> generate
>> default_binding)
>>   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 371, in 
>> _getSearchList
>> searchList += obj.get_extension_list(timespan, db_lookup)
>>   File "/usr/share/weewx/user/gdanalyzer.py", line 66, in 
>> get_extension_list
>> latest = stats.day().outTemp.latest()
>>   File "/usr/share/weewx/weewx/tags.py", line 326, in __getattr__
>> return self._do_query(aggregate_type)
>>   File "/usr/share/weewx/weewx/tags.py", line 340, in _do_query
>> val=val, **self.option_dict)
>>   File "/usr/share/weewx/weewx/wxmanager.py", line 60, in getAggregate
>> return weewx.manager.DaySummaryManager.getAggregate(self, timespan, 
>> obs_type, aggregateType, **option_dict)
>>   File "/usr/share/weewx/weewx/manager.py", line 1270, in getAggregate
>> _row = self.getSql(DaySummaryManager.sqlDict[aggregate_type] % 
>> interDict)
>> KeyError: 'latest'
>>
>>
>>
>> On Sunday, June 25, 2017 at 2:34:42 PM UTC-6, Tom Keffer wrote:
>>>
>>> Sorry. Forgot that "day 
>>> <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/tags.py#L69>" is 
>>> actually a function. If Cheetah gets this error it swallows it, then 
>>> silently tries the attribute again, this time as a function, using default 
>>> arguments. You, programming in plain old Python, have no such luxury, so 
>>> you have to call it as a function the first time. I should note that "
>>> latest 
>>> <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/tags.py#L364>" is 
>>> also a function, so it will also have to be called as one. The expression 
>>> becomes
>>>
>>> stats.day().outTemp.latest()
>>>
>>>
>>> If you follow the links for day and latest, you can see what optional 
>>> arguments they can take. 
>>>
>>> -tk
>>>
>>>
>>>
>>>
>>>
>>> On 

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

That makes sense.  I feel like we are making progress.  At least the error 
keeps changing.  Now I am getting a KeyError: 'latest' on the same line.  I 
get the same thing when I use 'current()' or 'latest'  without ().   
Traceback below:

Traceback (most recent call last):
  File "/usr/share/weewx/weewx/reportengine.py", line 239, in run
obj.start()
  File "/usr/share/weewx/weewx/reportengine.py", line 273, in start
self.run()
  File "/usr/share/weewx/weewx/cheetahgenerator.py", line 157, in run
ngen = self.generate(gen_dict[section_name], self.gen_ts)
  File "/usr/share/weewx/weewx/cheetahgenerator.py", line 231, in generate
ngen += self.generate(section[subsection], gen_ts)
  File "/usr/share/weewx/weewx/cheetahgenerator.py", line 231, in generate
ngen += self.generate(section[subsection], gen_ts)
  File "/usr/share/weewx/weewx/cheetahgenerator.py", line 319, in generate
default_binding)
  File "/usr/share/weewx/weewx/cheetahgenerator.py", line 371, in 
_getSearchList
searchList += obj.get_extension_list(timespan, db_lookup)
  File "/usr/share/weewx/user/gdanalyzer.py", line 66, in get_extension_list
latest = stats.day().outTemp.latest()
  File "/usr/share/weewx/weewx/tags.py", line 326, in __getattr__
return self._do_query(aggregate_type)
  File "/usr/share/weewx/weewx/tags.py", line 340, in _do_query
val=val, **self.option_dict)
  File "/usr/share/weewx/weewx/wxmanager.py", line 60, in getAggregate
return weewx.manager.DaySummaryManager.getAggregate(self, timespan, 
obs_type, aggregateType, **option_dict)
  File "/usr/share/weewx/weewx/manager.py", line 1270, in getAggregate
_row = self.getSql(DaySummaryManager.sqlDict[aggregate_type] % interDict
)
KeyError: 'latest'



On Sunday, June 25, 2017 at 2:34:42 PM UTC-6, Tom Keffer wrote:
>
> Sorry. Forgot that "day 
> <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/tags.py#L69>" is 
> actually a function. If Cheetah gets this error it swallows it, then 
> silently tries the attribute again, this time as a function, using default 
> arguments. You, programming in plain old Python, have no such luxury, so 
> you have to call it as a function the first time. I should note that "
> latest <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/tags.py#L364>" 
> is also a function, so it will also have to be called as one. The 
> expression becomes
>
> stats.day().outTemp.latest()
>
>
> If you follow the links for day and latest, you can see what optional 
> arguments they can take. 
>
> -tk
>
>
>
>
>
> On Sun, Jun 25, 2017 at 1:06 PM, Thomas Carlin  > wrote:
>
>> Thanks again Tom,
>>
>> Hopefully the last question for today!
>>
>> I currently have (truncated for brevity)
>>
>> from weewx.cheetahgenerator import SearchList
>> import weewx.tags
>>
>>
>> class GarageDoorAnalyzer(SearchList):
>> def __init__(self, generator):
>> SearchList.__init__(self, generator)
>>
>>
>> def get_extension_list(self, timespan, db_lookup):
>>
>> stats = weewx.tags.TimeBinder(
>> db_lookup,
>> timespan.stop,
>> formatter=self.generator.formatter,
>> converter=self.generator.converter)
>>
>>
>> latest = stats.day.outTemp.latest
>> print latest
>>
>> self.search_list_extension[tag_name] = self.
>> _DoSomethingWithLatest(latest)
>> return[self.search_list_extension]
>>
>>
>>
>> When I run this, i get the following traceback:
>>
>> , line 67, in get_extension_list
>> latest = stats.day.outTemp.latest
>> AttributeError: 'function' object has no attribute 'outTemp'
>>
>>
>>
>>
>>
>>
>> On Sunday, June 25, 2017 at 11:37:55 AM UTC-6, Tom Keffer wrote:
>>
>>> I cut-and-paste from the definition for function 
>>> Stats.get_extension_list() 
>>> <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/cheetahgenerator.py#L537>.
>>>  
>>> You can see where trend_dict came from there. However, unless you plan 
>>> to use something like stats.trend in your search list extension 
>>> (unlikely), it, and several other parameters, can be safely left out. A 
>>> more minimal assignment for stats would look like:
>>>
>>> stats = weewx.tags.TimeBinder(
>>> db_lookup,
>>> timespan.stop,
>>> formatter=self.generator.formatter

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Thanks again Tom,

Hopefully the last question for today!

I currently have (truncated for brevity)

from weewx.cheetahgenerator import SearchList
import weewx.tags


class GarageDoorAnalyzer(SearchList):
def __init__(self, generator):
SearchList.__init__(self, generator)


def get_extension_list(self, timespan, db_lookup):

stats = weewx.tags.TimeBinder(
db_lookup,
timespan.stop,
formatter=self.generator.formatter,
converter=self.generator.converter)


latest = stats.day.outTemp.latest
print latest

self.search_list_extension[tag_name] = self._DoSomethingWithLatest(
latest)
return[self.search_list_extension]
   


When I run this, i get the following traceback:

, line 67, in get_extension_list
latest = stats.day.outTemp.latest
AttributeError: 'function' object has no attribute 'outTemp'






On Sunday, June 25, 2017 at 11:37:55 AM UTC-6, Tom Keffer wrote:
>
> I cut-and-paste from the definition for function 
> Stats.get_extension_list() 
> <https://github.com/weewx/weewx/blob/v3.7.1/bin/weewx/cheetahgenerator.py#L537>.
>  
> You can see where trend_dict came from there. However, unless you plan to 
> use something like stats.trend in your search list extension (unlikely), 
> it, and several other parameters, can be safely left out. A more minimal 
> assignment for stats would look like:
>
> stats = weewx.tags.TimeBinder(
> db_lookup,
> timespan.stop,
> formatter=self.generator.formatter,
> converter=self.generator.converter)
>
> As for how often an SLE gets called, that depends on the usage pattern. 
> Without seeing the exact pattern you are using it's hard to say, but 
> something I see quite often is Cheetah expressions that look something like 
> this (this came up today):
>
> #if $day.UV.has_data
>   
>   UV
>   #if $current.UV.raw <= 2.4
> $current.UV
>   #else if $current.UV.raw >= 2.5 and $current.UV.raw <= 5.4
> $current.UV
>   #else if $current.UV.raw >= 5.5 and $current.UV.raw <= 7.4
> $current.UV
>   #else if $current.UV.raw >= 7.5 and $current.UV.raw <= 10.4
> $current.UV
>   #else if $current.UV.raw > 10.5
> $current.UV
>   #end if
>   
> #end if
>
>
> This snippet will use the Current SLE up to eight times, just to get the 
> UV value.  If the UV value is not available in the current record, that 
> means up to 8 SQL queries! A more efficient pattern is:
>
> #if $day.UV.has_data
>   #set $current_UV = $current.UV.raw
>   
>   UV
>   #if $current_UV <= 2.4
> $current.UV
>   #else if $current_UV >= 2.5 and $current_UV <= 5.4
> $current.UV
>   #else if $current_UV >= 5.5 and $current_UV <= 7.4
> $current.UV
>   #else if $current_UV >= 7.5 and $current_UV <= 10.4
> $current.UV
>   #else
> $current.UV
>   #end if
>   
> #end if
>
> -tk
>
>
>
> On Sun, Jun 25, 2017 at 9:49 AM, Thomas Carlin  > wrote:
>
>> Thanks Tom, that looks like exactly what I need.  Doing some testing this 
>> morning, I get an error on 
>> trend=trend_dict,
>>
>> NameError: global name 'trend_dict' is not defined
>>
>> Where does this dictionary originate?
>>
>> Unrelated to the original question, I noticed that my extension runs 
>> several times (7 i think) for each report generation.  It's trivial to 
>> write a catch in so it only runs once, but i'm curious why this is.  Could 
>> you shed any light on this?
>>
>> -Thomas
>>
>>
>> On Saturday, June 24, 2017 at 8:25:07 PM UTC-6, Tom Keffer wrote:
>>>
>>> Not a simple question at all! 
>>>
>>> Generally, yes, you can use something similar to the tags in your code 
>>> by following the same path through the objects. For example,
>>>
>>> $day.outTemp.latest
>>>
>>> can be obtained by using 
>>>
>>> stats = weewx.tags.TimeBinder(
>>> db_lookup,
>>> timespan.stop,
>>> formatter=self.generator.formatter,
>>> converter=self.generator.converter,
>>> week_start=self.generator.stn_info.week_start,
>>> rain_year_start=self.generator.stn_info.rain_year_start,
>>> trend=trend_dict,
>>> skin_dict=self.generator.skin_dict)
>>>
>>> latest = stats.day.outTemp.latest
>>>
>>> This is basically all Cheetah is doing!
>>>
>>> Hope this helps.
>>>
>>

Re: [weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Thanks Tom, that looks like exactly what I need.  Doing some testing this 
morning, I get an error on 
trend=trend_dict,

NameError: global name 'trend_dict' is not defined

Where does this dictionary originate?

Unrelated to the original question, I noticed that my extension runs 
several times (7 i think) for each report generation.  It's trivial to 
write a catch in so it only runs once, but i'm curious why this is.  Could 
you shed any light on this?

-Thomas


On Saturday, June 24, 2017 at 8:25:07 PM UTC-6, Tom Keffer wrote:
>
> Not a simple question at all! 
>
> Generally, yes, you can use something similar to the tags in your code by 
> following the same path through the objects. For example,
>
> $day.outTemp.latest
>
> can be obtained by using 
>
> stats = weewx.tags.TimeBinder(
> db_lookup,
> timespan.stop,
> formatter=self.generator.formatter,
> converter=self.generator.converter,
> week_start=self.generator.stn_info.week_start,
> rain_year_start=self.generator.stn_info.rain_year_start,
> trend=trend_dict,
> skin_dict=self.generator.skin_dict)
>
> latest = stats.day.outTemp.latest
>
> This is basically all Cheetah is doing!
>
> Hope this helps.
>
> -tk
>
>
> On Sat, Jun 24, 2017 at 8:39 AM, Thomas Carlin  > wrote:
>
>> Good morning everyone,
>>
>> I have a custom search list extension that I am working on to do some 
>> analysis on my collected data, and I have it to the point that I am pulling 
>> the data out, and can manipulate it, but to do so, I had to define my own 
>> timespan/binder inside my script.  (The historygenerator extension does the 
>> same thing).
>>
>> Is there any way to use the weewx defined 'latest'  or 'current' timespan 
>> inside my python extension?  Please forgive me if this is a simple 
>> question, the only programing experience that I have is self-inflicted, and 
>> far from complete.
>>
>> Thank you!
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Custom Search list, use already defined tag

[Thomas Carlin]

Good morning everyone,

I have a custom search list extension that I am working on to do some 
analysis on my collected data, and I have it to the point that I am pulling 
the data out, and can manipulate it, but to do so, I had to define my own 
timespan/binder inside my script.  (The historygenerator extension does the 
same thing).

Is there any way to use the weewx defined 'latest'  or 'current' timespan 
inside my python extension?  Please forgive me if this is a simple 
question, the only programing experience that I have is self-inflicted, and 
far from complete.

Thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Creating custom graphs to display archive data

[Thomas Carlin]

That is pretty much what I expected.  While I am not a programer by 
training or by trade, I do dabble from time to time, and as time allows, I 
might dig around and see if I can come up with anything.  I haven't looked 
through the developer docs yet, but do you have any thoughts or insight on 
how difficult (or even possible) this might be?

I have this fantastic image in my head of a graph with 4 temperature lines 
on it, the lines getting lighter for each older period, and I'm having a 
hard time getting it out of my mind!

-tk:
   I had that thought also, and I think that I will pursue it once I am 
done with my current project.  
-m:
   If this has already made it to the list of "Things that would be cool in 
the future" add a hash mark next to it for me please!

Thanks again for all your work on this project!

My station can be seen here:  http://carlincomputing.duckdns.org/weewx/

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Creating custom graphs to display archive data

[Thomas Carlin]

I have a couple questions about image generation that doesn't seem to be 
covered in the docs anywhere.

I would like to be able to create images with the same data value, but from 
multiple time ranges. For the yearly say, have the last 365 days as one 
line, but then have another line that has the same date range, but from 1 
year ago.  Is something like this possible with the current image generator?

I would also like to be able to generate historic graphs.  Is there a way 
to do this historically, and moving forward, so next year I can look at 
this February's graph?

Thank you!

My station can be seen here:  http://carlincomputing.duckdns.org/weewx/

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

That did the trick.  Thanks again Tom

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

Hopefully my last question for this project.  If you are interested in the 
progress, the link in my first post includes the added history sections.

I have defined several new tags in examples/ArchiveSearch.py
today_last_year
week_last_year
mont_last_year
last_year
last_year_to_date

and those all work as expected.  I can use them in my templates, and they 
give me the data that I expect to see (More or less, today_... is off one 
day, the wrong direction to compensate for leap year last year, but that is 
beside the point) but when it comes to the date fields, mintime, maxtime, 
etc. they are displaying using the 'current'  format defined in 
[[TimeFormats]]  I tried adding each of my tags to the [[TimeFormats]] 
list, but had no success.  

Is there something that I missed in the docs that explains this?

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

Thanks Tom, I knew it had to be there somewhere!

On Saturday, February 4, 2017 at 1:38:49 PM UTC-7, Tom Keffer wrote:
>
> http://weewx.com/docs/customizing.htm#wee_reports
>
> -tk
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

I'm making pretty good progress on this, but it is slow going to 
troubleshoot my extension script since I have to wait for the Cheetah 
Generator to run.  Is there any way to run that process manually?

Many thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Adding archive data to daily report

[Thomas Carlin]

That was the missing link, and should get me going in the right direction! 
 Hopefully I'll have a chance to play with it this weekend  a bit.

Thanks Gary!

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Adding archive data to daily report

[Thomas Carlin]

First I would like to thank TK and everyone else who has contributed to 
this project.  It is a fantastic project, and great to see an active 
community.  

I have been playing with modifying the reports to fit my wants, and one 
thing that I would like to do is be able to display historical records on 
the reports.  On the daily, High's and lows this day last year, the 
monthly, maybe averages from this month last year, etc.  I have spent some 
time searching the Group and the docs, but have not come up with much on 
the topic.  
Could someone point me in the right direction?

For those who are interested, a little back story on my project:
I have a Vantage Pro2, that I have cobbled together a wireless serial relay 
using an ESP8266 chip (Look Ma, No wires!).  I didn't want my station to be 
tied to a computer, and those Raspberry Pi's are so clunky!
Largely, I used the information that was compiled by DeKay, using his USB 
method, I simply adapted it to run over IP instead.  The project is still 
in its infancy, but I hope to put some lipstick on this pig, and post my 
progress somewhere that others may benefit from it.  

It has only been running for a week or so at this point, but my station can 
be seen here http://carlincomputing.duckdns.org/weewx/index.html

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.