Re: [weewx-user] Real Time Gauge Data & CPU Load
Hi Joe, I saw the same behavior and filed this issue: https://github.com/gjr80/weewx-realtime_gauge-data/issues/12 There is a diff in that post you can try if you like. -- Ken > Hello, > > I am considering converting from wview to weewx. I have stood a dev > instance and have it running based on the weewx simulator. When I enable > the rtgd extension my CPU load goes to 100%. I have changed the > "min_interval" setting within the [ReailTimeGaugeData] section from 0 to 5 > to 15, all with no effect. Is this normal expected behavior? > > Thanks in advance. > > --Joe > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Air quality sensors?
Jonathan,
I'm glad to hear this fix from Glenn fixed it.
-- Ken
> That appears to have been the problem.
>
> I'm really happy how easy this has been to integrate into my local setup.
>
> Thanks
>
> On Monday, January 15, 2018 at 9:59:16 PM UTC-8, Glenn McKechnie wrote:
>>
>> Hi jonathon
>>
>> There is a typo in the install script which I installed using
>> wee_extension, ie: 'wee_extension --install master.zip'
>>
>> process_services='user.cmon.PurpleAirMonitor',
>>
>> should be...
>>
>> process_services='user.purpleair.PurpleAirMonitor',
>>
>>
>> You can change it directly in weewx.conf and restart weewx (stop weewx
>> first, then start it as it maybe a bit messed up from the hard crash -
>> at least for me it was)
>>
>> It creates the database with archive_day tables so that may satisfy
>> the day summary complaint?
>>
>> I can't test it any further as there is nothing to query here - I
>> don't have one.
>>
>> Good luck!
>>
>>
>>
>> Cheers
>> Glenn
>>
>> rorpi - read only raspberry pi & various weewx addons
>> https://github.com/glennmckechnie
>>
>>
>> On 16 January 2018 at 15:48, gjr80 >
>> wrote:
>> > Hi,
>> >
>> > I'm not familiar with weewx-purpleair and I am not sure if the stack
>> trace
>> > you posted is with a manually created database or not, but one
>> suggestion if
>> > I may. If the weewx-purpleair is supposed to create its own database
>> then I
>> > would be trying to work through that issue rather than manually creating
>> the
>> > database. In my experience when weeWX related databases are manually
>> created
>> > and you subsequently have errors you then need to work out if the errors
>> are
>> > due to the manually created database having some defect/shorfall or is
>> it
>> > due to an error in the extension/driver/code you are running. You are
>> > usually far better to let the weeWX API maintain the databases, when
>> things
>> > don't work it takes one variable out of the equation.
>> >
>> > Gary
>> >
>> >
>> > On Tuesday, 16 January 2018 13:54:24 UTC+10, jonathan koren wrote:
>> >>
>> >> Apologies for the double post. I accidentally sent the message before I
>> >> was ready.
>> >>
>> >> Here's the stack trace.
>> >> Traceback (most recent call last):
>> >> File "/usr/share/weewx/weewx/reportengine.py", line 239, in run
>> >> obj.start()
>> >> File "/usr/share/weewx/weewx/reportengine.py", line 273, in start
>> >> self.run()
>> >> File "/usr/share/weewx/weewx/imagegenerator.py", line 32, in run
>> >> self.genImages(self.gen_ts)
>> >> File "/usr/share/weewx/weewx/imagegenerator.py", line 72, in
>> genImages
>> >> archive = self.db_binder.get_manager(binding)
>> >> File "/usr/share/weewx/weewx/manager.py", line 857, in get_manager
>> >> self.manager_cache[data_binding] = open_manager(manager_dict,
>> >> initialize)
>> >> File "/usr/share/weewx/weewx/manager.py", line 1009, in open_manager
>> >> manager_dict['table_name'])
>> >> File "/usr/share/weewx/weewx/manager.py", line 104, in open
>> >> dbmanager = cls(connection, table_name)
>> >> File "/usr/share/weewx/weewx/manager.py", line 1149, in __init__
>> >> raise weedb.OperationalError("No day summary schema for table '%s'
>> in
>> >> database '%s'" % (self.table_name, connection.database_name))
>> >> OperationalError: No day summary schema for table 'archive' in database
>> >> 'purpleair.sdb'
>> >> Jan 15 19:44:52 raspberrypi wee_reports[14415]: copygenerator: copied 8
>> >> files to /var/www/html/weewx
>> >>
>> >> It looks likes it's something with the user.purpleair.schema not being
>> >> passed through correctly or sometthing.
>> >>
>> >> Any suggestions? This is v0.2 release from github.
>> >>
>> >> Thanks.
>> >>
>> >> On Monday, January 15, 2018 at 7:45:10 PM UTC-8, jonathan koren wrote:
>> >>>
>> >>> I just installed my purple air sensor, and am tr
Re: [weewx-user] Re: Air quality sensors?
Hi Daniel, Thanks for the pull request I merged it a moment ago. You can see what I did with the data here: https://weather-rockport.keneli.org/ https://weather-rockport.keneli.org/history.html -- Ken > I finally had time to take a look at this today, it looks great! I just > sent you a pull request for the one remaining cmon reference in the > process_service value. > > Now I just have to figure out what to do with the data... I'm going to have > to find some time to add it into the Saratoga templates that I use at > https://www.lapseofthought.com/weather/. > > On Wednesday, January 10, 2018 at 8:14:48 AM UTC-8, Kenneth Baker wrote: >> >> Hi Daniel, >> >> Please give this a try: https://github.com/bakerkj/weewx-purpleair. The >> installation procedure is untested at the moment. Comments, suggestions, >> new >> feature requests are all welcome. >> >> -- Ken >> >> >> P.S. Matthew, I modeled this plugin on your cmon extension - Thanks! >> >> >> >> > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Air quality sensors?
Hi Daniel, Please give this a try: https://github.com/bakerkj/weewx-purpleair. The installation procedure is untested at the moment. Comments, suggestions, new feature requests are all welcome. -- Ken P.S. Matthew, I modeled this plugin on your cmon extension - Thanks! -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Air quality sensors?
Hi Daniel, I do have a plugin ready and working. I'll push it tonight or first thing tomorrow and post it here when I do. -- Ken > Ken, > > How goes your work on a PurpleAir plugin? Mine finally showed up last week > (they had some supplier issues) and I just got it up on the outside of the > house a few minutes ago. If you have something that works I'd be glad to > give it a try and let you know how it goes for me. > > On Tuesday, December 5, 2017 at 8:09:36 PM UTC-8, Kenneth Baker wrote: >> >> >> Daniel & Thomas, >> >> I started in on a 2nd data source collector today. The code is based on >> Matthew's >> cmon plugin. I plan to do some more testing and then I'll post it here >> for >> comments. >> >> Thanks! >> >> -- Ken >> >> >> > A friend of mine pointed me at the PurpleAir back when we had a valley >> full >> > of smoke from the Napa/Sonoma fires a few months ago. This thread just >> > convinced me to buy one. :-) If no one else has tackled it by the time >> it >> > shows up and I have time to install it, I'll look into adding a data >> source >> > for it. It shouldn't be too hard to pull the JSON and convert it to a >> 2nd >> > database in weewx (and in my InfluxDB as well). >> > >> > On Monday, December 4, 2017 at 8:51:03 AM UTC-8, Thomas Carlin wrote: >> >> >> >> I have not. It should be pretty easy with the google doc link above, >> and >> >> the documentation provided by TK and the crew here: >> >> http://www.weewx.com/docs/customizing.htm#Adding_2nd_source. You >> would >> >> just need to parse and sanitize the data coming in from the json >> output, >> >> and stuff it into the database. Once it is there, it's just a matter >> of >> >> modifying the weewx UI to display as desired. >> >> >> >> I have a handful of other projects going right now, but this is >> definitely >> >> on the list. When it happens is anyone's guess. >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "weewx-user" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to >> > weewx-user+...@googlegroups.com . >> > For more options, visit https://groups.go -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Air quality sensors?
I am pulling the data from the local device. -- Ken > That's great! You'll have to keep us updated. Are you accessing it using > you LAN (192.168.whatever) or pulling the data from their servers? > > On Tuesday, December 5, 2017 at 9:09:36 PM UTC-7, Kenneth Baker wrote: >> >> >> Daniel & Thomas, >> >> I started in on a 2nd data source collector today. The code is based on >> Matthew's >> cmon plugin. I plan to do some more testing and then I'll post it here >> for >> comments. >> >> Thanks! >> >> -- Ken >> >> >> > A friend of mine pointed me at the PurpleAir back when we had a valley >> full >> > of smoke from the Napa/Sonoma fires a few months ago. This thread just >> > convinced me to buy one. :-) If no one else has tackled it by the time >> it >> > shows up and I have time to install it, I'll look into adding a data >> source >> > for it. It shouldn't be too hard to pull the JSON and convert it to a >> 2nd >> > database in weewx (and in my InfluxDB as well). >> > >> > On Monday, December 4, 2017 at 8:51:03 AM UTC-8, Thomas Carlin wrote: >> >> >> >> I have not. It should be pretty easy with the google doc link above, >> and >> >> the documentation provided by TK and the crew here: >> >> http://www.weewx.com/docs/customizing.htm#Adding_2nd_source. You >> would >> >> just need to parse and sanitize the data coming in from the json >> output, >> >> and stuff it into the database. Once it is there, it's just a matter >> of >> >> modifying the weewx UI to display as desired. >> >> >> >> I have a handful of other projects going right now, but this is >> definitely >> >> on the list. When it happens is anyone's guess. >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "weewx-user" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to >> > weewx-user+...@googlegroups.com . >> > For more options, visit https://groups.google.com/d/optout. >> > >> >> >> > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Air quality sensors?
Daniel & Thomas, I started in on a 2nd data source collector today. The code is based on Matthew's cmon plugin. I plan to do some more testing and then I'll post it here for comments. Thanks! -- Ken > A friend of mine pointed me at the PurpleAir back when we had a valley full > of smoke from the Napa/Sonoma fires a few months ago. This thread just > convinced me to buy one. :-) If no one else has tackled it by the time it > shows up and I have time to install it, I'll look into adding a data source > for it. It shouldn't be too hard to pull the JSON and convert it to a 2nd > database in weewx (and in my InfluxDB as well). > > On Monday, December 4, 2017 at 8:51:03 AM UTC-8, Thomas Carlin wrote: >> >> I have not. It should be pretty easy with the google doc link above, and >> the documentation provided by TK and the crew here: >> http://www.weewx.com/docs/customizing.htm#Adding_2nd_source. You would >> just need to parse and sanitize the data coming in from the json output, >> and stuff it into the database. Once it is there, it's just a matter of >> modifying the weewx UI to display as desired. >> >> I have a handful of other projects going right now, but this is definitely >> on the list. When it happens is anyone's guess. >> > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[weewx-user] Re: Air quality sensors?
On Sunday, November 26, 2017 at 10:58:10 PM UTC-5, Thomas Carlin wrote: > > For anyone that is interested, I did finally get a response from the folks > at purple air, and you can remotely access the data from the sensors in > JSON format by going to http:///json. This sensor system > seems to be an excellent compromise between system quality, price, and > *hopefully* longevity and reliability. > > I don't know if the document is publically readable or not, but their > documentation can be found Here: > https://docs.google.com/document/d/15ijz94dXJ-YAZLi9iZ_RaBwrZ4KtYeCy08goGBwnbCU/edit#heading=h.l6r1b1hpfhod > Hi Thomas, My Purple Air device just arrived... Have you written a weewx driver for Purple Air? -- Ken -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Re: Thread leak in OpenWeatherMap REST service
Thanks! > On Saturday, November 4, 2017 at 2:12:59 PM UTC-4, Kenneth Baker wrote: >> >> >> Hi Matthew, >> >> I noticed that the amount of memory used by weewx was increasing over >> time. Keeping an eye on other properties of the weewx process I noticed >> that the number of threads in weewx was increasing. I added a bit of code >> to attach unique thread names to all of the weewx threads and noticed that >> specifically it was the number of OpenWeatherMap threads. >> >> > wow! thank you ken! impressive sleuthing! > > somehow the owm extension missed the _thread/archive_thread change when > restx was overhauled 4 years ago. > > your changes have been applied in weewx-owm-0.7 > > m > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Thread leak in OpenWeatherMap REST service
Hi Matthew,
I'm sorry I reversed my diff. This is the correct delta.
Proposed fix:
site_dict['manager_dict'] = weewx.manager.get_manager_dict(
config_dict['DataBindings'], config_dict['Databases'],
'wx_binding')
-self._queue = Queue.Queue()
-self._thread = OpenWeatherMapThread(self._queue, **site_dict)
-self._thread.start()
+self.archive_queue = Queue.Queue()
+self.archive_thread = OpenWeatherMapThread(self.archive_queue,
**site_dict)
+self.archive_thread.start()
self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
loginf("Data will be uploaded for %s" % site_dict['station_id'])
def new_archive_record(self, event):
-self._queue.put(event.record)
+self.archive_queue.put(event.record)
Possible update:
-class OpenWeatherMap(weewx.restx.StdRESTbase):
+class OpenWeatherMap(weewx.restx.StdRESTful):
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[weewx-user] Thread leak in OpenWeatherMap REST service
Hi Matthew,
I noticed that the amount of memory used by weewx was increasing over
time. Keeping an eye on other properties of the weewx process I noticed
that the number of threads in weewx was increasing. I added a bit of code
to attach unique thread names to all of the weewx threads and noticed that
specifically it was the number of OpenWeatherMap threads.
My system was occasionally throwing two different errors:
weewx[11771]: engine: Caught WeeWxIOError: Max tries exceeded while
getting LOOP data
and
weewx[11771]: engine: Database OperationalError exception: database is
locked
This would trigger the engine to shutdown all of the REST threads via the
StdRESTful shutdown code and then reload the engine:
def shutDown(self):
"""Shut down any threads"""
if hasattr(self, 'loop_queue') and hasattr(self, 'loop_thread'):
StdRESTful.shutDown_thread(self.loop_queue, self.loop_thread)
if hasattr(self, 'archive_queue') and hasattr(self,
'archive_thread'):
StdRESTful.shutDown_thread(self.archive_queue,
self.archive_thread)
However the OpenWeatherMap code doesn't use those instance names for the
queue and the thread. I think the owm.py code needs to be updated as
follows:
site_dict['manager_dict'] = weewx.manager.get_manager_dict(
config_dict['DataBindings'], config_dict['Databases'],
'wx_binding')
-self.archive_queue = Queue.Queue()
-self.archive_thread = OpenWeatherMapThread(self.archive_queue,
**site_dict)
-self.archive_thread.start()
+self._queue = Queue.Queue()
+self._thread = OpenWeatherMapThread(self._queue, **site_dict)
+self._thread.start()
self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
loginf("Data will be uploaded for %s" % site_dict['station_id'])
def new_archive_record(self, event):
-self.archive_queue.put(event.record)
+self._queue.put(event.record)
Also, I think this could be updated as well:
-class OpenWeatherMap(weewx.restx.StdRESTful):
+class OpenWeatherMap(weewx.restx.StdRESTbase):
Thanks,
Ken
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [weewx-user] Releasing V3.7.0
Hi all,
I set post_interval in my configuration file for the WOW plugin. This worked
prior
to v3.7.0. In 3.7.0 I get the following error:
Mar 11 18:25:03 host weewx[12696]: engine: Caught unrecoverable exception in
engine:
Mar 11 18:25:03 host weewx[12696]: type object got multiple values for
keyword argument 'post_interval'
Mar 11 18:25:03 host weewx[12696]: Traceback (most recent call last):
Mar 11 18:25:03 host weewx[12696]: File
"/usr/share/weewx/weewx/engine.py", line 874, in main
Mar 11 18:25:03 host weewx[12696]: engine =
engine_class(config_dict)
Mar 11 18:25:03 host weewx[12696]: File
"/usr/share/weewx/weewx/engine.py", line 77, in __init__
Mar 11 18:25:03 host weewx[12696]: self.loadServices(config_dict)
Mar 11 18:25:03 host weewx[12696]: File
"/usr/share/weewx/weewx/engine.py", line 141, in loadServices
Mar 11 18:25:03 host weewx[12696]:
self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))
Mar 11 18:25:03 host weewx[12696]: File
"/usr/share/weewx/weewx/restx.py", line 756, in __init__
Mar 11 18:25:03 host weewx[12696]: **_ambient_dict)
Mar 11 18:25:03 host weewx[12696]: TypeError: type object got multiple
values for keyword argument 'post_interval'
Mar 11 18:25:03 host weewx[12696]: Exiting.
(The line numbers may be off given that I have some local changes elsewhere in
the
file.)
However I think the problem is in this diff:
git show 7dec60374
commit 7dec6037447fa10d68d7e8910015d733ba109b51
Author: Alf Høgemark
Date: Thu Dec 15 19:01:13 2016 +0100
Let weewx.restx.StdWOW use post_interval=900
The WOW API documentation says
"
Frequency of submitting observations
It is recommended that the interval between automatic observations from
your AWS
is at least 15 minutes.
"
Respect this by using a post_interval of 900.
Without the post_interval, we get a HTTP status 429: Too frequent
observations.
And the log gets the following entries
Dec 15 18:54:25 weather1 weewx[2400]: restx: WOW: Failed upload attempt 1:
HTTP
Error 429: Too frequent observations.
Dec 15 18:54:30 weather1 weewx[2400]: restx: WOW: Failed upload attempt 2:
HTTP
Error 429: Too frequent observations.
Dec 15 18:54:35 weather1 weewx[2400]: restx: WOW: Failed upload attempt 3:
HTTP
Error 429: Too frequent observations.
diff --git a/bin/weewx/restx.py b/bin/weewx/restx.py
index 098d72e1..881e2e1f 100644
--- a/bin/weewx/restx.py
+++ b/bin/weewx/restx.py
@@ -683,7 +683,8 @@ class StdWOW(StdRESTful):
_ambient_dict.setdefault('server_url', StdWOW.archive_url)
self.archive_queue = Queue.Queue()
self.archive_thread = WOWThread(self.archive_queue, _manager_dict,
-protocol_name="WOW",
+protocol_name="WOW",
+post_interval=900,
**_ambient_dict)
self.archive_thread.start()
self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
I think this should be altered as follows to have the desired change (to set the
default to 900, but allow the user to override it in the configuration file:
diff --git a/bin/weewx/restx.py b/bin/weewx/restx.py
index 00d6c63a..3de5e72f 100644
--- a/bin/weewx/restx.py
+++ b/bin/weewx/restx.py
@@ -749,10 +749,10 @@ class StdWOW(StdRESTful):
config_dict, 'wx_binding')
_ambient_dict.setdefault('server_url', StdWOW.archive_url)
+_ambient_dict.setdefault('post_interval', 900)
self.archive_queue = Queue.Queue()
self.archive_thread = WOWThread(self.archive_queue, _manager_dict,
protocol_name="WOW",
-post_interval=900,
**_ambient_dict)
self.archive_thread.start()
self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
-- Ken
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
