Ah, that'll do it. Explains why your database is so large and why it takes a while to generate the graph json files. That's a lot of records to process when generating the graphs.
You may want to consider going to 300 seconds for archive which is the aggregation of the loops during those 300 seconds <http://www.weewx.com/docs/customizing.htm#_________LOOP_packets_vs._archive_records_______>. Up to you though, it's your data. Changing it now won't cure the size and slowness issues though. Suppose you could run a database query to drop any dateTime from archive that isn't on a 5 minute interval (0, 5, 10, 15, etc). Don't know if Tom or Matthew have another way. On Monday, August 27, 2018 at 12:40:15 PM UTC-4, David Hathaway wrote: > > [StdArchive] > > # If the station hardware supports data logging then the archive > interval > # will be downloaded from the station. Otherwise, specify it (in > seconds). > archive_interval = 60 > > On Mon, Aug 27, 2018 at 11:36 AM Pat <p...@obrienphoto.net <javascript:>> > wrote: > >> The year graph don't total like the others, just shows you which days had >> rain. That being said your year.json is HUGE at 8mb (by contrast my >> year.json is 231kb). I wonder why yours is so big? This is why your report >> generation is so slow. Month and Year look rather large. >> >> How often are you doing archive? weewx.conf StdArchive.archive_interval >> >> On Monday, August 27, 2018 at 12:29:26 PM UTC-4, David Hathaway wrote: >>> >>> Pat, >>> >>> If you look at my site and graphs, the year rain is very odd. >>> >>> http://dave77459.mynetgear.com/weewx/belchertown/graphs/ >>> >>> On the yearly graph, rain is far less than 0.02. Switch to month, and >>> you can see it exceeds 1/2" a few days. I also miss the total rainfall >>> line on the year graph. >>> >>> Dave >>> >>> >>> On Mon, Aug 27, 2018 at 11:24 AM David Hathaway <hathaw...@gmail.com> >>> wrote: >>> >>>> The .tar.gz backup of the database is 71MB. That is to say, it is >>>> big. It is possible that it needs tuning, but running a query to create a >>>> cumulative rain table for the past year takes less than a second. When I >>>> get home, I can investigate further. Maybe there is a deadlock or >>>> something. >>>> >>>> Meanwhile, yay! It runs and looks nice. >>>> http://dave77459.mynetgear.com/weewx/belchertown/ >>>> >>>> TO-DO: >>>> >>>> - Figure out the wind records >>>> - Figure out the server load issue >>>> - NOAA data when 0.5 comes out >>>> - Personalize the headings, etc >>>> - Figure out how to get my smokeping graph to show >>>> - Figure out how to get pwrstat, which indirectly tracks server >>>> loads >>>> - Update my URL >>>> >>>> Thanks for the help! Many hours of fun ahead. :-) >>>> >>>> >>>> >>>> On Mon, Aug 27, 2018 at 11:06 AM Pat <p...@obrienphoto.net> wrote: >>>> >>>>> > Do you mean this section of weewx.conf? >>>>> >>>>> Yep >>>>> >>>>> > Will that mess up StandardReport? >>>>> >>>>> Don't believe so. It will change your URL that is displayed on >>>>> weewx.com if you have register_this_station = true >>>>> >>>>> > I am on a quad core Xeon CPU at 3.30Ghz, 16GB RAM, and an SSD main >>>>> drive. >>>>> >>>>> So basically a server. Hmm.... the highcharts do a lot of queries so >>>>> an extended generation time is expected but not sure why it's 2 minutes. >>>>> Maybe your tables needs to be optimized? >>>>> >>>>> There are ways to tune MySQL which may be something to check out too? >>>>> >>>>> >>>>> >>>>> On Monday, August 27, 2018 at 11:58:28 AM UTC-4, David Hathaway wrote: >>>>>> >>>>>> Do you mean this section of weewx.conf? >>>>>> >>>>>> # If you have a website, you may specify an URL >>>>>> station_url = http://dave77459.mynetgear.com/weewx/ >>>>>> >>>>>> Will that mess up StandardReport? Heck, I'll try it and see. Easy >>>>>> to change back. >>>>>> >>>>>> I am on a quad core Xeon CPU at 3.30Ghz, 16GB RAM, and an SSD main >>>>>> drive. Hardly anything takes time on this system. >>>>>> >>>>>> I'll give --rebuild-daily a try. I have a backup from yesterday, but >>>>>> I'll make another first. >>>>>> >>>>>> >>>>>> On Mon, Aug 27, 2018 at 10:31 AM Pat <p...@obrienphoto.net> wrote: >>>>>> >>>>>>> This way is fine. Problems and resolutions can help everyone if it's >>>>>>> in the public forum. >>>>>>> >>>>>>> I think I just resolved the error. In my highcharts Day function I'm >>>>>>> defining an end point, yet still using the timespan (which causes the >>>>>>> problem). Moving to my defined end point may have fixed it. Doing some >>>>>>> testing with it now across a few lab systems to make sure. Then I'll >>>>>>> issue >>>>>>> 0.5 and we can give that a test. >>>>>>> >>>>>>> To answer some of your questions: >>>>>>> >>>>>>> The link to styles.css file is wrong. To fix this, change your >>>>>>> weewx.conf's URL to " >>>>>>> http://dave77459.mynetgear.com/weewx/belchertown"; and re-run the >>>>>>> report. >>>>>>> >>>>>>> The 2 minute generation is for the highcharts section and it does >>>>>>> seem high. Are you on a Pi? I have 3 years of data in my database and >>>>>>> it >>>>>>> takes 15 seconds to generate on a PC. A pi would be a bit slower. >>>>>>> >>>>>>> No column error with wind? Hmm. Not too sure on this. Could be >>>>>>> database related. You can try to run sudo wee_database >>>>>>> --rebuild-daily to rebuild the daily values and see if that helps. >>>>>>> Make sure you have a *working backup of your database* just in case >>>>>>> you need to revert back to it. >>>>>>> >>>>>>> >>>>>>> On Monday, August 27, 2018 at 11:25:07 AM UTC-4, David Hathaway >>>>>>> wrote: >>>>>>>> >>>>>>>> I feel like a problem child at this point. Maybe I should email >>>>>>>> you directly rather than via this group? No worries if you want to >>>>>>>> continue this way though. >>>>>>>> >>>>>>>> Commenting out the NOAA reports got it to run...ish. >>>>>>>> >>>>>>>> >>>>>>>> - I got a "NoColumnError" in the wind section. See log below. >>>>>>>> I commented out the Wind Records section of the >>>>>>>> records\index.html.tmpl to >>>>>>>> continue on. >>>>>>>> - The skin is taking 2 minutes to be generated. Is this >>>>>>>> expected? In contrast, StandardReport takes 0.75 seconds. >>>>>>>> - The resulting page is broken as if it is missing all >>>>>>>> content. See it here: >>>>>>>> http://dave77459.mynetgear.com/weewx/belchertown/ >>>>>>>> >>>>>>>> >>>>>>>> Aug 27 09:53:16 localhost weewx[22819]: cheetahgenerator: Generated >>>>>>>> 14 files for report StandardReport in 1.18 seconds >>>>>>>> Aug 27 09:53:17 localhost weewx[22819]: imagegenerator: Generated >>>>>>>> 16 images for StandardReport in 1.54 seconds >>>>>>>> Aug 27 09:54:14 localhost weewx[22819]: manager: Added record >>>>>>>> 2018-08-27 09:54:00 CDT (1535381640) to database 'weewx' >>>>>>>> Aug 27 09:54:14 localhost weewx[22819]: manager: Added record >>>>>>>> 2018-08-27 09:54:00 CDT (1535381640) to daily summary in 'weewx' >>>>>>>> Aug 27 09:54:14 localhost weewx[22819]: engine: Launch of report >>>>>>>> thread aborted: existing report thread still running >>>>>>>> Aug 27 09:54:15 localhost weewx[22819]: restx: Wunderground-PWS: >>>>>>>> Published record 2018-08-27 09:54:00 CDT (1535381640) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: cheetahgenerator: Generate >>>>>>>> failed with exception '<class 'weedb.NoColumnError'>' >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: cheetahgenerator: **** >>>>>>>> Ignoring template /etc/weewx/skins/Belchertown/records/index.html.tmpl >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: cheetahgenerator: **** >>>>>>>> Reason: (1054, "Unknown column 'wind' in 'where clause'") >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** Traceback (most >>>>>>>> recent call last): >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 330, in generate >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** print >> _file, >>>>>>>> compiled_template >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in >>>>>>>> __str__ >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** rc = >>>>>>>> getattr(self, mainMethName)() >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "_etc_weewx_skins_Belchertown_records_index_html_tmpl.py", line 1109, >>>>>>>> in >>>>>>>> respond >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "_etc_weewx_skins_Belchertown_records_index_html_tmpl.py", line 385, >>>>>>>> in >>>>>>>> __errorCatcher46 >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File "<string>", >>>>>>>> line 1, in <module> >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/tags.py", line 329, in __getattr__ >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** return >>>>>>>> self._do_query(aggregate_type) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/tags.py", line 343, in _do_query >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** val=val, >>>>>>>> **self.option_dict) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/wxmanager.py", line 60, in getAggregate >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** return >>>>>>>> weewx.manager.DaySummaryManager.getAggregate(self, timespan, obs_type, >>>>>>>> aggregateType, **option_dict) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/manager.py", line 1249, in getAggregate >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** **option_dict) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/manager.py", line 458, in getAggregate >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** _row = >>>>>>>> self.getSql(select_stmt % interpolate_dict) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weewx/manager.py", line 394, in getSql >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** >>>>>>>> _cursor.execute(sql, sqlargs) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** File >>>>>>>> "/usr/share/weewx/weedb/mysql.py", line 48, in guarded_fn >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** raise klass(e) >>>>>>>> Aug 27 09:54:35 localhost weewx[22819]: **** NoColumnError: (1054, >>>>>>>> "Unknown column 'wind' in 'where clause'") >>>>>>>> Aug 27 09:54:54 localhost weewx[22819]: cheetahgenerator: Generated >>>>>>>> 13 files for report Belchertown in 96.59 seconds >>>>>>>> Aug 27 09:54:54 localhost weewx[22819]: copygenerator: copied 22 >>>>>>>> files to /var/www/html/weewx/belchertown >>>>>>>> >>>>>>>> >>>>>>>> On Monday, August 27, 2018 at 9:44:55 AM UTC-5, David Hathaway >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> OK, I will try that and see if that gets the rest going. >>>>>>>>> >>>>>>>>> On Mon, Aug 27, 2018 at 9:37 AM Pat <> wrote: >>>>>>>>> >>>>>>>>>> Thanks. I'm aware of this error but haven't resolved it yet. It's >>>>>>>>>> a problem with NOAA reports being generated. >>>>>>>>>> >>>>>>>>>> My thought is to move the NOAA reports to another skin, but I'm >>>>>>>>>> afraid that'll be too much extra work for everyone who installs the >>>>>>>>>> skin. >>>>>>>>>> >>>>>>>>>> If you want to stop it from happening, open skin.conf and remove >>>>>>>>>> the NOAA_month and NOAA_year sections - which will get the skin >>>>>>>>>> working but will not produce the NOAA reports. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Monday, August 27, 2018 at 10:22:23 AM UTC-4, David Hathaway >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I updated MySQL to stop throwing that error, updated to weewx v >>>>>>>>>>> 3.8.2, and installed Belchertown 0.4. Now I get this error: >>>>>>>>>>> >>>>>>>>>>> Aug 27 09:17:16 localhost weewx[4131]: imagegenerator: Generated >>>>>>>>>>> 16 images for StandardReport in 0.75 seconds >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: reportengine: Caught >>>>>>>>>>> unrecoverable exception in generator >>>>>>>>>>> weewx.cheetahgenerator.CheetahGenerator >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** start time >>>>>>>>>>> (1535346000) is greater than stop time (1420092000) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** Traceback >>>>>>>>>>> (most recent call last): >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/reportengine.py", line 239, in run >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** >>>>>>>>>>> obj.start() >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/reportengine.py", line 273, in start >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** >>>>>>>>>>> self.run() >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 158, in run >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** ngen = >>>>>>>>>>> self.generate(gen_dict[section_name], self.gen_ts) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** ngen += >>>>>>>>>>> self.generate(section[subsection], gen_ts) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** ngen += >>>>>>>>>>> self.generate(section[subsection], gen_ts) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 320, in generate >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** >>>>>>>>>>> default_binding) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weewx/cheetahgenerator.py", line 372, in >>>>>>>>>>> _getSearchList >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** >>>>>>>>>>> searchList += obj.get_extension_list(timespan, db_lookup) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/user/belchertown_highchartsSearchX.py", line 88, >>>>>>>>>>> in >>>>>>>>>>> get_extension_list >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** >>>>>>>>>>> (time_start_vt, time_stop_vt, outTemp_vt) = >>>>>>>>>>> db_lookup().getSqlVectors(TimeSpan(_start_ts, timespan.stop), >>>>>>>>>>> 'outTemp') >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** File >>>>>>>>>>> "/usr/share/weewx/weeutil/weeutil.py", line 315, in __new__ >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** raise >>>>>>>>>>> ValueError("start time (%d) is greater than stop time (%d)" % >>>>>>>>>>> (args[0], >>>>>>>>>>> args[1])) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** ValueError: >>>>>>>>>>> start time (1535346000) is greater than stop time (1420092000) >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: **** Generator >>>>>>>>>>> terminated >>>>>>>>>>> Aug 27 09:17:47 localhost weewx[4131]: copygenerator: copied 1 >>>>>>>>>>> files to /var/www/html/weewx/belchertown >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Monday, August 27, 2018 at 8:35:00 AM UTC-5, David Hathaway >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Thanks Pat. I moved the options into weewx.conf after reading >>>>>>>>>>>> that suggestion. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Aug 27, 2018 at 8:03 AM Pat <> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'd suggest using the latest release >>>>>>>>>>>>> <https://github.com/poblabs/weewx-belchertown/releases>. No >>>>>>>>>>>>> issues with upgrading if all your Belchertown options are within >>>>>>>>>>>>> weewx.conf. See the GitHub readme >>>>>>>>>>>>> <https://github.com/poblabs/weewx-belchertown> for those >>>>>>>>>>>>> options. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "weewx-user" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> weewx-user+...@googlegroups.com. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "weewx-user" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> weewx-user+...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "weewx-user" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> weewx-user+...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
