Re: [weewx-user] Re: Database Error Message

[Michael Sanphillipo]

Here is a sanitized copy of my Weewx.conf. I do have all of my Belchertown
settings in the weewx conf , so it adds a bunch of lines. I'm really not
sure about the synthetic measurements.


# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2019 Tom Keffer 
# See the file LICENSE.txt for your rights.

##

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 0

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /

# Whether to log successful operations
log_success = True

# Whether to log unsuccessful operations
log_failure = True

# How long to wait before timing out a socket (FTP, HTTP) connection
socket_timeout = 20

# Do not modify this. It is used when installing and updating weewx.
version = 4.10.2

##

#   This section is for information about the station.

[Station]

# Description of the station location
location = "Riverside Belcamp, MD"

# Latitude in decimal degrees. Negative for southern hemisphere
latitude = 39.478306
# Longitude in decimal degrees. Negative for western hemisphere.
longitude = -76.252562

# Altitude of the station, with unit it is in. This is downloaded from
# from the station if the hardware supports it.
altitude = 100, foot

# Set to type of station hardware. There must be a corresponding stanza
# in this file with a 'driver' parameter indicating the driver to be
used.
station_type = AcuRite

# If you have a website, you may specify an URL
station_url = https://belcampmdweather.tech/weewx/belchertown/

# The start of the rain year (1=January; 10=October, etc.). This is
# downloaded from the station if the hardware supports it.
rain_year_start = 1

# Start of week (0=Monday, 6=Sunday)
week_start = 6

##

[AcuRite]
# This section is for AcuRite weather stations.

# The station model, e.g., 'AcuRite 01025' or 'AcuRite 02032C'
model = AcuRite 02032C

# The driver to use:
driver = weewx.drivers.acurite
use_constants = False
##

#   This section is for uploading data to Internet sites

[StdRESTful]

[[StationRegistry]]
# To register this weather station with weewx, set this to true
register_this_station = true

[[AWEKAS]]
# This section is for configuring posts to AWEKAS.

# If you wish to do this, set the option 'enable' to true,
# and specify a username and password.
# To guard against parsing errors, put the password in quotes.
enable = true
username = ##
password = #

[[CWOP]]
# This section is for configuring posts to CWOP.

# If you wish to do this, set the option 'enable' to true,
# and specify the station ID (e.g., CW1234).
enable = true
station = FW5836

# If this is an APRS (radio amateur) station, uncomment
# the following and replace with a passcode (e.g., 12345).
#passcode = replace_me (APRS stations only)

[[PWSweather]]
# This section is for configuring posts to PWSweather.com.

# If you wish to do this, set the option 'enable' to true,
# and specify a station and password.
# To guard against parsing errors, put the password in quotes.
enable = true
station = KMDBELCA3
password = #

[[WOW]]
# This section is for configuring posts to WOW.

# If you wish to do this, set the option 'enable' to true,
# and specify a station and password.
# To guard against parsing errors, put the password in quotes.
enable = false
station = replace_me
password = replace_me

[[Wunderground]]
# This section is for configuring posts to the Weather Underground.

# If you wish to do this, set the option 'enable' to true,
# and specify a station (e.g., 'KORHOODR3') and password.
# To guard against parsing errors, put the password in quotes.
enable = false
station = KMDBELCA3
password = 

# If you plan on using wunderfixer, set the following
# to your API key:
api_key = ###

# Set the following to True to have weewx use the WU "Rapidfire"
# protocol. Not all hardware can support it. See the User's Guide.
rapidfire = True

post_indoor_observations = true

[[WeatherCloud]]
id = f
key = ##

[[MQTT]]
server_url = mqtt://:#@localhost:1883/
topic = weather
#server_url = mqtt://broker.hivemq.com:1883/
#topic = weather/belcamp
 

Re: [weewx-user] Re: Database Error Message

[Michael Sanphillipo]

I'm also using my Pi3 as a MQTT server feeding Belchertown I'm not sure if
that makes a difference or not. With all of that being said everything runs
smoothly on WeeWxVersion 4. I know Tom said that version 5 does a lot more.
I also use my PI3 for ADSB sending data to a few ADSB sites.

On Wed, Jan 17, 2024 at 7:25 PM vince  wrote:

> It's not Belchertown.  It's not db size.  I ran many tests.
>
> v5 and Belchertown can easily coexist on a pi3 or so system.  If every
> time through you're exceeding the default archive period you're doing
> something custom to make that happen.  I keep coming back to the known
> issue/slowness with synthetic measurements that Tom mentioned a few times
> recently.
>
> Here's the test I just did with a newly flashed 32-bit pi3+ running a
> fully patched latest raspios...
>
> On a clean pi3+ with weewx v5 via pip and Belchertown installed, using the
> Simulator driver and no old pre-existing db present:
>
> Generated 8 files for report SeasonsReport in 3.45 seconds
> Generated 56 images for report SeasonsReport in 3.43 seconds
> Generated 12 files for report Belchertown in 6.93 seconds
> (elapsed time from writing to the db to Belchertown done was 14 seconds)
>
> I then stopped weewx, overwrote my weewx.sdb with my production system
> data with 17+ years of data, pre-seeded the Seasons NOAA tree so I didn't
> need to wait for them to be regenerated once for this test, then restarted
> weewx.  So it's running the same skins with 17+ years of data, with the
> Seasons NOAA files having been pregenerated and dropped into place
>
> Generated 8 files for report SeasonsReport in 4.28 seconds
> Generated 44 images for report SeasonsReport in 6.75 seconds
> *Generated 236 files for report Belchertown in 266.59 seconds.  <=*
> (elapsed time from writing to db to Belchertown done was 4:37 so pretty
> close to a 5-minute default archive period)
>
> Why ?  Because the 'first' Belchertown run generates its 'own' copy of
> the NOAA files down under the Belchertown public_html tree.  It had to do
> that (for me) with 17+ years of month and year files which takes some time
> to complete.   Running htop I saw two virtual cpus pegged at 100%
> throughout until it completed.
>
> After that Belchertown NOAA files are also pre-seeded there too so it's
> just doing normal incremental updates for future report runs.  Each skin
> that generates NOAA files has to work hard once to do so vs. a pre-existing
> db if you start with a legacy db.   You only have to bleed through that
> once.  Subsequent runs should be quick.  Lets see...
>
> So I let it run another cycle.looks fine
>
> Generated 8 files for report SeasonsReport in 3.05 seconds
> Generated 18 images for report SeasonsReport in 1.22 seconds
> Generated 12 files for report Belchertown in 8.26 seconds
> (elapsed time this cycle was 14 seconds)
>
> And one more cycle.looks fine
>
> Generated 8 files for report SeasonsReport in 2.94 seconds
> Generated 18 images for report SeasonsReport in 1.68 seconds
> Generated 12 files for report Belchertown in 7.80 seconds
> (elapsed time for this cycle was 14 seconds)
>
> Again - if you are continually exceeding perhaps 15 seconds for
> Belchertown to run on a pi3 or better you are doing something custom to
> make that happen.  I just keep coming beck mentally to Tom's previous
> responses re: synthetic measurements.
>
> The original poster uses an Acurite which I don't know enough about to
> know for certain, but are you using that feature and not even knowing it ?
> Would providing your (sanitized) weewx.conf perhaps be of use in helping
> figure it out ?
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/9815dc0c-c883-43a3-b9e4-87f4e3a3986fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAAsGudpq8kEiq7rP5J%3DhhULuyZfhzG0yJoKh0fyyvCPFogGHAQ%40mail.gmail.com.

Re: [weewx-user] Re: Database Error Message

[vince]

It's not Belchertown.  It's not db size.  I ran many tests.

v5 and Belchertown can easily coexist on a pi3 or so system.  If every time 
through you're exceeding the default archive period you're doing something 
custom to make that happen.  I keep coming back to the known issue/slowness 
with synthetic measurements that Tom mentioned a few times recently. 

Here's the test I just did with a newly flashed 32-bit pi3+ running a fully 
patched latest raspios...

On a clean pi3+ with weewx v5 via pip and Belchertown installed, using the 
Simulator driver and no old pre-existing db present:

Generated 8 files for report SeasonsReport in 3.45 seconds
Generated 56 images for report SeasonsReport in 3.43 seconds
Generated 12 files for report Belchertown in 6.93 seconds
(elapsed time from writing to the db to Belchertown done was 14 seconds)

I then stopped weewx, overwrote my weewx.sdb with my production system data 
with 17+ years of data, pre-seeded the Seasons NOAA tree so I didn't need 
to wait for them to be regenerated once for this test, then restarted 
weewx.  So it's running the same skins with 17+ years of data, with the 
Seasons NOAA files having been pregenerated and dropped into place

Generated 8 files for report SeasonsReport in 4.28 seconds
Generated 44 images for report SeasonsReport in 6.75 seconds
*Generated 236 files for report Belchertown in 266.59 seconds.  <=*
(elapsed time from writing to db to Belchertown done was 4:37 so pretty 
close to a 5-minute default archive period)

Why ?  Because the 'first' Belchertown run generates its 'own' copy of the 
NOAA files down under the Belchertown public_html tree.  It had to do that 
(for me) with 17+ years of month and year files which takes some time to 
complete.   Running htop I saw two virtual cpus pegged at 100% throughout 
until it completed.

After that Belchertown NOAA files are also pre-seeded there too so it's 
just doing normal incremental updates for future report runs.  Each skin 
that generates NOAA files has to work hard once to do so vs. a pre-existing 
db if you start with a legacy db.   You only have to bleed through that 
once.  Subsequent runs should be quick.  Lets see...

So I let it run another cycle.looks fine

Generated 8 files for report SeasonsReport in 3.05 seconds
Generated 18 images for report SeasonsReport in 1.22 seconds
Generated 12 files for report Belchertown in 8.26 seconds
(elapsed time this cycle was 14 seconds)

And one more cycle.looks fine

Generated 8 files for report SeasonsReport in 2.94 seconds
Generated 18 images for report SeasonsReport in 1.68 seconds
Generated 12 files for report Belchertown in 7.80 seconds
(elapsed time for this cycle was 14 seconds)

Again - if you are continually exceeding perhaps 15 seconds for Belchertown 
to run on a pi3 or better you are doing something custom to make that 
happen.  I just keep coming beck mentally to Tom's previous responses re: 
synthetic measurements.

The original poster uses an Acurite which I don't know enough about to know 
for certain, but are you using that feature and not even knowing it ? 
 Would providing your (sanitized) weewx.conf perhaps be of use in helping 
figure it out ?

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/9815dc0c-c883-43a3-b9e4-87f4e3a3986fn%40googlegroups.com.

Re: [weewx-user] Re: Database Error Message

[p q]

It's not "old" that I'm thinking, but it might be big. Though, it didn't
suddenly get bigger with 5.0, so it would be in combination with something
else. I'm betting on something going on in Belchertown.

On Wed, Jan 17, 2024 at 3:19 PM Michael Sanphillipo 
wrote:

> My current database goes back to 2018 so I guess it could be an issue. I
> really don't know if that is considered old or not. The fact that it runs
> fine on version 4 leads me to think that might not be it.
>
> On Wed, Jan 17, 2024 at 6:14 PM p q  wrote:
>
>> You might also swap out your database with an empty one to see if the
>> volume of data in the database is causing the slowness. I really doubt it,
>> though.
>>
>> On Wed, Jan 17, 2024 at 2:32 PM vince  wrote:
>>
>>> Michael -  guessing you have something customized under the hood causing
>>> this.  Again you have a few choices
>>>
>>>- disable Belchertown (temporarily) and see if it goes away
>>>- switch to the Simulator (temporarily) after saving your weewx
>>>setup and see if it's related to the Acurite stuff you're using
>>>- disable everything 'except' weewx (temporarily) and see if it goes
>>>away
>>>- throw hardware at it and get a faster computer.  You might even be
>>>paging on a pi3 and a 4GB pi4 or pi5 might be enough
>>>
>>> Kinda impossible to know with skeletal data to look at, but these things
>>> can be difficult to dig into.
>>>
>>> I did cause something similar yesterday trying to run on a pi5 where the
>>> StdReport never finished in even 5 minutes but didn't figure out why.  I
>>> 'think' it was something related to having ancient simulated data in the db
>>> from timesync issues (2014 dates in the NOAA files is definitely wrong) and
>>> switching to a different driver.  Moving the old db aside seemed to help,
>>> but again I didn't dig much into it yesterday on a test system.
>>>
>>> On Wednesday, January 17, 2024 at 12:37:35 PM UTC-8 Tom Keffer wrote:
>>>
>>>> Maybe, although I've run V5 without any problems on a RPi Zero! There's
>>>> something in your Belchertown skins that is triggering the long runtimes.
>>>>
>>>> On Wed, Jan 17, 2024 at 6:42 AM Michael Sanphillipo 
>>>> wrote:
>>>>
>>>>> FWIW I went back to 4.10.2 and no more database errors and the CPU
>>>>> spikes are gone. Might be time to upgrade to a PI4 or PI5.
>>>>>
>>>>> On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:
>>>>>
>>>>>> The difference is that V5 could be doing more. We don't know for
>>>>>> sure, yet, but it is probably reporting on synthetic types, which V4 was
>>>>>> unable to do.
>>>>>>
>>>>>> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo 
>>>>>> wrote:
>>>>>>
>>>>>>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the
>>>>>>> archive to 300 so we'll see how it goes. Never had the issue before
>>>>>>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed 
>>>>>>> that
>>>>>>> CPU usage and temps are slightly higher than average since I upgraded 
>>>>>>> last
>>>>>>> night. Thanks for the suggestions.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sent from my Verizon, Samsung Galaxy smartphone
>>>>>>>
>>>>>>>
>>>>>>>  Original message 
>>>>>>> From: vince 
>>>>>>> Date: 1/16/24 4:12 PM (GMT-05:00)
>>>>>>> To: weewx-user 
>>>>>>> Subject: Re: [weewx-user] Re: Database Error Message
>>>>>>>
>>>>>>> The log excerpt is a little short but perhaps.   I see in your log
>>>>>>> you have a 120 second archive period yet Belchertown took 125 seconds to
>>>>>>> run (which is 'very' slow), so it's possible your archive period is too
>>>>>>> fast for your hardware.  What kind of pi are you on ?  Something slower
>>>>>>> than a pi4 perhaps ?
>>>>>>>
>>>>>>> If you are on slow hardware try disabling Belchertown temporarily
>>>>>>> and see if the problem goes away.  Or alternately set your archive 
>>>>>>> per

Re: [weewx-user] Re: Database Error Message

[Michael Sanphillipo]

My current database goes back to 2018 so I guess it could be an issue. I
really don't know if that is considered old or not. The fact that it runs
fine on version 4 leads me to think that might not be it.

On Wed, Jan 17, 2024 at 6:14 PM p q  wrote:

> You might also swap out your database with an empty one to see if the
> volume of data in the database is causing the slowness. I really doubt it,
> though.
>
> On Wed, Jan 17, 2024 at 2:32 PM vince  wrote:
>
>> Michael -  guessing you have something customized under the hood causing
>> this.  Again you have a few choices
>>
>>- disable Belchertown (temporarily) and see if it goes away
>>- switch to the Simulator (temporarily) after saving your weewx setup
>>and see if it's related to the Acurite stuff you're using
>>- disable everything 'except' weewx (temporarily) and see if it goes
>>away
>>- throw hardware at it and get a faster computer.  You might even be
>>paging on a pi3 and a 4GB pi4 or pi5 might be enough
>>
>> Kinda impossible to know with skeletal data to look at, but these things
>> can be difficult to dig into.
>>
>> I did cause something similar yesterday trying to run on a pi5 where the
>> StdReport never finished in even 5 minutes but didn't figure out why.  I
>> 'think' it was something related to having ancient simulated data in the db
>> from timesync issues (2014 dates in the NOAA files is definitely wrong) and
>> switching to a different driver.  Moving the old db aside seemed to help,
>> but again I didn't dig much into it yesterday on a test system.
>>
>> On Wednesday, January 17, 2024 at 12:37:35 PM UTC-8 Tom Keffer wrote:
>>
>>> Maybe, although I've run V5 without any problems on a RPi Zero! There's
>>> something in your Belchertown skins that is triggering the long runtimes.
>>>
>>> On Wed, Jan 17, 2024 at 6:42 AM Michael Sanphillipo 
>>> wrote:
>>>
>>>> FWIW I went back to 4.10.2 and no more database errors and the CPU
>>>> spikes are gone. Might be time to upgrade to a PI4 or PI5.
>>>>
>>>> On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:
>>>>
>>>>> The difference is that V5 could be doing more. We don't know for sure,
>>>>> yet, but it is probably reporting on synthetic types, which V4 was unable
>>>>> to do.
>>>>>
>>>>> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo 
>>>>> wrote:
>>>>>
>>>>>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the
>>>>>> archive to 300 so we'll see how it goes. Never had the issue before
>>>>>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed 
>>>>>> that
>>>>>> CPU usage and temps are slightly higher than average since I upgraded 
>>>>>> last
>>>>>> night. Thanks for the suggestions.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sent from my Verizon, Samsung Galaxy smartphone
>>>>>>
>>>>>>
>>>>>>  Original message 
>>>>>> From: vince 
>>>>>> Date: 1/16/24 4:12 PM (GMT-05:00)
>>>>>> To: weewx-user 
>>>>>> Subject: Re: [weewx-user] Re: Database Error Message
>>>>>>
>>>>>> The log excerpt is a little short but perhaps.   I see in your log
>>>>>> you have a 120 second archive period yet Belchertown took 125 seconds to
>>>>>> run (which is 'very' slow), so it's possible your archive period is too
>>>>>> fast for your hardware.  What kind of pi are you on ?  Something slower
>>>>>> than a pi4 perhaps ?
>>>>>>
>>>>>> If you are on slow hardware try disabling Belchertown temporarily and
>>>>>> see if the problem goes away.  Or alternately set your archive period to
>>>>>> perhaps 180 seconds and try that.  Or as always you might throw hardware 
>>>>>> at
>>>>>> it and use a faster pibut it looks like you're asking your hardware 
>>>>>> to
>>>>>> do too much too often.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to a topic in
>>>>>> the Google Groups "weewx-user" group.
>>>>>> To unsubscribe from this topic, visit
>>>>>> https://

Re: [weewx-user] Re: Database Error Message

[p q]

You might also swap out your database with an empty one to see if the
volume of data in the database is causing the slowness. I really doubt it,
though.

On Wed, Jan 17, 2024 at 2:32 PM vince  wrote:

> Michael -  guessing you have something customized under the hood causing
> this.  Again you have a few choices
>
>- disable Belchertown (temporarily) and see if it goes away
>- switch to the Simulator (temporarily) after saving your weewx setup
>and see if it's related to the Acurite stuff you're using
>- disable everything 'except' weewx (temporarily) and see if it goes
>away
>- throw hardware at it and get a faster computer.  You might even be
>paging on a pi3 and a 4GB pi4 or pi5 might be enough
>
> Kinda impossible to know with skeletal data to look at, but these things
> can be difficult to dig into.
>
> I did cause something similar yesterday trying to run on a pi5 where the
> StdReport never finished in even 5 minutes but didn't figure out why.  I
> 'think' it was something related to having ancient simulated data in the db
> from timesync issues (2014 dates in the NOAA files is definitely wrong) and
> switching to a different driver.  Moving the old db aside seemed to help,
> but again I didn't dig much into it yesterday on a test system.
>
> On Wednesday, January 17, 2024 at 12:37:35 PM UTC-8 Tom Keffer wrote:
>
>> Maybe, although I've run V5 without any problems on a RPi Zero! There's
>> something in your Belchertown skins that is triggering the long runtimes.
>>
>> On Wed, Jan 17, 2024 at 6:42 AM Michael Sanphillipo 
>> wrote:
>>
>>> FWIW I went back to 4.10.2 and no more database errors and the CPU
>>> spikes are gone. Might be time to upgrade to a PI4 or PI5.
>>>
>>> On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:
>>>
>>>> The difference is that V5 could be doing more. We don't know for sure,
>>>> yet, but it is probably reporting on synthetic types, which V4 was unable
>>>> to do.
>>>>
>>>> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo  wrote:
>>>>
>>>>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the
>>>>> archive to 300 so we'll see how it goes. Never had the issue before
>>>>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed 
>>>>> that
>>>>> CPU usage and temps are slightly higher than average since I upgraded last
>>>>> night. Thanks for the suggestions.
>>>>>
>>>>>
>>>>>
>>>>> Sent from my Verizon, Samsung Galaxy smartphone
>>>>>
>>>>>
>>>>>  Original message 
>>>>> From: vince 
>>>>> Date: 1/16/24 4:12 PM (GMT-05:00)
>>>>> To: weewx-user 
>>>>> Subject: Re: [weewx-user] Re: Database Error Message
>>>>>
>>>>> The log excerpt is a little short but perhaps.   I see in your log you
>>>>> have a 120 second archive period yet Belchertown took 125 seconds to run
>>>>> (which is 'very' slow), so it's possible your archive period is too fast
>>>>> for your hardware.  What kind of pi are you on ?  Something slower than a
>>>>> pi4 perhaps ?
>>>>>
>>>>> If you are on slow hardware try disabling Belchertown temporarily and
>>>>> see if the problem goes away.  Or alternately set your archive period to
>>>>> perhaps 180 seconds and try that.  Or as always you might throw hardware 
>>>>> at
>>>>> it and use a faster pibut it looks like you're asking your hardware to
>>>>> do too much too often.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to a topic in the
>>>>> Google Groups "weewx-user" group.
>>>>> To unsubscribe from this topic, visit
>>>>> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>> weewx-user+...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>>> --
>>>>>
>>&

Re: [weewx-user] Re: Database Error Message

[vince]

Michael -  guessing you have something customized under the hood causing 
this.  Again you have a few choices

   - disable Belchertown (temporarily) and see if it goes away
   - switch to the Simulator (temporarily) after saving your weewx setup 
   and see if it's related to the Acurite stuff you're using
   - disable everything 'except' weewx (temporarily) and see if it goes away
   - throw hardware at it and get a faster computer.  You might even be 
   paging on a pi3 and a 4GB pi4 or pi5 might be enough

Kinda impossible to know with skeletal data to look at, but these things 
can be difficult to dig into.

I did cause something similar yesterday trying to run on a pi5 where the 
StdReport never finished in even 5 minutes but didn't figure out why.  I 
'think' it was something related to having ancient simulated data in the db 
from timesync issues (2014 dates in the NOAA files is definitely wrong) and 
switching to a different driver.  Moving the old db aside seemed to help, 
but again I didn't dig much into it yesterday on a test system.

On Wednesday, January 17, 2024 at 12:37:35 PM UTC-8 Tom Keffer wrote:

> Maybe, although I've run V5 without any problems on a RPi Zero! There's 
> something in your Belchertown skins that is triggering the long runtimes.
>
> On Wed, Jan 17, 2024 at 6:42 AM Michael Sanphillipo  
> wrote:
>
>> FWIW I went back to 4.10.2 and no more database errors and the CPU spikes 
>> are gone. Might be time to upgrade to a PI4 or PI5. 
>>
>> On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:
>>
>>> The difference is that V5 could be doing more. We don't know for sure, 
>>> yet, but it is probably reporting on synthetic types, which V4 was unable 
>>> to do.
>>>
>>> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo  wrote:
>>>
>>>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the 
>>>> archive to 300 so we'll see how it goes. Never had the issue before 
>>>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed 
>>>> that 
>>>> CPU usage and temps are slightly higher than average since I upgraded last 
>>>> night. Thanks for the suggestions. 
>>>>
>>>>
>>>>
>>>> Sent from my Verizon, Samsung Galaxy smartphone
>>>>
>>>>
>>>>  Original message 
>>>> From: vince  
>>>> Date: 1/16/24 4:12 PM (GMT-05:00) 
>>>> To: weewx-user  
>>>> Subject: Re: [weewx-user] Re: Database Error Message 
>>>>
>>>> The log excerpt is a little short but perhaps.   I see in your log you 
>>>> have a 120 second archive period yet Belchertown took 125 seconds to run 
>>>> (which is 'very' slow), so it's possible your archive period is too fast 
>>>> for your hardware.  What kind of pi are you on ?  Something slower than a 
>>>> pi4 perhaps ?
>>>>
>>>> If you are on slow hardware try disabling Belchertown temporarily and 
>>>> see if the problem goes away.  Or alternately set your archive period to 
>>>> perhaps 180 seconds and try that.  Or as always you might throw hardware 
>>>> at 
>>>> it and use a faster pibut it looks like you're asking your hardware to 
>>>> do too much too often.
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to a topic in the 
>>>> Google Groups "weewx-user" group.
>>>> To unsubscribe from this topic, visit 
>>>> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
>>>> To unsubscribe from this group and all its topics, send an email to 
>>>> weewx-user+...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
>>>>  
>>>> <https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>>> -- 
>>>>
>>> You received this message because you are subscribed to the Google 
>>>> Groups "weewx-user" group.
>>>>
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to weewx-user+...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com
>>>>  
>>>> <https://groups.google.com

Re: [weewx-user] Re: Database Error Message

[Tom Keffer]

Maybe, although I've run V5 without any problems on a RPi Zero! There's
something in your Belchertown skins that is triggering the long runtimes.

On Wed, Jan 17, 2024 at 6:42 AM Michael Sanphillipo 
wrote:

> FWIW I went back to 4.10.2 and no more database errors and the CPU spikes
> are gone. Might be time to upgrade to a PI4 or PI5.
>
> On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:
>
>> The difference is that V5 could be doing more. We don't know for sure,
>> yet, but it is probably reporting on synthetic types, which V4 was unable
>> to do.
>>
>> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo  wrote:
>>
>>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the
>>> archive to 300 so we'll see how it goes. Never had the issue before
>>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed that
>>> CPU usage and temps are slightly higher than average since I upgraded last
>>> night. Thanks for the suggestions.
>>>
>>>
>>>
>>> Sent from my Verizon, Samsung Galaxy smartphone
>>>
>>>
>>>  Original message 
>>> From: vince 
>>> Date: 1/16/24 4:12 PM (GMT-05:00)
>>> To: weewx-user 
>>> Subject: Re: [weewx-user] Re: Database Error Message
>>>
>>> The log excerpt is a little short but perhaps.   I see in your log you
>>> have a 120 second archive period yet Belchertown took 125 seconds to run
>>> (which is 'very' slow), so it's possible your archive period is too fast
>>> for your hardware.  What kind of pi are you on ?  Something slower than a
>>> pi4 perhaps ?
>>>
>>> If you are on slow hardware try disabling Belchertown temporarily and
>>> see if the problem goes away.  Or alternately set your archive period to
>>> perhaps 180 seconds and try that.  Or as always you might throw hardware at
>>> it and use a faster pibut it looks like you're asking your hardware to
>>> do too much too often.
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "weewx-user" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> weewx-user+...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>> --
>>>
>> You received this message because you are subscribed to the Google Groups
>>> "weewx-user" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to weewx-user+...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com
>>> <https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/11b0239b-0bd7-4d2f-a5ee-12cc99e402aan%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/11b0239b-0bd7-4d2f-a5ee-12cc99e402aan%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEAsW87u9EJ27WaXysfzS0M2VfeuzLQg%3DHgMNyHs%2Bvf_2A%40mail.gmail.com.

Re: [weewx-user] Re: Database Error Message

[Michael Sanphillipo]

FWIW I went back to 4.10.2 and no more database errors and the CPU spikes 
are gone. Might be time to upgrade to a PI4 or PI5. 

On Tuesday, January 16, 2024 at 5:41:39 PM UTC-5 Tom Keffer wrote:

> The difference is that V5 could be doing more. We don't know for sure, 
> yet, but it is probably reporting on synthetic types, which V4 was unable 
> to do.
>
> On Tue, Jan 16, 2024 at 2:36 PM sanphillipo  wrote:
>
>> Yes I'm on a PI3 with ADSB tracking running as well. I changed the 
>> archive to 300 so we'll see how it goes. Never had the issue before 
>> upgrading to 5.0 so it must be somewhat related to that.  I've noticed that 
>> CPU usage and temps are slightly higher than average since I upgraded last 
>> night. Thanks for the suggestions. 
>>
>>
>>
>> Sent from my Verizon, Samsung Galaxy smartphone
>>
>>
>>  Original message 
>> From: vince  
>> Date: 1/16/24 4:12 PM (GMT-05:00) 
>> To: weewx-user  
>> Subject: Re: [weewx-user] Re: Database Error Message 
>>
>> The log excerpt is a little short but perhaps.   I see in your log you 
>> have a 120 second archive period yet Belchertown took 125 seconds to run 
>> (which is 'very' slow), so it's possible your archive period is too fast 
>> for your hardware.  What kind of pi are you on ?  Something slower than a 
>> pi4 perhaps ?
>>
>> If you are on slow hardware try disabling Belchertown temporarily and see 
>> if the problem goes away.  Or alternately set your archive period to 
>> perhaps 180 seconds and try that.  Or as always you might throw hardware at 
>> it and use a faster pibut it looks like you're asking your hardware to 
>> do too much too often.
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "weewx-user" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> weewx-user+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> -- 
>>
> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>>
> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com
>>  
>> <https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/11b0239b-0bd7-4d2f-a5ee-12cc99e402aan%40googlegroups.com.

Re: [weewx-user] Re: Database Error Message

[Tom Keffer]

The difference is that V5 could be doing more. We don't know for sure, yet,
but it is probably reporting on synthetic types, which V4 was unable to do.

On Tue, Jan 16, 2024 at 2:36 PM sanphillipo  wrote:

> Yes I'm on a PI3 with ADSB tracking running as well. I changed the archive
> to 300 so we'll see how it goes. Never had the issue before upgrading to
> 5.0 so it must be somewhat related to that.  I've noticed that CPU usage
> and temps are slightly higher than average since I upgraded last night.
> Thanks for the suggestions.
>
>
>
> Sent from my Verizon, Samsung Galaxy smartphone
>
>
>  Original message 
> From: vince 
> Date: 1/16/24 4:12 PM (GMT-05:00)
> To: weewx-user 
> Subject: Re: [weewx-user] Re: Database Error Message
>
> The log excerpt is a little short but perhaps.   I see in your log you
> have a 120 second archive period yet Belchertown took 125 seconds to run
> (which is 'very' slow), so it's possible your archive period is too fast
> for your hardware.  What kind of pi are you on ?  Something slower than a
> pi4 perhaps ?
>
> If you are on slow hardware try disabling Belchertown temporarily and see
> if the problem goes away.  Or alternately set your archive period to
> perhaps 180 seconds and try that.  Or as always you might throw hardware at
> it and use a faster pibut it looks like you're asking your hardware to
> do too much too often.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com
> <https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEBwQvaRVkDa4YKomDdTb6CyUK9rPndafCkFpC1hBir81g%40mail.gmail.com.

Re: [weewx-user] Re: Database Error Message

[Tom Keffer]

On a MacBook Air M1 I'm getting "Generated 32 files for report Belchertown
in 1.21 seconds"

We know from our benchmarks

that it's about 20x faster than an RPi3, so I would expect times in the
24-30 second range.

On Tue, Jan 16, 2024 at 2:09 PM vince  wrote:

> Tom - FWIW on a pi4 with multiple skins+extensions enabled I'm seeing
> Belchertown generate in 4 to 11 seconds depending on time of day so over
> 120 definitely seems like a hardware problem or that synthetic reading
> thing if they've customized at all.  Typical is under 5 secs with hourly
> perhaps a little more.
>
> On Tuesday, January 16, 2024 at 1:14:51 PM UTC-8 Tom Keffer wrote:
>
> Agree with Vince.
>
> However, it seems that the Belchertown skin is very slow under V5. I'll
> try and do some investigation later. In the meantime, switching to a 5
> minute archive interval will probably solve your problem.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/1d874032-ca50-47df-ac89-ce632ec9389fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEAmi7CO-NTj5Cn99GS3-q1b66ebutzJ3OVKVJQHDGwChA%40mail.gmail.com.

Re: [weewx-user] Re: Database Error Message

[sanphillipo]

Yes I'm on a PI3 with ADSB tracking running as well. I changed the archive to 
300 so we'll see how it goes. Never had the issue before upgrading to 5.0 so it 
must be somewhat related to that.  I've noticed that CPU usage and temps are 
slightly higher than average since I upgraded last night. Thanks for the 
suggestions. Sent from my Verizon, Samsung Galaxy smartphone
 Original message From: vince  Date: 
1/16/24  4:12 PM  (GMT-05:00) To: weewx-user  
Subject: Re: [weewx-user] Re: Database Error Message The log excerpt is a 
little short but perhaps.   I see in your log you have a 120 second archive 
period yet Belchertown took 125 seconds to run (which is 'very' slow), so it's 
possible your archive period is too fast for your hardware.  What kind of pi 
are you on ?  Something slower than a pi4 perhaps ?If you are on slow hardware 
try disabling Belchertown temporarily and see if the problem goes away.  Or 
alternately set your archive period to perhaps 180 seconds and try that.  Or as 
always you might throw hardware at it and use a faster pibut it looks like 
you're asking your hardware to do too much too often.



-- 
You received this message because you are subscribed to a topic in the Google 
Groups "weewx-user" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/weewx-user/7tMPKNiRpZg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/65a704ff.050a0220.9fe86.27f4%40mx.google.com.

Re: [weewx-user] Re: Database Error Message

[vince]

Tom - FWIW on a pi4 with multiple skins+extensions enabled I'm seeing 
Belchertown generate in 4 to 11 seconds depending on time of day so over 
120 definitely seems like a hardware problem or that synthetic reading 
thing if they've customized at all.  Typical is under 5 secs with hourly 
perhaps a little more.

On Tuesday, January 16, 2024 at 1:14:51 PM UTC-8 Tom Keffer wrote:

Agree with Vince.

However, it seems that the Belchertown skin is very slow under V5. I'll try 
and do some investigation later. In the meantime, switching to a 5 minute 
archive interval will probably solve your problem.

 

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/1d874032-ca50-47df-ac89-ce632ec9389fn%40googlegroups.com.

Re: [weewx-user] Re: Database Error Message

[Tom Keffer]

Agree with Vince.

However, it seems that the Belchertown skin is very slow under V5. I'll try
and do some investigation later. In the meantime, switching to a 5 minute
archive interval will probably solve your problem.

-tk

On Tue, Jan 16, 2024 at 1:12 PM vince  wrote:

> The log excerpt is a little short but perhaps.   I see in your log you
> have a 120 second archive period yet Belchertown took 125 seconds to run
> (which is 'very' slow), so it's possible your archive period is too fast
> for your hardware.  What kind of pi are you on ?  Something slower than a
> pi4 perhaps ?
>
> If you are on slow hardware try disabling Belchertown temporarily and see
> if the problem goes away.  Or alternately set your archive period to
> perhaps 180 seconds and try that.  Or as always you might throw hardware at
> it and use a faster pibut it looks like you're asking your hardware to
> do too much too often.
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zECyoBvMOo-FToAnVKkBEGDUJ9GzpSF6YWbxDsTO8bk66w%40mail.gmail.com.

Re: [weewx-user] Re: Database Error Message

[vince]

The log excerpt is a little short but perhaps.   I see in your log you have 
a 120 second archive period yet Belchertown took 125 seconds to run (which 
is 'very' slow), so it's possible your archive period is too fast for your 
hardware.  What kind of pi are you on ?  Something slower than a pi4 
perhaps ?

If you are on slow hardware try disabling Belchertown temporarily and see 
if the problem goes away.  Or alternately set your archive period to 
perhaps 180 seconds and try that.  Or as always you might throw hardware at 
it and use a faster pibut it looks like you're asking your hardware to 
do too much too often.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/23f37da3-0240-473d-85c1-8102ac757590n%40googlegroups.com.

Re: [weewx-user] Re: Database Error Message

[Michael Sanphillipo]

Hope this is the information you were looking for.

Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Initializing
weewxd version 5.0.0
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Command line:
/usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Using Python
3.7.3 (default, Oct 11 2023, 09:51:27) #012[GCC 8.3.0]
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Located at
/usr/bin/python3
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Platform
Linux-5.10.103-v7+-armv7l-with-debian-10.13
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Locale:
'en_US.UTF-8'
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Entry path:
/usr/share/weewx/weewxd.py
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: WEEWX_ROOT: /
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Configuration
file: /etc/weewx/weewx.conf
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: User module:
/etc/weewx/bin/user
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO __main__: Debug: 1
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.engine: Loading
station type AcuRite (weewx.drivers.acurite)
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.drivers.acurite:
driver version is 0.4
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.engine: StdConvert
target unit is 0x1
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.wxservices:
StdWXCalculate will use data binding wx_binding
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: service version
is 0.21
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: realtime txt
output goes to /var/tmp/realtime.txt
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: 'None' values
will be displayed as NULL
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: units will be
displayed as US
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: zambretti
forecast: False
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.crt: binding is archive
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.engine: Archive will
use data binding wx_binding
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.engine: Record
generation will be attempted in 'hardware'
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.engine: Using archive
interval of 120 seconds (specified in weewx configuration)
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx:
StationRegistry: Station will be registered.
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx: Wunderground:
Posting not enabled.
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx: PWSWeather:
Data for station KMDBELCA3 will be posted
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx: CWOP: Data for
station FW5836 will be posted
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx: WOW: Posting
not enabled.
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO weewx.restx: AWEKAS: Data
will be uploaded for user ms372591
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.wcloud: service
version is 0.13
Jan 16 14:10:52 raspberrypi weewxd[23682]: INFO user.wcloud: Data will be
uploaded for id=fabdb92c4cbfe11a
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: service version
is 0.23
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: binding to
['archive', 'loop']
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: topic is weather
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: desired unit
system is US
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: data will be
uploaded to mqtt://ms372591:xxx@localhost:1883/
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.windy: version is 0.7
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.windy: Data will be
uploaded to https://stations.windy.com/pws/update
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.engine: 'pyephem'
detected, extended almanac data is available
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO __main__: Starting up weewx
version 5.0.0
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.engine: Using binding
'wx_binding' to database 'weewx.sdb'
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.manager: Starting
backfill of daily summaries
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.manager: Daily
summaries up to date
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.engine: Starting main
packet loop.
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO weewx.restx: MQTT:
Published record 2024-01-16 14:10:53 EST (1705432253)
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: service version
is 0.23
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: binding to
['archive', 'loop']
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: topic is weather
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: desired unit
system is US
Jan 16 14:10:53 raspberrypi weewxd[23682]: INFO user.mqtt: data will be
uploaded to 

[weewx-user] Re: Database Error Message

[vince]

>From the FAQ 
- 
https://github.com/weewx/weewx/wiki/WeeWX-Frequently-Asked-Questions#where-do-i-find-the-log
>From the wiki - https://github.com/weewx/weewx/wiki/view-logs

On Tuesday, January 16, 2024 at 10:59:42 AM UTC-8 Michael Sanphillipo wrote:

> I set the debug to 1 and restarted Weewx. Where will the log be on a 
> Raspberry Pi3 running Buster?
>
> On Tuesday, January 16, 2024 at 12:58:53 PM UTC-5 vince wrote:
>
>> Usual ‘how to report a problem’ answer applies…
>>
>> Set debug=1, restart weewx, post the log from beginning of restart 
>> through ‘two’ reporting intervals. Some more info on what kind of pi might 
>> help too.
>>
>> On Tuesday, January 16, 2024 at 8:13:48 AM UTC-8 Michael Sanphillipo 
>> wrote:
>>
>>> I upgraded from v4 to v5 last night and I'm getting the following 
>>> message in my syslog every 6 minutes. My webpage is staying updated so I'm 
>>> not sure if this is anything to be concerned about.
>>>
>>> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Main loop 
>>> exiting. Shutting engine down.
>>> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Shutting 
>>> down StdReport thread
>>> Jan 16 11:08:53 raspberrypi weewxd[6863]: ERROR weewx.engine: Unable to 
>>> shut down StdReport thread
>>> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__: Database 
>>> OperationalError exception: database is locked
>>> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__:  
>>>  Waiting 2 minutes then retrying...
>>>
>>> Thanks,
>>> Mike
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/e2324ce0-22ba-4fa9-9d29-d7f3d2a464e2n%40googlegroups.com.

[weewx-user] Re: Database Error Message

[Michael Sanphillipo]

I set the debug to 1 and restarted Weewx. Where will the log be on a 
Raspberry Pi3 running Buster?

On Tuesday, January 16, 2024 at 12:58:53 PM UTC-5 vince wrote:

> Usual ‘how to report a problem’ answer applies…
>
> Set debug=1, restart weewx, post the log from beginning of restart through 
> ‘two’ reporting intervals. Some more info on what kind of pi might help too.
>
> On Tuesday, January 16, 2024 at 8:13:48 AM UTC-8 Michael Sanphillipo wrote:
>
>> I upgraded from v4 to v5 last night and I'm getting the following message 
>> in my syslog every 6 minutes. My webpage is staying updated so I'm not sure 
>> if this is anything to be concerned about.
>>
>> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Main loop 
>> exiting. Shutting engine down.
>> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Shutting 
>> down StdReport thread
>> Jan 16 11:08:53 raspberrypi weewxd[6863]: ERROR weewx.engine: Unable to 
>> shut down StdReport thread
>> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__: Database 
>> OperationalError exception: database is locked
>> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__:  
>>  Waiting 2 minutes then retrying...
>>
>> Thanks,
>> Mike
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/9f388eee-b8f3-4e3c-b37c-e343750a41ccn%40googlegroups.com.

[weewx-user] Re: Database Error Message

[vince]

Usual ‘how to report a problem’ answer applies…

Set debug=1, restart weewx, post the log from beginning of restart through 
‘two’ reporting intervals. Some more info on what kind of pi might help too.

On Tuesday, January 16, 2024 at 8:13:48 AM UTC-8 Michael Sanphillipo wrote:

> I upgraded from v4 to v5 last night and I'm getting the following message 
> in my syslog every 6 minutes. My webpage is staying updated so I'm not sure 
> if this is anything to be concerned about.
>
> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Main loop 
> exiting. Shutting engine down.
> Jan 16 11:08:33 raspberrypi weewxd[6863]: INFO weewx.engine: Shutting down 
> StdReport thread
> Jan 16 11:08:53 raspberrypi weewxd[6863]: ERROR weewx.engine: Unable to 
> shut down StdReport thread
> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__: Database 
> OperationalError exception: database is locked
> Jan 16 11:08:53 raspberrypi weewxd[6863]: CRITICAL __main__:  
>  Waiting 2 minutes then retrying...
>
> Thanks,
> Mike
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/b8d756ef-f1cd-430b-8793-87133038fe26n%40googlegroups.com.

Re: [weewx-user] Re: Database error

[Phil Owers]

Thomas
Using SQLITE on my raspberry with the 7" screen attached to it SQLITE was
twice the size and I couldn't get it smaller so was very difficult to use,
therefore decided to try it on the laptop with the bigger screen plus the
fact as I had a lot to change I could take it work and do the adjustments
there in between trsins. Hope my boss doesn't see this ???

On Wed, 31 Oct 2018, 15:51 Jules,  wrote:

> It worked!  installed Sqlite3 on my Pi, dumped, rebuilt, no more errors!
>
> Thanks again.
>
>
>
> On Wednesday, 31 October 2018 12:41:57 UTC, Jules wrote:
>>
>> Hi,
>>
>> I'm seeing database is malformed errors.  I've searched and found post on
>> how to fix this by running sqlite3 and dumping to ascii and re-importing.
>>
>> My problem is running sqlite3 !  My weewx Pi doesn't respond to sqlite3
>> as a command (I thought weewx used sqlite3?).  Do I install sqlite3 on my
>> weewx Pi or will that cause problems?
>>
>> I've tried installing the sqlite3 utilites on my windows machine.  I can
>> make it dump my weewx.sdb to my screen but can't work out how to make it
>> send it back to a new file - it doesn't chcuk an error but also doesn't
>> create the file.
>>
>> Can anyone point me in the right direction?
>>
>> Thanks,
>>
>>
>> Jules
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Database error

[Jules]

It worked!  installed Sqlite3 on my Pi, dumped, rebuilt, no more errors!

Thanks again.



On Wednesday, 31 October 2018 12:41:57 UTC, Jules wrote:
>
> Hi,
>
> I'm seeing database is malformed errors.  I've searched and found post on 
> how to fix this by running sqlite3 and dumping to ascii and re-importing.
>
> My problem is running sqlite3 !  My weewx Pi doesn't respond to sqlite3 as 
> a command (I thought weewx used sqlite3?).  Do I install sqlite3 on my 
> weewx Pi or will that cause problems?
>
> I've tried installing the sqlite3 utilites on my windows machine.  I can 
> make it dump my weewx.sdb to my screen but can't work out how to make it 
> send it back to a new file - it doesn't chcuk an error but also doesn't 
> create the file.
>
> Can anyone point me in the right direction?
>
> Thanks,
>
>
> Jules
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Database error

[Jules]

Hi,

I'll give that a go, thanks.

Jules



On Wednesday, 31 October 2018 12:41:57 UTC, Jules wrote:
>
> Hi,
>
> I'm seeing database is malformed errors.  I've searched and found post on 
> how to fix this by running sqlite3 and dumping to ascii and re-importing.
>
> My problem is running sqlite3 !  My weewx Pi doesn't respond to sqlite3 as 
> a command (I thought weewx used sqlite3?).  Do I install sqlite3 on my 
> weewx Pi or will that cause problems?
>
> I've tried installing the sqlite3 utilites on my windows machine.  I can 
> make it dump my weewx.sdb to my screen but can't work out how to make it 
> send it back to a new file - it doesn't chcuk an error but also doesn't 
> create the file.
>
> Can anyone point me in the right direction?
>
> Thanks,
>
>
> Jules
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: database error

['Ralf Buck' via weewx-user]

works again.

i restore my backup. now i change the system to orange pi +2 and install 
complete to emmc.


Am Samstag, 2. September 2017 12:12:56 UTC+2 schrieb Ralf Buck:
>
> hello
>
> it seems that my database had a error
>
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weewx/manager.py", line 214, in firstGoodStamp
> Sep 02 12:11:56 raspberrypi weewx[1385]:   _row = 
> self.getSql("SELECT MIN(dateTime) FROM %s" % self.table_name)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weewx/manager.py", line 394, in getSql
> Sep 02 12:11:56 raspberrypi weewx[1385]:   _cursor.execute(sql, 
> sqlargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weedb/sqlite.py", line 29, in guarded_fn
> Sep 02 12:11:56 raspberrypi weewx[1385]:   return fn(*args, 
> **kwargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weedb/sqlite.py", line 211, in execute
> Sep 02 12:11:56 raspberrypi weewx[1385]:   return 
> sqlite3.Cursor.execute(self, *args, **kwargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]:   DatabaseError: database 
> disk image is malformed
> Sep 02 12:11:56 raspberrypi weewx[1385]:   Exiting.
>
>
> how can i fix this?
>
> regard
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: database error

[Thomas Keffer]

Most likely the USB stick is corrupted. They are not designed to be primary
storage, so it has probably worn out and failed.

One thing to try. The error occurred while dumping the daily summaries,
which need not be saved --- they can recreated from the archive. So, you
can try just dumping the archive:

echo '.dump archive' | sqlite 3 ./weewx.sdb

If that does not work, you will have to restore from a backup.

Then use a top-quality SD card or, better yet, a SSD or hard drive.

-tk

On Sat, Sep 2, 2017 at 6:17 AM, 'Ralf Buck' via weewx-user <
weewx-user@googlegroups.com> wrote:

> echo '.dump' | sqlite3 ./weewx.sdb
>
> 
> INSERT INTO "archive_day_wind" VALUES(1494972000,0.0,
> 1494973048,13.7,1495019248,3.62385185185185498824e+02,288,
> 3.62385185185185498824e+02,288,NULL,2.34090915588452297413e+02,-8.
> 79672687838567135543e+01,259,9.18866833690265671918e+02,9.
> 18866833690265671918e+02);
> / ERROR: (11) database disk image is malformed */
> COMMIT;
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: database error

['Ralf Buck' via weewx-user]

echo '.dump' | sqlite3 ./weewx.sdb


INSERT INTO "archive_day_wind" 
VALUES(1494972000,0.0,1494973048,13.7,1495019248,3.62385185185185498824e+02,288,3.62385185185185498824e+02,288,NULL,2.34090915588452297413e+02,-8.79672687838567135543e+01,259,9.18866833690265671918e+02,9.18866833690265671918e+02);
/ ERROR: (11) database disk image is malformed */
COMMIT;

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: database error

['Ralf Buck' via weewx-user]

i will read this.

my sdb file is on a usb stick


Am Samstag, 2. September 2017 12:12:56 UTC+2 schrieb Ralf Buck:
>
> hello
>
> it seems that my database had a error
>
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weewx/manager.py", line 214, in firstGoodStamp
> Sep 02 12:11:56 raspberrypi weewx[1385]:   _row = 
> self.getSql("SELECT MIN(dateTime) FROM %s" % self.table_name)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weewx/manager.py", line 394, in getSql
> Sep 02 12:11:56 raspberrypi weewx[1385]:   _cursor.execute(sql, 
> sqlargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weedb/sqlite.py", line 29, in guarded_fn
> Sep 02 12:11:56 raspberrypi weewx[1385]:   return fn(*args, 
> **kwargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]: File 
> "/usr/share/weewx/weedb/sqlite.py", line 211, in execute
> Sep 02 12:11:56 raspberrypi weewx[1385]:   return 
> sqlite3.Cursor.execute(self, *args, **kwargs)
> Sep 02 12:11:56 raspberrypi weewx[1385]:   DatabaseError: database 
> disk image is malformed
> Sep 02 12:11:56 raspberrypi weewx[1385]:   Exiting.
>
>
> how can i fix this?
>
> regard
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.