[weewx-user] Re: Removing erroneous rain readings from the database
Thanks for the pointers everyone. The datetime function comes in handy, and I appreciate the schema and other references. I have cleared out the bad data, although I followed an example and set the value to NULL, perhaps I should have gone with 0. Now when I try to find the NULL values (where rain=NULL) I do not get any hits, but they show up in other queries. I was thinking about setting them to 0. Most of the reports look better now. The annual report is still showing a peak rain rate for the data I deleted, I'm not sure where that comes from. I deleted the daily summaries and they have been recreated. Its raining here in Southern California, second set of storms for the season. I don't get much opportunity to test the rain collector but it appears to be working. Dana On Sunday, December 22, 2019 at 1:51:41 PM UTC-8, Dana Roode wrote: > > Hello, > > I've had some issues with the rain collector on my Vantage Pro2 this year, > I replaced it a few weeks back. There were some spurious high readings, > plus my testing generated false readings. The data indicates over 60 > inches of rain in my desert location this year. I'd like to find the bad > values and zero them out, and I see how to do this with sqlite3 on the > database if I understood the data fields. "rain" is the field in the > archive tables, but I don't know what time period thats for. The dateTime > field has numbers like 1539650100 and I don't know how to convert to > something meanful. > > Is there documentation on the database schema and fields somewhere? I > haven't found it in the wiki information. > > Any pointers would be appreciated. > > Dana Roode > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/dd10c094-3958-4dfb-8134-67de37cd432b%40googlegroups.com.
Re: [weewx-user] Removing erroneous rain readings from the database
Thanks Greg! I am running on Unix, so this is very helpful. Dana On Sunday, December 22, 2019 at 1:55:16 PM UTC-8, Greg Troxel wrote: > > Dana Roode > writes: > > > Hello, > > > > I've had some issues with the rain collector on my Vantage Pro2 this > year, > > I replaced it a few weeks back. There were some spurious high readings, > > plus my testing generated false readings. The data indicates over 60 > > inches of rain in my desert location this year. I'd like to find the > bad > > values and zero them out, and I see how to do this with sqlite3 on the > > database if I understood the data fields. "rain" is the field in the > > archive tables, but I don't know what time period thats for. The > dateTime > > field has numbers like 1539650100 and I don't know how to convert to > > something meanful. > > I would guess that rain is either the rain that arrived in that archive > interval, or some sort of cumulative value. > > The dateTime is in seconds since the Unix epoch, which was > 1970-01-01T00:00:00Z (Jan 1, 1970, midnight UTC). > > On Unix, date -r will take that and convert it to human time. This is > just what is does normally, except it uses the provided value vs > gettimeofday(). > > $ date -r 1539650100 > Mon Oct 15 20:35:00 EDT 2018 > > which isn't a few weeks ago. > > On windows, you could install Unix and then run date -r. Probably > there's an easier way... > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6212715b-9662-4bcc-bf02-4a657d909a1f%40googlegroups.com.
[weewx-user] Removing erroneous rain readings from the database
Hello, I've had some issues with the rain collector on my Vantage Pro2 this year, I replaced it a few weeks back. There were some spurious high readings, plus my testing generated false readings. The data indicates over 60 inches of rain in my desert location this year. I'd like to find the bad values and zero them out, and I see how to do this with sqlite3 on the database if I understood the data fields. "rain" is the field in the archive tables, but I don't know what time period thats for. The dateTime field has numbers like 1539650100 and I don't know how to convert to something meanful. Is there documentation on the database schema and fields somewhere? I haven't found it in the wiki information. Any pointers would be appreciated. Dana Roode -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f853d32c-80ce-4114-80a3-7313eb32234a%40googlegroups.com.
Re: [weewx-user] adding records from one weewx database to another?
Ok, thanks. Yes, I'm using 5 minute intervals. So, all the data the
logger has is already in the archive. This is good, as the wee_device
--dump didn't work.
Dana
/var/lib/weewx# wee_device --dump
Using configuration file /etc/weewx/weewx.conf
Using Vantage driver version 3.1.1 (weewx.drivers.vantage)
Proceeding will dump all data in the logger.
Are you sure you want to proceed (y/n)? y
Starting dump ...
Traceback (most recent call last):
File "/usr/bin/wee_device", line 66, in
main()
File "/usr/bin/wee_device", line 62, in main
device.configure(config_dict)
File "/usr/share/weewx/weewx/drivers/__init__.py", line 69, in configure
self.do_options(options, parser, config_dict, prompt)
File "/usr/share/weewx/weewx/drivers/vantage.py", line 2018, in do_options
self.dump_logger(station, config_dict, options.noprompt)
File "/usr/share/weewx/weewx/drivers/vantage.py", line 2618, in
dump_logger
for record in converted_generator:
File "/usr/share/weewx/weewx/units.py", line 1247, in next
_record = self.input_generator.next()
File "/usr/share/weewx/weewx/drivers/vantage.py", line 660, in
genArchiveDump
_page = self.port.get_data_with_crc16(267, prompt=_ack,
max_tries=self.max_tries)
File "/usr/share/weewx/weewx/drivers/vantage.py", line 207, in
get_data_with_crc16
raise weewx.CRCError("Unable to pass CRC16 check while getting data")
weewx.CRCError: Unable to pass CRC16 check while getting data
On Wed, Sep 25, 2019 at 1:42 PM vince wrote:
> On Wednesday, September 25, 2019 at 12:52:15 PM UTC-7, Dana Roode wrote:
>>
>> I see that weewx has been able to go back a ways and get most of the
>> missing data, but I don't know what triggers that and whether or not there
>> is any more data in the weather station that has not already been restored.
>>
>>
>>
> Weewx will grab whatever data iit hasn't previously put into your database
> from your Davis datalogger.If you're running the typical 5-minute
> archive period, that's about 8 or so days of storage. If your archive
> period is quicker/longer than that, you'd need to do the math for how long
> you can be down before the datalogger starts deleting oldest-first to save
> the most current N days (actually N readings) of data.
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/t52g84-JiOo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/98c822c4-1b93-435f-89c6-cec68182a433%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/98c822c4-1b93-435f-89c6-cec68182a433%40googlegroups.com?utm_medium=email_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAEhu_sxE57XBfi-n2W660ObQ-e-DPWv_S5WDA22NFUDXZS_ivA%40mail.gmail.com.
Re: [weewx-user] adding records from one weewx database to another?
Thanks everyone, I appreciate the info. I may dump the databases with sqlite3 and verify what I have and what I'm missing. I took another look this morning and I may only be missing the days when the system was down after the initial failure. The only way to get that data back would be if the Vantage weather station had it. I see that weewx has been able to go back a ways and get most of the missing data, but I don't know what triggers that and whether or not there is any more data in the weather station that has not already been restored. I see a "dump" option in wee_device to move all the data in the weather station to the archive, perhaps that would do the trick. Its not overly relevant at this point, but since I confused folks with my post, let me explain again "for the record" my recent hardware/system-building fun: - Production system in place 12/2018->9/9/2019, died on 9/9 - Got to site on 9/12, the production system would not power up. Installed the old system I had replaced in 12/2018; it had an old weewx.sdb on it (referred to as "interim" in my original post) - Later in the month, I moved the production disk into "replacement system 1;" it had the production weewx.sdb file. Took system to the site on 9/22 and it ran for a few hours. But it had troubles mounting sysroot, making me suspicious of the disk drive. I reverted to the interim system. - Built "replacement system 2" (new computer and disk) with the former production weewx.sdb on it (with data from 12/2018 through 9/9/2019 and some that was evidently added by "replacement system 1" on 9/22) - On 9/24 took "replacement system 2" to site and brought it up; message in the log indicated weewx added records to weewx.sdb that came from 9/22 until present I believe most of the other data was placed in the production database during the short period using "replacement system 1" on 9/22. So, this is why I am not missing as much data as I thought. I am only missing data from 9/9 until 9/12 when no system was up. Weewx is pretty smart! Sorry for confusing everyone. I hope my luck in hardware improves and the new system stays up for awhile. Dana On Wednesday, September 25, 2019 at 8:43:01 AM UTC-7, vince wrote: > > On Wednesday, September 25, 2019 at 4:41:54 AM UTC-7, Greg Troxel wrote: >> >> Dana Roode writes: >> >> > Is there a way to extract archive records from one weewx.sdb file and >> > import then into another? I see there is a utility that allows >> importing >> > of CSV data files. >> > >> > >> > > Yes - there are many ways. > > Simplest is probably to do a sqlite3 dump of all the database archive > tables, then restore from 'each' of them into your aggregate database. > It'll whine when it inevitably finds duplicate records, but nothing bad > will happen. The result should be the union of all the databases. Weewx > should handle the summary tables, but you could always rebuild them > manually too if you wanted. > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/3d62be2e-a902-465f-ab36-a67129b6b7ae%40googlegroups.com.
[weewx-user] adding records from one weewx database to another?
Is there a way to extract archive records from one weewx.sdb file and import then into another? I see there is a utility that allows importing of CSV data files. My linux system died on September 9th and I did not have time to build a new one right away (the system and weather station is located 80 miles away from me). I did have the old one from about 10 months ago, and I put that in line. It received a couple of weeks of records from the Davis Vantage Pro2 weather station. I setup the new system with the database I had through September 9th and now that database is the active one. I'd like to move a few weeks of data from the interim system to the active one. I'm running the latest weewx, now using Ubuntu rather than Centos. It went very well, other than a bug I previously reported (fixed with a vantage.py change). I will post about that later. Dana -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7370f9a4-2c9c-496e-b198-c98dd6cf5007%40googlegroups.com.
[weewx-user] Re: Centos Install Questions - Davis Vantage Pro2
Yes; I used the weewx config program and the configuration file is updated fine. I found another post about this problem and made this edit to vantage.py - adjusting "for_loopPacket in self.genDavisLoopPackets(200)" from 200 to 2. I did that and it all started working now. Dana On Monday, February 19, 2018 at 6:21:08 PM UTC-8, Greg from Oz wrote: > > Did you edit /etc/weewx/weewx.conf as that is the file that has the > information about your station name etc? > > On Tuesday, 20 February 2018 12:05:13 UTC+11, Dana Roode wrote: >> >> Hello, >> >> Just installed Weewx on Centos 7.4 with a Davis Vantage Pro2; install >> seemed to go well. Two questions: >> >> a) After configuration, entering wx station type, location, etc, I was >> expecting the web page that had been installed at /weewx/ to be updated to >> reflect my information rather than the initial "Hood River Oregon" >> simulator page. Is there something I need to do to have the information >> reflected in the configuration file shown on this web page? Perhaps its a >> manual thing. >> >> b) When I run weewx as a daemon (/etc/init.d/weewx start) I get the >> messages about Wunderground etc posting not enabled, then it starts the >> main packet loop. It tries several times but gets an error "expected to >> read 99 chars; got 0 instead" and then pauses for 60 seconds. When I run >> weewxd in the shell, every few seconds a long record preceded by LOOP: is >> displayed with all the data from my weather station. It would seem that >> weewx can access the data when I run interactively but not as a daemon? >> Does that make sense? >> >> Thanks for any pointers and I will keep reading the documentation >> meanwhile. >> >> Dana Roode >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[weewx-user] Centos Install Questions - Davis Vantage Pro2
Hello, Just installed Weewx on Centos 7.4 with a Davis Vantage Pro2; install seemed to go well. Two questions: a) After configuration, entering wx station type, location, etc, I was expecting the web page that had been installed at /weewx/ to be updated to reflect my information rather than the initial "Hood River Oregon" simulator page. Is there something I need to do to have the information reflected in the configuration file shown on this web page? Perhaps its a manual thing. b) When I run weewx as a daemon (/etc/init.d/weewx start) I get the messages about Wunderground etc posting not enabled, then it starts the main packet loop. It tries several times but gets an error "expected to read 99 chars; got 0 instead" and then pauses for 60 seconds. When I run weewxd in the shell, every few seconds a long record preceded by LOOP: is displayed with all the data from my weather station. It would seem that weewx can access the data when I run interactively but not as a daemon? Does that make sense? Thanks for any pointers and I will keep reading the documentation meanwhile. Dana Roode -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
