Solved thanks. More stupid me I think – a clear case of RTFM!
Sent from Mail for Windows 10
From: Tom Keffer
Sent: 20 August 2020 14:07
To: David Marshall
Cc: weewx-user@googlegroups.com
Subject: Re: [weewx-user] UnicodeDecodeError in database
Stupid me. The answer is in how you are invoking wee_database. Any positional
argument is interpreted as the path to weewx.conf, so wee_database is thinking
that /home/weewx/archive/weewx.sdb is your configuration file, weewx.conf.
Naturally, it being a sqlite database, it has all sorts of unusual byte
sequences.
wee_database finds your database from the information in the configuration
file, not from the command line. This way, it can work on MySQL databases,
which do not use a single file. So, you want
wee_database --reconfigure /home/weewx/weewx.conf
On Thu, Aug 20, 2020 at 4:43 AM David Marshall <marshalld...@gmail.com> wrote:
1. Yes
2. Seemed to work ok
3. Syslog attached – but seems to me to be ok.
The problem only occurs when using wee_database -interestingly the offending
byte has moved from position 26 to 31!
pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
File "/home/weewx/bin/wee_database", line 974, in <module>
main()
File "/home/weewx/bin/wee_database", line 128, in main
config_path, config_dict = weecfg.read_config(options.config_path, args)
File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
default_encoding='utf-8')
File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
self._load(infile, configspec)
File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
content = self._handle_bom(content)
File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
return self._decode(infile, self.encoding)
File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 31:
invalid continuation byte
Sent from Mail for Windows 10
From: Tom Keffer
Sent: 20 August 2020 00:51
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database
I don't have any problem with either copy of weewx.conf that you sent to me.
Things to try:
1. This is /home/weewx/weewx.conf, right?
2. Try reading it directly:
python -c "import configobj; c=configobj.ConfigObj('/home/weewx/weewx.conf')"
3. Set debug=1 in weewx.conf, then try running weewxd, then send us the log.
-tk
On Wed, Aug 19, 2020 at 7:19 AM David Marshall <marshalld...@gmail.com> wrote:
Hi Tom
1. Yes wee_database
2. Yes same path different copies
3. No edits – in fact I tried not to change anything in the final install apart
from setting it to simulator.
I attach the last version of weewx.conf, which is still producing the error.
David
Sent from Mail for Windows 10
From: Tom Keffer
Sent: 19 August 2020 15:59
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database
1. All these attempts were with running wee_database? Or, something else?
2. I assume all these different attempts used the same path, namely
/home/weewx/weewx.conf. And, we're sure they used different copies (albeit, at
the same path location)?
3. It's hard to explain how fresh, untouched versions of weewx.conf are all
getting the same error. Are you editing these copies before running?
Can you send me the offending weewx.conf to my personal email? tkeffer at
gmail.com
-tk
On Wed, Aug 19, 2020 at 6:35 AM David Marshall <marshalld...@gmail.com> wrote:
Thanks Tom
Tried your code to show the offending line but nothing appears. Then tried a
clean weewx.conf from the install folder but still the same error message. Then
tried a full reinstall, but still the same error.
Then tried it with the the default database using simulator and still the same
error - so you are right it is not my database. What next?
On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
The decode error is actually in your configuration file, weewx.conf, and not in
the database.
Somehow, your copy of weewx.conf got corrupted. One of the lines has a
character sequence that starts out as a utf-8 encoding, but the following byte
is not what was expected.
Unfortunately, the error does not tell you which line the bad character
sequence is in, only that it is in byte position 26 of that line. Perhaps this
will help (NOT TESTED):
od -x -A d weewx.conf | grep e0
This should print out the line and offending sequence.
-tk
On Wed, Aug 19, 2020 at 5:25 AM David Marshall <marsha...@gmail.com> wrote:
Hi
Installed a fresh version of weewx 4.1.1 without problem using setup.py on a
raspberry.
Transferred my existing sqlite database from weewx 3.9.1. It is big, about 3
years of data. This worked fine.
But when I tried to use reconfigure I got the following error
pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
File "/home/weewx/bin/wee_database", line 974, in <module>
main()
File "/home/weewx/bin/wee_database", line 128, in main
config_path, config_dict = weecfg.read_config(options.config_path, args)
File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
default_encoding='utf-8')
File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
self._load(infile, configspec)
File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
content = self._handle_bom(content)
File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
return self._decode(infile, self.encoding)
File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26:
invalid continuation byte
Anyone know how to fix this?
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zEDyDz5xh8gtJRyeCKxmCbJGuAYqChLJeq4d%2BGQrD3vUFA%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/5f3d34eb.1c69fb81.4c509.d9cd%40mx.google.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zEB99R%3D5MrZB6EYfi1MOPcW96E06bPs3ayV5FAPF_aK7NA%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKFYrufMVdf%3DL3YKuy2uXp%2B5sXdQeD77DV7LEWXB6qrg%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/5f3e7362.1c69fb81.54f8f.a1dc%40mx.google.com.
