RE: [weewx-user] UnicodeDecodeError in database

[David Marshall]

Solved thanks. More stupid me I think – a clear case of RTFM!

Sent from Mail for Windows 10

From: Tom Keffer
Sent: 20 August 2020 14:07
To: David Marshall
Cc: weewx-user@googlegroups.com
Subject: Re: [weewx-user] UnicodeDecodeError in database

Stupid me. The answer is in how you are invoking wee_database. Any positional 
argument is interpreted as the path to weewx.conf, so wee_database is thinking 
that /home/weewx/archive/weewx.sdb is your configuration file, weewx.conf. 
Naturally, it being a sqlite database, it has all sorts of unusual byte 
sequences.

wee_database finds your database from the information in the configuration 
file, not from the command line. This way, it can work on MySQL databases, 
which do not use a single file. So, you want

wee_database --reconfigure /home/weewx/weewx.conf



On Thu, Aug 20, 2020 at 4:43 AM David Marshall  wrote:
1. Yes
2. Seemed to work ok
3. Syslog attached – but seems to me to be ok.
 
The problem only occurs when using wee_database -interestingly the offending 
byte has moved from position 26 to 31!
 
pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
  File "/home/weewx/bin/wee_database", line 974, in 
    main()
  File "/home/weewx/bin/wee_database", line 128, in main
    config_path, config_dict = weecfg.read_config(options.config_path, args)
  File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
    default_encoding='utf-8')
  File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
    self._load(infile, configspec)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
    content = self._handle_bom(content)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
    return self._decode(infile, self.encoding)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
    infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 31: 
invalid continuation byte
 
Sent from Mail for Windows 10
 
From: Tom Keffer
Sent: 20 August 2020 00:51
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database
 
I don't have any problem with either copy of weewx.conf that you sent to me. 
Things to try:
 
1. This is /home/weewx/weewx.conf, right?  
 
2. Try reading it directly:
 
python -c "import configobj; c=configobj.ConfigObj('/home/weewx/weewx.conf')"
 
3. Set debug=1 in weewx.conf, then try running weewxd, then send us the log. 
 
-tk
 
 
 
On Wed, Aug 19, 2020 at 7:19 AM David Marshall  wrote:
Hi Tom
 
1. Yes wee_database
2. Yes same path different  copies
3. No edits – in fact I tried not to change anything in the final install apart 
from setting it to simulator.
 
I attach the last version of weewx.conf, which is still producing the error.
 
David
 
Sent from Mail for Windows 10
 
From: Tom Keffer
Sent: 19 August 2020 15:59
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database
 
1. All these attempts were with running wee_database? Or, something else?
 
2. I assume all these different attempts used the same path, namely 
/home/weewx/weewx.conf. And, we're sure they used different copies (albeit, at 
the same path location)? 
 
3. It's hard to explain how fresh, untouched versions of weewx.conf are all 
getting the same error. Are you editing these copies before running?
 
Can you send me the offending weewx.conf to my personal email? tkeffer at 
gmail.com
 
-tk
 
On Wed, Aug 19, 2020 at 6:35 AM David Marshall  wrote:
Thanks Tom
 
Tried your code to show the offending line but nothing appears. Then tried a 
clean weewx.conf from the install folder but still the same error message. Then 
tried a full reinstall, but still the same error.
Then tried it with the the default database using simulator and still the same 
error - so you are right it is not my database. What next?

On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
The decode error is actually in your configuration file, weewx.conf, and not in 
the database.
 
Somehow, your copy of weewx.conf got corrupted. One of the lines has a 
character sequence that starts out as a utf-8 encoding, but the following byte 
is not what was expected. 
 
Unfortunately, the error does not tell you which line the bad character 
sequence is in, only that it is in byte position 26 of that line. Perhaps this 
will help (NOT TESTED):
 
od -x -A d weewx.conf | grep e0
 
This should print out the line and offending sequence.
 
-tk
 
 
 
On Wed, Aug 19, 2020 at 5:25 AM David Marshall  wrote:
Hi
 
Installed a fresh version of weewx 4.1.1 without problem using setup.py on a 
raspberry.
 
Transferred my existing sqlite database from weewx 3.9.1. It is big, about 3 
years of data. This worked fine.
 
But when I tried to use reconfigure I got the following error
 
pi@raspberrypi:~ $ /home/wee

Re: [weewx-user] UnicodeDecodeError in database

[Tom Keffer]

Stupid me. The answer is in how you are invoking wee_database. Any
positional argument is interpreted as the path to weewx.conf, so
wee_database is thinking that /home/weewx/archive/weewx.sdb is your
configuration file, weewx.conf. Naturally, it being a sqlite database, it
has all sorts of unusual byte sequences.

wee_database finds your database from the information in the configuration
file, not from the command line. This way, it can work on MySQL databases,
which do not use a single file. So, you want

*wee_database --reconfigure /home/weewx/weewx.conf*



On Thu, Aug 20, 2020 at 4:43 AM David Marshall 
wrote:

>
>1. Yes
>2. Seemed to work ok
>3. Syslog attached – but seems to me to be ok.
>
>
>
> The problem only occurs when using wee_database -interestingly the
> offending byte has moved from position 26 to 31!
>
>
>
> pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
> /home/weewx/archive/weewx.sdb
>
> Traceback (most recent call last):
>
>   File "/home/weewx/bin/wee_database", line 974, in 
>
> main()
>
>   File "/home/weewx/bin/wee_database", line 128, in main
>
> config_path, config_dict = weecfg.read_config(options.config_path,
> args)
>
>   File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
>
> default_encoding='utf-8')
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in
> __init__
>
> self._load(infile, configspec)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
>
> content = self._handle_bom(content)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in
> _handle_bom
>
> return self._decode(infile, self.encoding)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
>
> infile[i] = line.decode(encoding)
>
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 31:
> invalid continuation byte
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Tom Keffer 
> *Sent: *20 August 2020 00:51
> *To: *weewx-user 
> *Subject: *Re: [weewx-user] UnicodeDecodeError in database
>
>
>
> I don't have any problem with either copy of weewx.conf that you sent to
> me. Things to try:
>
>
>
> 1. This is /home/weewx/weewx.conf, right?
>
>
>
> 2. Try reading it directly:
>
>
>
> *python -c "import configobj;
> c=configobj.ConfigObj('/home/weewx/weewx.conf')"*
>
>
>
> 3. Set debug=1 in weewx.conf, then try running weewxd, then send us the
> log.
>
>
>
> -tk
>
>
>
>
>
>
>
> On Wed, Aug 19, 2020 at 7:19 AM David Marshall 
> wrote:
>
> Hi Tom
>
>
>
>1. Yes wee_database
>2. Yes same path different  copies
>    3. No edits – in fact I tried not to change anything in the final
>install apart from setting it to simulator.
>
>
>
> I attach the last version of weewx.conf, which is still producing the
> error.
>
>
>
> David
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Tom Keffer 
> *Sent: *19 August 2020 15:59
> *To: *weewx-user 
> *Subject: *Re: [weewx-user] UnicodeDecodeError in database
>
>
>
> 1. All these attempts were with running wee_database? Or, something else?
>
>
>
> 2. I assume all these different attempts used the same path, namely
> /home/weewx/weewx.conf. And, we're sure they used different copies (albeit,
> at the same path location)?
>
>
>
> 3. It's hard to explain how fresh, untouched versions of weewx.conf are
> all getting the same error. Are you editing these copies before running?
>
>
>
> Can you send me the offending weewx.conf to my personal email? tkeffer at
> gmail.com
>
>
>
> -tk
>
>
>
> On Wed, Aug 19, 2020 at 6:35 AM David Marshall 
> wrote:
>
> Thanks Tom
>
>
>
> Tried your code to show the offending line but nothing appears. Then tried
> a clean weewx.conf from the install folder but still the same error
> message. Then tried a full reinstall, but still the same error.
>
> Then tried it with the the default database using simulator and still the
> same error - so you are right it is not my database. What next?
>
> On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
>
> The decode error is actually in your configuration file, weewx.conf, and
> not in the database.
>
>
>
> Somehow, your copy of weewx.conf got corrupted. One of the lines has a
> character sequence th

RE: [weewx-user] UnicodeDecodeError in database

[David Marshall]

1. Yes
2. Seemed to work ok
3. Syslog attached – but seems to me to be ok.

The problem only occurs when using wee_database -interestingly the offending 
byte has moved from position 26 to 31!

pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
  File "/home/weewx/bin/wee_database", line 974, in 
main()
  File "/home/weewx/bin/wee_database", line 128, in main
config_path, config_dict = weecfg.read_config(options.config_path, args)
  File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
default_encoding='utf-8')
  File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
self._load(infile, configspec)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
content = self._handle_bom(content)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
return self._decode(infile, self.encoding)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 31: 
invalid continuation byte

Sent from Mail for Windows 10

From: Tom Keffer
Sent: 20 August 2020 00:51
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database

I don't have any problem with either copy of weewx.conf that you sent to me. 
Things to try:

1. This is /home/weewx/weewx.conf, right?  

2. Try reading it directly:

python -c "import configobj; c=configobj.ConfigObj('/home/weewx/weewx.conf')"

3. Set debug=1 in weewx.conf, then try running weewxd, then send us the log. 

-tk



On Wed, Aug 19, 2020 at 7:19 AM David Marshall  wrote:
Hi Tom
 
1. Yes wee_database
2. Yes same path different  copies
3. No edits – in fact I tried not to change anything in the final install apart 
from setting it to simulator.
 
I attach the last version of weewx.conf, which is still producing the error.
 
David
 
Sent from Mail for Windows 10
 
From: Tom Keffer
Sent: 19 August 2020 15:59
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database
 
1. All these attempts were with running wee_database? Or, something else?
 
2. I assume all these different attempts used the same path, namely 
/home/weewx/weewx.conf. And, we're sure they used different copies (albeit, at 
the same path location)? 
 
3. It's hard to explain how fresh, untouched versions of weewx.conf are all 
getting the same error. Are you editing these copies before running?
 
Can you send me the offending weewx.conf to my personal email? tkeffer at 
gmail.com
 
-tk
 
On Wed, Aug 19, 2020 at 6:35 AM David Marshall  wrote:
Thanks Tom
 
Tried your code to show the offending line but nothing appears. Then tried a 
clean weewx.conf from the install folder but still the same error message. Then 
tried a full reinstall, but still the same error.
Then tried it with the the default database using simulator and still the same 
error - so you are right it is not my database. What next?

On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
The decode error is actually in your configuration file, weewx.conf, and not in 
the database.
 
Somehow, your copy of weewx.conf got corrupted. One of the lines has a 
character sequence that starts out as a utf-8 encoding, but the following byte 
is not what was expected. 
 
Unfortunately, the error does not tell you which line the bad character 
sequence is in, only that it is in byte position 26 of that line. Perhaps this 
will help (NOT TESTED):
 
od -x -A d weewx.conf | grep e0
 
This should print out the line and offending sequence.
 
-tk
 
 
 
On Wed, Aug 19, 2020 at 5:25 AM David Marshall  wrote:
Hi
 
Installed a fresh version of weewx 4.1.1 without problem using setup.py on a 
raspberry.
 
Transferred my existing sqlite database from weewx 3.9.1. It is big, about 3 
years of data. This worked fine.
 
But when I tried to use reconfigure I got the following error
 
pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
  File "/home/weewx/bin/wee_database", line 974, in 
    main()
  File "/home/weewx/bin/wee_database", line 128, in main
    config_path, config_dict = weecfg.read_config(options.config_path, args)
  File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
    default_encoding='utf-8')
  File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
    self._load(infile, configspec)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
    content = self._handle_bom(content)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
    return self._decode(infile, self.encoding)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1517

Re: [weewx-user] UnicodeDecodeError in database

[Tom Keffer]

I don't have any problem with either copy of weewx.conf that you sent to
me. Things to try:

1. This is /home/weewx/weewx.conf, right?

2. Try reading it directly:

*python -c "import configobj;
c=configobj.ConfigObj('/home/weewx/weewx.conf')"*

3. Set debug=1 in weewx.conf, then try running weewxd, then send us the
log.

-tk



On Wed, Aug 19, 2020 at 7:19 AM David Marshall 
wrote:

> Hi Tom
>
>
>
>1. Yes wee_database
>2. Yes same path different  copies
>3. No edits – in fact I tried not to change anything in the final
>install apart from setting it to simulator.
>
>
>
> I attach the last version of weewx.conf, which is still producing the
> error.
>
>
>
> David
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Tom Keffer 
> *Sent: *19 August 2020 15:59
> *To: *weewx-user 
> *Subject: *Re: [weewx-user] UnicodeDecodeError in database
>
>
>
> 1. All these attempts were with running wee_database? Or, something else?
>
>
>
> 2. I assume all these different attempts used the same path, namely
> /home/weewx/weewx.conf. And, we're sure they used different copies (albeit,
> at the same path location)?
>
>
>
> 3. It's hard to explain how fresh, untouched versions of weewx.conf are
> all getting the same error. Are you editing these copies before running?
>
>
>
> Can you send me the offending weewx.conf to my personal email? tkeffer at
> gmail.com
>
>
>
> -tk
>
>
>
> On Wed, Aug 19, 2020 at 6:35 AM David Marshall 
> wrote:
>
> Thanks Tom
>
>
>
> Tried your code to show the offending line but nothing appears. Then tried
> a clean weewx.conf from the install folder but still the same error
> message. Then tried a full reinstall, but still the same error.
>
> Then tried it with the the default database using simulator and still the
> same error - so you are right it is not my database. What next?
>
> On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
>
> The decode error is actually in your configuration file, weewx.conf, and
> not in the database.
>
>
>
> Somehow, your copy of weewx.conf got corrupted. One of the lines has a
> character sequence that starts out as a utf-8 encoding, but the following
> byte is not what was expected.
>
>
>
> Unfortunately, the error does not tell you which line the bad character
> sequence is in, only that it is in byte position 26 of that line. Perhaps
> this will help (NOT TESTED):
>
>
>
> *od -x -A d weewx.conf | grep e0*
>
>
>
> This should print out the line and offending sequence.
>
>
>
> -tk
>
>
>
>
>
>
>
> On Wed, Aug 19, 2020 at 5:25 AM David Marshall 
> wrote:
>
> Hi
>
>
>
> Installed a fresh version of weewx 4.1.1 without problem using setup.py on
> a raspberry.
>
>
>
> Transferred my existing sqlite database from weewx 3.9.1. It is big, about
> 3 years of data. This worked fine.
>
>
>
> But when I tried to use reconfigure I got the following error
>
>
>
> pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
> /home/weewx/archive/weewx.sdb
>
> Traceback (most recent call last):
>
>   File "/home/weewx/bin/wee_database", line 974, in 
>
> main()
>
>   File "/home/weewx/bin/wee_database", line 128, in main
>
> config_path, config_dict = weecfg.read_config(options.config_path,
> args)
>
>   File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
>
> default_encoding='utf-8')
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in
> __init__
>
> self._load(infile, configspec)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
>
> content = self._handle_bom(content)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in
> _handle_bom
>
> return self._decode(infile, self.encoding)
>
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
>
> infile[i] = line.decode(encoding)
>
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26:
> invalid continuation byte
>
>
>
> Anyone know how to fix this?
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/we

RE: [weewx-user] UnicodeDecodeError in database

[David Marshall]

Hi Tom

1. Yes wee_database
2. Yes same path different  copies
3. No edits – in fact I tried not to change anything in the final install apart 
from setting it to simulator.

I attach the last version of weewx.conf, which is still producing the error.

David

Sent from Mail for Windows 10

From: Tom Keffer
Sent: 19 August 2020 15:59
To: weewx-user
Subject: Re: [weewx-user] UnicodeDecodeError in database

1. All these attempts were with running wee_database? Or, something else?

2. I assume all these different attempts used the same path, namely 
/home/weewx/weewx.conf. And, we're sure they used different copies (albeit, at 
the same path location)? 

3. It's hard to explain how fresh, untouched versions of weewx.conf are all 
getting the same error. Are you editing these copies before running?

Can you send me the offending weewx.conf to my personal email? tkeffer at 
gmail.com

-tk

On Wed, Aug 19, 2020 at 6:35 AM David Marshall  wrote:
Thanks Tom

Tried your code to show the offending line but nothing appears. Then tried a 
clean weewx.conf from the install folder but still the same error message. Then 
tried a full reinstall, but still the same error.
Then tried it with the the default database using simulator and still the same 
error - so you are right it is not my database. What next?

On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
The decode error is actually in your configuration file, weewx.conf, and not in 
the database.

Somehow, your copy of weewx.conf got corrupted. One of the lines has a 
character sequence that starts out as a utf-8 encoding, but the following byte 
is not what was expected. 

Unfortunately, the error does not tell you which line the bad character 
sequence is in, only that it is in byte position 26 of that line. Perhaps this 
will help (NOT TESTED):

od -x -A d weewx.conf | grep e0

This should print out the line and offending sequence.

-tk



On Wed, Aug 19, 2020 at 5:25 AM David Marshall  wrote:
Hi

Installed a fresh version of weewx 4.1.1 without problem using setup.py on a 
raspberry.

Transferred my existing sqlite database from weewx 3.9.1. It is big, about 3 
years of data. This worked fine.

But when I tried to use reconfigure I got the following error

pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
  File "/home/weewx/bin/wee_database", line 974, in 
    main()
  File "/home/weewx/bin/wee_database", line 128, in main
    config_path, config_dict = weecfg.read_config(options.config_path, args)
  File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
    default_encoding='utf-8')
  File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
    self._load(infile, configspec)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
    content = self._handle_bom(content)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in _handle_bom
    return self._decode(infile, self.encoding)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
    infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26: 
invalid continuation byte

Anyone know how to fix this?




-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com.
-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com.
-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEDyDz5xh8gtJRyeCKxmCbJGuAYqChLJeq4d%2BGQrD3vUFA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/5f3d34eb.1c69fb81.4c509.d9cd%40mx.google.com.


weewx.conf
Description: Binary data

Re: [weewx-user] UnicodeDecodeError in database

[Tom Keffer]

1. All these attempts were with running wee_database? Or, something else?

2. I assume all these different attempts used the same path, namely
/home/weewx/weewx.conf. And, we're sure they used different copies (albeit,
at the same path location)?

3. It's hard to explain how fresh, untouched versions of weewx.conf are all
getting the same error. Are you editing these copies before running?

Can you send me the offending weewx.conf to my personal email? tkeffer at
gmail.com

-tk

On Wed, Aug 19, 2020 at 6:35 AM David Marshall 
wrote:

> Thanks Tom
>
> Tried your code to show the offending line but nothing appears. Then tried
> a clean weewx.conf from the install folder but still the same error
> message. Then tried a full reinstall, but still the same error.
> Then tried it with the the default database using simulator and still the
> same error - so you are right it is not my database. What next?
>
> On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
>>
>> The decode error is actually in your configuration file, weewx.conf, and
>> not in the database.
>>
>> Somehow, your copy of weewx.conf got corrupted. One of the lines has a
>> character sequence that starts out as a utf-8 encoding, but the following
>> byte is not what was expected.
>>
>> Unfortunately, the error does not tell you which line the bad character
>> sequence is in, only that it is in byte position 26 of that line. Perhaps
>> this will help (NOT TESTED):
>>
>> *od -x -A d weewx.conf | grep e0*
>>
>> This should print out the line and offending sequence.
>>
>> -tk
>>
>>
>>
>> On Wed, Aug 19, 2020 at 5:25 AM David Marshall 
>> wrote:
>>
>>> Hi
>>>
>>> Installed a fresh version of weewx 4.1.1 without problem using setup.py
>>> on a raspberry.
>>>
>>> Transferred my existing sqlite database from weewx 3.9.1. It is big,
>>> about 3 years of data. This worked fine.
>>>
>>> But when I tried to use reconfigure I got the following error
>>>
>>> pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
>>> /home/weewx/archive/weewx.sdb
>>> Traceback (most recent call last):
>>>   File "/home/weewx/bin/wee_database", line 974, in 
>>> main()
>>>   File "/home/weewx/bin/wee_database", line 128, in main
>>> config_path, config_dict = weecfg.read_config(options.config_path,
>>> args)
>>>   File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
>>> default_encoding='utf-8')
>>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in
>>> __init__
>>> self._load(infile, configspec)
>>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
>>> content = self._handle_bom(content)
>>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in
>>> _handle_bom
>>> return self._decode(infile, self.encoding)
>>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in
>>> _decode
>>> infile[i] = line.decode(encoding)
>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26:
>>> invalid continuation byte
>>>
>>> Anyone know how to fix this?
>>>
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "weewx-user" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to weewx...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEDyDz5xh8gtJRyeCKxmCbJGuAYqChLJeq4d%2BGQrD3vUFA%40mail.gmail.com.

Re: [weewx-user] UnicodeDecodeError in database

[David Marshall]

Thanks Tom

Tried your code to show the offending line but nothing appears. Then tried 
a clean weewx.conf from the install folder but still the same error 
message. Then tried a full reinstall, but still the same error.
Then tried it with the the default database using simulator and still the 
same error - so you are right it is not my database. What next?

On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote:
>
> The decode error is actually in your configuration file, weewx.conf, and 
> not in the database.
>
> Somehow, your copy of weewx.conf got corrupted. One of the lines has a 
> character sequence that starts out as a utf-8 encoding, but the following 
> byte is not what was expected. 
>
> Unfortunately, the error does not tell you which line the bad character 
> sequence is in, only that it is in byte position 26 of that line. Perhaps 
> this will help (NOT TESTED):
>
> *od -x -A d weewx.conf | grep e0*
>
> This should print out the line and offending sequence.
>
> -tk
>
>
>
> On Wed, Aug 19, 2020 at 5:25 AM David Marshall  > wrote:
>
>> Hi
>>
>> Installed a fresh version of weewx 4.1.1 without problem using setup.py 
>> on a raspberry.
>>
>> Transferred my existing sqlite database from weewx 3.9.1. It is big, 
>> about 3 years of data. This worked fine.
>>
>> But when I tried to use reconfigure I got the following error
>>
>> pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
>> /home/weewx/archive/weewx.sdb
>> Traceback (most recent call last):
>>   File "/home/weewx/bin/wee_database", line 974, in 
>> main()
>>   File "/home/weewx/bin/wee_database", line 128, in main
>> config_path, config_dict = weecfg.read_config(options.config_path, 
>> args)
>>   File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
>> default_encoding='utf-8')
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in 
>> __init__
>> self._load(infile, configspec)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
>> content = self._handle_bom(content)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in 
>> _handle_bom
>> return self._decode(infile, self.encoding)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in 
>> _decode
>> infile[i] = line.decode(encoding)
>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26: 
>> invalid continuation byte
>>
>> Anyone know how to fix this?
>>
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com.

Re: [weewx-user] UnicodeDecodeError in database

[Tom Keffer]

The decode error is actually in your configuration file, weewx.conf, and
not in the database.

Somehow, your copy of weewx.conf got corrupted. One of the lines has a
character sequence that starts out as a utf-8 encoding, but the following
byte is not what was expected.

Unfortunately, the error does not tell you which line the bad character
sequence is in, only that it is in byte position 26 of that line. Perhaps
this will help (NOT TESTED):

*od -x -A d weewx.conf | grep e0*

This should print out the line and offending sequence.

-tk



On Wed, Aug 19, 2020 at 5:25 AM David Marshall 
wrote:

> Hi
>
> Installed a fresh version of weewx 4.1.1 without problem using setup.py on
> a raspberry.
>
> Transferred my existing sqlite database from weewx 3.9.1. It is big, about
> 3 years of data. This worked fine.
>
> But when I tried to use reconfigure I got the following error
>
> pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure
> /home/weewx/archive/weewx.sdb
> Traceback (most recent call last):
>   File "/home/weewx/bin/wee_database", line 974, in 
> main()
>   File "/home/weewx/bin/wee_database", line 128, in main
> config_path, config_dict = weecfg.read_config(options.config_path,
> args)
>   File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
> default_encoding='utf-8')
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in
> __init__
> self._load(infile, configspec)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
> content = self._handle_bom(content)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in
> _handle_bom
> return self._decode(infile, self.encoding)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
> infile[i] = line.decode(encoding)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26:
> invalid continuation byte
>
> Anyone know how to fix this?
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zECdAYF6MxE6Lyd2o8Svq_94pt%2ByMyiy_AniiBOZnT6H_g%40mail.gmail.com.

[weewx-user] UnicodeDecodeError in database

[David Marshall]

Hi

Installed a fresh version of weewx 4.1.1 without problem using setup.py on 
a raspberry.

Transferred my existing sqlite database from weewx 3.9.1. It is big, about 
3 years of data. This worked fine.

But when I tried to use reconfigure I got the following error

pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure 
/home/weewx/archive/weewx.sdb
Traceback (most recent call last):
  File "/home/weewx/bin/wee_database", line 974, in 
main()
  File "/home/weewx/bin/wee_database", line 128, in main
config_path, config_dict = weecfg.read_config(options.config_path, args)
  File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config
default_encoding='utf-8')
  File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in __init__
self._load(infile, configspec)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load
content = self._handle_bom(content)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in 
_handle_bom
return self._decode(infile, self.encoding)
  File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode
infile[i] = line.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26: 
invalid continuation byte

Anyone know how to fix this?




-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com.