[weewx-user] Re: 4.6.2. update problem with wee_import
Gary, yes, my mistake! I used the wrong version of weeimport.py while I was returning the original. I have many... Thanks for advising me. Sorry. Now everything works and I could import data! Your patch was working great. Thank you! You are really doing a good work! best regards Janne tiistai 22. helmikuuta 2022 klo 12.17.31 UTC+2 gjr80 kirjoitti: > Janne, > > "It doesn't work" is not very helpful. Please return weeimport.py to it's > original 4.6.2 state, ensure you have the patched weeutil.py and then run > the import again. Please post an exact copy of the console output including > the command issued and responses/error traces. > > Gary > > On Tuesday, 22 February 2022 at 18:55:59 UTC+10 jannep...@gmail.com wrote: > >> Gary, >> >> Thanks! >> I installed the patch, but it did not help. I also tried to uncomment my >> own fix from weeimport.py, (the fix which you don't recommend to do...) but >> that returned my first problem back. >> >> My original date.csv has datetime format as in my csv.conf >> raw_datetime_format expect. Thanks anyway. >> >> Janne >> >> tiistai 22. helmikuuta 2022 klo 4.42.37 UTC+2 gjr80 kirjoitti: >> >>> Janne, >>> >>> You've encountered a bug that shows when wee_import is run under python >>> 2 using a user defined field map. The fix required was not to wee_import >>> but elsewhere in WeeWX and will appear in the next release (expected to be >>> 4.6.3). In the meantime since you are using WeeWX v4.6.2 you can safely >>> just download the patched file, replace the corresponding file on your >>> system and wee_import should work fine. To download and install the >>> patched file: >>> >>> 1. move aside the file to be patched: >>> $ mv /usr/share/weewx/weeutil/weeutil.py >>> /usr/share/weewx/weeutil/weeutil_orig.py >>> >>> 2. download the patched file: >>> $ wget -P /usr/share/weewx/weeutil/ >>> https://raw.githubusercontent.com/weewx/weewx/master/bin/weeutil/weeutil.py >>> >>> 3. stop WeeWX (if it is running) and perform your import >>> >>> I did a test import using the patched file, your import config file and >>> your CSV data. The raw_datetime_format setting in your import config >>> file needs to be changed to match the format of the date-time field your >>> are using in your source data, it needs to be set as follows: >>> >>> raw_datetime_format = "%d.%m.%y %H:%M" >>> >>> With this change your data imported correctly and with out error. Note >>> that you will likely receive the following message during the import: >>> >>> Warning: Import field 'gustDir' is mapped to WeeWX field 'windGustDir' >>> but the >>> import field 'gustDir' could not be found in one or more >>> records. >>> WeeWX field 'windGustDir' will be set to 'None' in these >>> records. >>> >>> This is just a warning that you have a mapped field for which there is >>> no source data. It can be safely ignored. >>> >>> Gary >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a6dbe838-a328-4576-8c3f-ec6e1284a827n%40googlegroups.com.
[weewx-user] Re: 4.6.2. update problem with wee_import
Gary, Thanks! I installed the patch, but it did not help. I also tried to uncomment my own fix from weeimport.py, (the fix which you don't recommend to do...) but that returned my first problem back. My original date.csv has datetime format as in my csv.conf raw_datetime_format expect. Thanks anyway. Janne tiistai 22. helmikuuta 2022 klo 4.42.37 UTC+2 gjr80 kirjoitti: > Janne, > > You've encountered a bug that shows when wee_import is run under python 2 > using a user defined field map. The fix required was not to wee_import > but elsewhere in WeeWX and will appear in the next release (expected to be > 4.6.3). In the meantime since you are using WeeWX v4.6.2 you can safely > just download the patched file, replace the corresponding file on your > system and wee_import should work fine. To download and install the > patched file: > > 1. move aside the file to be patched: > $ mv /usr/share/weewx/weeutil/weeutil.py > /usr/share/weewx/weeutil/weeutil_orig.py > > 2. download the patched file: > $ wget -P /usr/share/weewx/weeutil/ > https://raw.githubusercontent.com/weewx/weewx/master/bin/weeutil/weeutil.py > > 3. stop WeeWX (if it is running) and perform your import > > I did a test import using the patched file, your import config file and > your CSV data. The raw_datetime_format setting in your import config file > needs to be changed to match the format of the date-time field your are > using in your source data, it needs to be set as follows: > > raw_datetime_format = "%d.%m.%y %H:%M" > > With this change your data imported correctly and with out error. Note > that you will likely receive the following message during the import: > > Warning: Import field 'gustDir' is mapped to WeeWX field 'windGustDir' but > the > import field 'gustDir' could not be found in one or more records. > WeeWX field 'windGustDir' will be set to 'None' in these records. > > This is just a warning that you have a mapped field for which there is no > source data. It can be safely ignored. > > Gary > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ad8f32e3-727e-46e6-acb1-8348a1aba4e7n%40googlegroups.com.
[weewx-user] Re: 4.6.2. update problem with wee_import
Hi Gary I sent you PM Janne maanantai 21. helmikuuta 2022 klo 23.06.58 UTC+2 gjr80 kirjoitti: > Would you also please provide a copy of the import config file you are > using and a sample of the data (at least the header line and a few lines of > data) you are trying to import. > > I would also stop trying to patch the code to make it work; you will > likely only continue to encounter other issues, most likely caused by your > previous changes. Give me some time to work through the initial problem. > > Gary > > On Tuesday, 22 February 2022 at 06:06:04 UTC+10 jannep...@gmail.com wrote: > >> Hi, more info. >> I fixed with ducktape the first problem by commenting out the line 635 >> from weeimport.py >> >> #and _val['units'] not in >> weewx.units.USUnits.values() >> >> But this did not solve my import troubles. Now it looks that csv.conf and >> weewx.conf does not understand each other. csv.conf is asking for unkonwn >> units >> >>Using WeeWX configuration file /etc/weewx/weewx.conf >>Starting wee_import... >>A CSV import from source file '/var/tmp/data.csv' has been requested. >>Using database binding 'wx_binding', which is bound to database >> 'weewx.sdb' >>Destination table 'archive' unit system is '0x11' (METRICWX). >>Missing derived observations will be calculated. >>This is a dry run, imported data will not be saved to archive. >>Starting dry run import ... >> Unknown units 'uv_index' specified for field 'UV' in >> /var/tmp/csv.conf. >> Nothing done, exiting. >> >> If I comment out UV, from my csv.conf wee_import finds next unknown >> unit... I also have tried to create group_uv to weewx.conf. The unkonwn >> units are uv_index, percent, degree_compass. >> >> regards >> Janne >> maanantai 21. helmikuuta 2022 klo 14.44.40 UTC+2 jannep...@gmail.com >> kirjoitti: >> >>> hi >>> python 2. Here is log. When I start weewx, LOOP is working. I can get >>> weather station info to terminal, but maybe not to database. I have a >>> Ventus weather station and I download data every now and then to windows. >>> From there I import the data in csv to weewx. >>> >>> Janne >>> >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Initializing weewx >>> version 4.6.2 >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using Python >>> 2.7.17 (default, Feb 27 2021, 15:10:58) #012[GCC 7.5.0] >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Platform >>> Linux-5.4.0-99-generic-x86_64-with-LinuxMint-19.3-tricia >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Locale is >>> 'fi_FI.UTF-8' >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using >>> configuration file /etc/weewx/weewx.conf >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Debug is 0 >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Loading >>> station type WS6in1 (user.ws6in1) >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: StdConvert >>> target unit is 0x11 >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.wxservices: >>> StdWXCalculate will use data binding wx_binding >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Archive will >>> use data binding wx_binding >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Record >>> generation will be attempted in 'hardware' >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Using archive >>> interval of 300 seconds (specified in weewx configuration) >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: >>> StationRegistry: Registration not requested. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: Wunderground: >>> Posting not enabled. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: PWSweather: >>> Posting not enabled. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: CWOP: Posting >>> not enabled. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: WOW: Posting >>> not enabled. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: AWEKAS: Posting >>> not enabled. >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: 'pyephem' not >>> detected, extended almanac data is not available >>> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Starting up weewx >>> version 4.6.
[weewx-user] Re: 4.6.2. update problem with wee_import
Hi, more info.
I fixed with ducktape the first problem by commenting out the line 635 from
weeimport.py
#and _val['units'] not in
weewx.units.USUnits.values()
But this did not solve my import troubles. Now it looks that csv.conf and
weewx.conf does not understand each other. csv.conf is asking for unkonwn
units
Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database
'weewx.sdb'
Destination table 'archive' unit system is '0x11' (METRICWX).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...
Unknown units 'uv_index' specified for field 'UV' in
/var/tmp/csv.conf.
Nothing done, exiting.
If I comment out UV, from my csv.conf wee_import finds next unknown unit...
I also have tried to create group_uv to weewx.conf. The unkonwn units are
uv_index, percent, degree_compass.
regards
Janne
maanantai 21. helmikuuta 2022 klo 14.44.40 UTC+2 jannep...@gmail.com
kirjoitti:
> hi
> python 2. Here is log. When I start weewx, LOOP is working. I can get
> weather station info to terminal, but maybe not to database. I have a
> Ventus weather station and I download data every now and then to windows.
> From there I import the data in csv to weewx.
>
> Janne
>
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Initializing weewx
> version 4.6.2
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using Python 2.7.17
> (default, Feb 27 2021, 15:10:58) #012[GCC 7.5.0]
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Platform
> Linux-5.4.0-99-generic-x86_64-with-LinuxMint-19.3-tricia
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Locale is
> 'fi_FI.UTF-8'
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using configuration
> file /etc/weewx/weewx.conf
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Debug is 0
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Loading station
> type WS6in1 (user.ws6in1)
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: StdConvert
> target unit is 0x11
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.wxservices:
> StdWXCalculate will use data binding wx_binding
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Archive will use
> data binding wx_binding
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Record
> generation will be attempted in 'hardware'
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Using archive
> interval of 300 seconds (specified in weewx configuration)
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: StationRegistry:
> Registration not requested.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: Wunderground:
> Posting not enabled.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: PWSweather:
> Posting not enabled.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: CWOP: Posting not
> enabled.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: WOW: Posting not
> enabled.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: AWEKAS: Posting
> not enabled.
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: 'pyephem' not
> detected, extended almanac data is not available
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Starting up weewx
> version 4.6.2
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Using binding
> 'wx_binding' to database 'weewx.sdb'
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.manager: Starting
> backfill of daily summaries
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.manager: Daily summaries
> up to date
> Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Starting main
> packet loop.
>
> maanantai 21. helmikuuta 2022 klo 9.37.18 UTC+2 gjr80 kirjoitti:
>
>> Janne,
>>
>> Could you please let me know whether WeeWX is being run under python 2 or
>> 3? You can tell by looking at the WeeWX log during WeeWX startup (the
>> python version is logged) or you can look at the first line of any of the
>> wee_xx (eg wee_database) utilities in the /usr/share/weewx directory.
>>
>> If yo cannot work it out please restart WeeWX and post a log extract
>> showing the full WeeWX startup.
>>
>> Gary
>>
>> On Monday, 21 February 2022 at 04:21:57 UTC+10 jannep...@gmail.com wrote:
>>
>>> Hi, I updated from 4.5 to 4.6.2 and my wee_import stop to next message:
>>>
>>> I use WS6in1-driver with my Ventus.
>>>
>>> ...
>
[weewx-user] Re: 4.6.2. update problem with wee_import
hi
python 2. Here is log. When I start weewx, LOOP is working. I can get
weather station info to terminal, but maybe not to database. I have a
Ventus weather station and I download data every now and then to windows.
>From there I import the data in csv to weewx.
Janne
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Initializing weewx
version 4.6.2
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using Python 2.7.17
(default, Feb 27 2021, 15:10:58) #012[GCC 7.5.0]
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Platform
Linux-5.4.0-99-generic-x86_64-with-LinuxMint-19.3-tricia
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Locale is
'fi_FI.UTF-8'
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Using configuration
file /etc/weewx/weewx.conf
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Debug is 0
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Loading station
type WS6in1 (user.ws6in1)
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: StdConvert target
unit is 0x11
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.wxservices:
StdWXCalculate will use data binding wx_binding
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Archive will use
data binding wx_binding
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Record generation
will be attempted in 'hardware'
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Using archive
interval of 300 seconds (specified in weewx configuration)
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: StationRegistry:
Registration not requested.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: Wunderground:
Posting not enabled.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: PWSweather:
Posting not enabled.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: CWOP: Posting not
enabled.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: WOW: Posting not
enabled.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.restx: AWEKAS: Posting
not enabled.
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: 'pyephem' not
detected, extended almanac data is not available
Feb 21 14:31:09 janne-Lat weewx[21011] INFO __main__: Starting up weewx
version 4.6.2
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Using binding
'wx_binding' to database 'weewx.sdb'
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.manager: Starting
backfill of daily summaries
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.manager: Daily summaries
up to date
Feb 21 14:31:09 janne-Lat weewx[21011] INFO weewx.engine: Starting main
packet loop.
maanantai 21. helmikuuta 2022 klo 9.37.18 UTC+2 gjr80 kirjoitti:
> Janne,
>
> Could you please let me know whether WeeWX is being run under python 2 or
> 3? You can tell by looking at the WeeWX log during WeeWX startup (the
> python version is logged) or you can look at the first line of any of the
> wee_xx (eg wee_database) utilities in the /usr/share/weewx directory.
>
> If yo cannot work it out please restart WeeWX and post a log extract
> showing the full WeeWX startup.
>
> Gary
>
> On Monday, 21 February 2022 at 04:21:57 UTC+10 jannep...@gmail.com wrote:
>
>> Hi, I updated from 4.5 to 4.6.2 and my wee_import stop to next message:
>>
>> I use WS6in1-driver with my Ventus.
>>
>> ...
>> Using WeeWX configuration file /etc/weewx/weewx.conf
>> Starting wee_import...
>> A CSV import from source file '/var/tmp/data.csv' has been requested.
>> Using database binding 'wx_binding', which is bound to database
>> 'weewx.sdb'
>> Destination table 'archive' unit system is '0x11' (METRICWX).
>> Missing derived observations will be calculated.
>> This is a dry run, imported data will not be saved to archive.
>> Starting dry run import ...
>> Traceback (most recent call last):
>> File "/usr/share/weewx/wee_import", line 899, in
>> main()
>> File "/usr/share/weewx/wee_import", line 829, in main
>> source_obj.run()
>> File "/usr/share/weewx/weeimport/weeimport.py", line 372, in run
>> _raw_data = self.getRawData(period)
>> File "/usr/share/weewx/weeimport/csvimport.py", line 246, in getRawData
>> self.map = self.parseMap('CSV', _csv_reader, self.csv_config_dict)
>> File "/usr/share/weewx/weeimport/weeimport.py", line 635, in parseMap
>> and _val['units'] not in weewx.units.USUnits.values():
>> AttributeError: 'ListOfDicts' object has no attribute 'values'
>> ..
>>
>> What this does mean? Any help?
>>
>> Is there easy way to get back to working configuration (4.5)?
>>
>> regards
>> Janne
>>
>
--
You received this message because yo
[weewx-user] 4.6.2. update problem with wee_import
Hi, I updated from 4.5 to 4.6.2 and my wee_import stop to next message:
I use WS6in1-driver with my Ventus.
...
Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x11' (METRICWX).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...
Traceback (most recent call last):
File "/usr/share/weewx/wee_import", line 899, in
main()
File "/usr/share/weewx/wee_import", line 829, in main
source_obj.run()
File "/usr/share/weewx/weeimport/weeimport.py", line 372, in run
_raw_data = self.getRawData(period)
File "/usr/share/weewx/weeimport/csvimport.py", line 246, in getRawData
self.map = self.parseMap('CSV', _csv_reader, self.csv_config_dict)
File "/usr/share/weewx/weeimport/weeimport.py", line 635, in parseMap
and _val['units'] not in weewx.units.USUnits.values():
AttributeError: 'ListOfDicts' object has no attribute 'values'
..
What this does mean? Any help?
Is there easy way to get back to working configuration (4.5)?
regards
Janne
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/68f5ce52-0ead-46ff-a293-43691c2e079cn%40googlegroups.com.
