[weewx-user] Re: Import from a csv-file

[Ha Ba]

Hi Gary

I compared the two files, but I cannot see an error. 

The bigest changes I maked in the "fieldmap", please take a look on it.

Thx, Hans

Am Mittwoch, 27. März 2019 00:28:39 UTC+1 schrieb gjr80:
>
> Hans,
>
> On Wednesday, 27 March 2019 09:22:18 UTC+1, Ha Ba wrote:
>>
>> Do you think, this is something a newbie can fix?
>>
>
> Of course, anybody can fix it but the trick is in knowing what is wrong 
> first :) There is more than one error (most likely structure) in 
> /home/weewx/csv.conf. You can try and locate and fix the errors yourself 
> by comparing the format/structure of /home/weewx/csv.conf to the example 
> csv import config file at /home/weewx/util/import/csv-example.conf. If 
> you can't find the problems post a copy of /home/weewx/csv.conf.
>
> Gary
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES
#
# Copyright (c) 2009-2016 Tom Keffer  and Gary Roderick.
# See the file LICENSE.txt for your rights.

##

# Specify the source. Available options are:
#   CSV - import obs from a single CSV format file
#   WU - import obs from a Weather Underground PWS history
#   Cumulus - import obs from a one or more Cumulus monthly log files
# Format is:
#   source = (CSV | WU | Cumulus)
source = CSV

##

[CSV]
# Parameters used when importing from a CSV file

# Path and name of our CSV source file. Format is:
#   file = full path and filename
file = /var/tmp/data.csv

# If there is no mapped interval field how will the interval field be
# determined for the imported records. Available options are:
#   derive - Derive the interval field from the timestamp of successive
#records. This setting is best used when the imported records
#are equally spaced in time and there are no missing records.
#   conf   - Use the interval setting from weewx.conf. This setting is
#best used if the records to be imported have been produced by
#WeeWX using the same archive interval as set in weewx.conf on
#this machine.
#   x  - Use a fixed interval of x minutes for every record. This
#setting is best used if the records to be imported are
#equally based in time but there are some missing records.
#
# Note: If there is a mapped interval field then this setting will be
#   ignored.
# Format is:
#   interval = (derive | conf | x)
interval = derive

# Should the [StdQC] max/min limits in weewx.conf be applied to the
# imported data. This may be useful if the source has extreme values that
# are clearly incorrect for some observations. Available options are:
#   True  - weewx.conf [StdQC] max/min limits are applied.
#   False - weewx.conf [StdQC] max/min limits are not applied.
# Format is:
#   qc = (True | False)
qc = True

# Should any missing derived observations be calculated from the imported
# data if possible. Available options are:
#   True  - Any missing derived observations are calculated.
#   False - Any missing derived observations are not calculated.
# Format is:
#   calc_missing = (True | False)
calc_missing = True

# Specify how imported data fields that contain invalid data (eg a numeric
# field containing non-numeric data) are handled. Available options are:
#   True  - The invalid data is ignored, the WeeWX target field is set to
#   None and the import continues.
#   False - The import is halted.
# Format is:
#   ignore_invalid_data = (True | False)
# Default is True.
ignore_invalid_data = True

# Imported records are written to archive in transactions of tranche
# records at a time. Increase for faster throughput, decrease to reduce
# memory requirements. Format is:
#   tranche = x
# where x is an integer
tranche = 250

# Specify whether a UV sensor was used to produce any UV observations.
# Available options are:
#   True  - UV sensor was used and UV data will be imported.
#   False - UV sensor was not used and any UV data will not be imported.
#   UV fields will be set to None/NULL.
# For a CSV import UV_sensor should be set to False if a UV sensor was
# NOT present when the import data was created. Otherwise it may be set to
# True or omitted. Format is:
#   UV_sensor = (True | False)
UV_sensor = False

# Specify whether a solar radiation sensor was used to produce any solar
# radiation observations. 

[weewx-user] Re: Import from a csv-file

[gjr80]

Hans,

On Wednesday, 27 March 2019 09:22:18 UTC+10, Ha Ba wrote:
>
> Do you think, this is something a newbie can fix?
>

Of course, anybody can fix it but the trick is in knowing what is wrong 
first :) There is more than one error (most likely structure) in 
/home/weewx/csv.conf. You can try and locate and fix the errors yourself by 
comparing the format/structure of /home/weewx/csv.conf to the example csv 
import config file at /home/weewx/util/import/csv-example.conf. If you 
can't find the problems post a copy of /home/weewx/csv.conf.

Gary

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Import from a csv-file

[Ha Ba]

Hi Gary,

Thank you for response.

I was able to restore the wee_import file and the errormessage changed.

Now I have this:

**3
al@xxx420:~$ sudo /home/weewx/bin/wee_import 
--config=/home/weewx/weewx.conf --import-config=/home/weewx/csv.conf
Starting wee_import...
Traceback (most recent call last):
  File "/home/weewx/bin/wee_import", line 703, in 
main()
  File "/home/weewx/bin/wee_import", line 652, in main
wlog)
  File "/home/weewx/bin/weeimport/weeimport.py", line 294, in sourceFactory
file_name=options.import_config_path)
  File "/home/weewx/bin/weecfg/__init__.py", line 175, in read_config
config_dict = configobj.ConfigObj(config_path, file_error=True)
  File "/usr/lib/python2.7/dist-packages/configobj.py", line 1229, in 
__init__
self._load(infile, configspec)
  File "/usr/lib/python2.7/dist-packages/configobj.py", line 1318, in _load
raise error
configobj.ConfigObjError: Parsing failed with several errors.
First error at line 183.


Do you think, this is something a newbie can fix?

Hans


When using wee_import the only file you should modify is the import config 
> file, and then the recommended approach is to take a copy of one of the 
> example import config files in the /home/weewx/util directory. The 
> wee_import instructions 
>  cover this 
> fairly thoroughly though if English is not your main language you might 
> find the instructions heavy going. Let us know if you are unsure and we can 
> help you out. If you prefer post your questions using your native language 
> and someone or Google translate will do the translation.
>
> Gary
>
> On Wednesday, 27 March 2019 04:18:32 UTC+10, Ha Ba wrote:
>>
>> Hello
>>
>> First, excuse my bad English. I hope you understand what I mean.
>>
>> Working with weewx 3.9.1. Driver is the te923.py. My archive-database is 
>> sqlite.
>>
>> I started with recording data a few days ago and I have a big file with 
>> csv-data, exported over many weeks with the original 
>> windows-stationsoftware.
>>
>> The csv.conf-file I created form the csv-example.conf.
>>
>> Now I will bring in this data to my archive-database with this command:
>>
>> *sudo /home/weewx/bin/wee_import --config=/home/weewx/weewx.conf 
>> --import-config=/home/weewx/csv.conf*
>>
>> and get this error:
>>
>> ***
>>   File "/home/weewx/bin/wee_import", line 76
>> file = /var/tmp/data.csv
>> ^
>> IndentationError: unexpected indent
>> ***
>>
>> I think, I know what "IndentationError" means, but I cannot see what is 
>> wrong in the csv.conf 
>>
>> * this is the part in my csv.conf, where I assume the error **
>>
>> # Specify the source. Available options are:
>> #   CSV - import obs from a single CSV format file
>> #   WU - import obs from a Weather Underground PWS history
>> #   Cumulus - import obs from a one or more Cumulus monthly log files
>> # Format is:
>> #   source = (CSV | WU | Cumulus)
>> source = CSV
>>
>>
>> ##
>>
>> [CSV]
>> # Parameters used when importing from a CSV file
>>
>> # Path and name of our CSV source file. Format is:
>> #   file = full path and filename
>> file = /var/tmp/data.csv
>>
>> # If there is no mapped interval field how will the interval field be
>> # determined for the imported records. Available options are:
>>
>> ** End part 
>>
>> I'm a absolte beginner in Python ;(
>>
>> regards, 
>> Hans
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Import from a csv-file

[gjr80]

Hi,

I suspect that what you have done is made modification to the file 
/home/weewx/bin/wee_import in an attempt to do a csv import (the chnages 
may be as simple as removing some comments around line 76 but it is 
impossible for me to tell). If you did not intentionally make changes to 
/home/weewx/bin/wee_import then it has been changed by some other means. 
Irrespective, these changes need to be removed and 
/home/weewx/bin/wee_import restored to its original state or you will not 
be able to run wee_import. If you cannot put the file back to its original 
state let me know and I will give you some detailed instructions for how to 
download a replacement copy.

When using wee_import the only file you should modify is the import config 
file, and then the recommended approach is to take a copy of one of the 
example import config files in the /home/weewx/util directory. The 
wee_import instructions 
 cover this fairly 
thoroughly though if English is not your main language you might find the 
instructions heavy going. Let us know if you are unsure and we can help you 
out. If you prefer post your questions using your native language and 
someone or Google translate will do the translation.

Gary

On Wednesday, 27 March 2019 04:18:32 UTC+10, Ha Ba wrote:
>
> Hello
>
> First, excuse my bad English. I hope you understand what I mean.
>
> Working with weewx 3.9.1. Driver is the te923.py. My archive-database is 
> sqlite.
>
> I started with recording data a few days ago and I have a big file with 
> csv-data, exported over many weeks with the original 
> windows-stationsoftware.
>
> The csv.conf-file I created form the csv-example.conf.
>
> Now I will bring in this data to my archive-database with this command:
>
> *sudo /home/weewx/bin/wee_import --config=/home/weewx/weewx.conf 
> --import-config=/home/weewx/csv.conf*
>
> and get this error:
>
> ***
>   File "/home/weewx/bin/wee_import", line 76
> file = /var/tmp/data.csv
> ^
> IndentationError: unexpected indent
> ***
>
> I think, I know what "IndentationError" means, but I cannot see what is 
> wrong in the csv.conf 
>
> * this is the part in my csv.conf, where I assume the error **
>
> # Specify the source. Available options are:
> #   CSV - import obs from a single CSV format file
> #   WU - import obs from a Weather Underground PWS history
> #   Cumulus - import obs from a one or more Cumulus monthly log files
> # Format is:
> #   source = (CSV | WU | Cumulus)
> source = CSV
>
>
> ##
>
> [CSV]
> # Parameters used when importing from a CSV file
>
> # Path and name of our CSV source file. Format is:
> #   file = full path and filename
> file = /var/tmp/data.csv
>
> # If there is no mapped interval field how will the interval field be
> # determined for the imported records. Available options are:
>
> ** End part 
>
> I'm a absolte beginner in Python ;(
>
> regards, 
> Hans
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.