Thanks Gary,
I'll give it a test drive as soon as I get a moment or three.
On a related note (ie:- related to using TEXT fields in the database)
I went and checked out wee_database and found that it baulked at the
TEXT field in the database.
Details are in the attached file, which should keep the formatting sane.
Let me know if you want it raised as a seperate issue and I'll do that instead.
On 22/01/2022, gjr80 <gjroder...@gmail.com> wrote:
> Glenn,
>
> I have created a new branch wee_import_text off development for this and
> have modified the wee_import code to import text to a WeeWX TEXT field.
> There are a dew conditions:
>
> 1. text import only works with CSV imports
>
> 2. import source fields that are to be imported as text must use 'text' as
> the WeeWX units specifier in the field map in the import config file, eg:
> [[FieldMap]]
> ....
> aComment = import_text_field, text
>
> 3. the destination text field has been added to the WeeWX database and
> WeeWX configured as per the Storing text in the database
> <https://github.com/weewx/weewx/wiki/Storing-text-in-the-database> wiki
> page
>
> I have only done basic testing of the code so there may be some corner
> cases I have not covered, but you should be right to give it a go if you
> want. As I said I have taken a branch off development, since there has been
>
> quite a few changes in development since the last release the most reliable
>
> option is for you to clone and install wee_import_text. However, the
> wee_import code does not change often so you could probably get away with
> running v4.5.1 and just downloading and installing the modified
> weeimport.py
> .
>
> To use the new wee_import:
>
> 1. Either clone and install from the wee_import_text branch or if using
> v4.5.1 (or any other recent version at your own peril) replace
> BIN/weeimport/weeimport.py with weeimport.py from the wee_import_text
> branch. You can download weeimport.py from the wee_import_text branch
> directly using wget as follows:
>
> $ wget
> https://raw.githubusercontent.com/weewx/weewx/wee_import_text/bin/weeimport/weeimport.py
>
> 2. edit your import config file and add your text field import to the field
>
> map as outlined above
>
> 3. stop WeeWX and run your import using --dry-run and then without
> --dry-run
>
> Let me know if you have any issues.
>
> Gary
>
--
Cheers
Glenn
rorpi - read only raspberry pi & various weewx addons
https://github.com/glennmckechnie
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAAraAzjMzv1_GLT4AAs6b8FSBy1Y0xJj5JQS8ariGu3fEnrZ5w%40mail.gmail.com.
# Glenn McKechnie - modified 22/01/22
weewx.4.5.1
==== If a database with a field type of TEXT is checked using `wee_database
--check-strings` it fails with the following stack trace.
:graybeard@whitebeard:/home/weewx/bin
06:50 PM $ ./wee_database --check-strings
Using configuration file /home/weewx451/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'
Preparing Null String Fix, this may take a while...
Traceback (most recent call last):
File "./wee_database", line 1138, in <module>
main()
File "./wee_database", line 227, in main
check_strings(config_dict, db_binding, options, fix=False)
File "./wee_database", line 934, in check_strings
if record[icol] is not None and not check_type(record[icol],
obs_type_list[icol]):
File "./wee_database", line 879, in check_type
raise ValueError("Unknown type %s" % expected)
ValueError: Unknown type TEXT
====== FIXED (for me) with the following changes to wee_database
xterm root@whitebeard:/home
04:44 PM $ diff -au weewx451/bin/wee_database weewx451/bin/wee_database-text
--- weewx451/bin/wee_database 2021-04-03 06:02:14.000000000 +1100
+++ weewx451/bin/wee_database-text 2022-01-22 16:25:26.293912710 +1100
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/python3
#
# Copyright (c) 2009-2021 Tom Keffer <tkef...@gmail.com>
#
@@ -875,6 +875,8 @@
return isinstance(val, float)
elif expected == 'STR':
return isinstance(val, six.string_types)
+ elif expected == 'TEXT':
+ return isinstance(val, six.string_types)
else:
raise ValueError("Unknown type %s" % expected)
@@ -886,6 +888,8 @@
return float(val)
elif target == 'STR':
return six.ensure_str(val)
+ elif target == 'TEXT':
+ return six.ensure_str(val)
else:
raise ValueError("Unknown type %s" % target)
==== Null strings in test record (which also has a TEXT field) are then
detected as expected.
:graybeard@whitebeard:/home/weewx/bin
07:34 PM $ sudo ./wee_database --fix-strings
Using configuration file /home/weewx451/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'
Preparing Null String Fix, this may take a while...
hecking record: 253; Timestamp: 2022-01-17 16:50:00 AEDT (1642398600)
The following null strings were found:
Timestamp = 1641720720; record['extraTemp1'] = 'Null'; ... changed to None
Timestamp = 1641720720; record['extraTemp2'] = 'Null'; ... changed to None
Timestamp = 1641720720; record['extraTemp3'] = 'Null'; ... changed to None
3 of 3 null strings found were fixed.
Applied Null String Fix in 0.29 seconds.
