Thank you Gary for your work and your explanations. Your explanations were very clear and I had already spotted all these files.
I had some errors which did not seem to me to be related to the import, I tried an update with *python3 -m pip install weewx --upgrade* who actually updated my weewx and everything was back to normal. I was able to do my wee_import without difficulty. I even found that the process was faster than the first time. However, I have a question, when I do an update with the --date=2023-10-27 option, why does the import want to do an import of 2023-10-28 too? This gives me the errors: ___________________________________________________________________________________________________________________________________________________________________________________ $ p *ython3 wee_import.py --config=/home/sylvain/weewx-data/weewx.conf --import-config=/home/sylvain/weewx-data/util/import/wu-example.conf --date=2023- 10-27 --verboseUsing WeeWX configuration file /home/sylvain/weewx-data/weewx.confStarting wee_import...Observation history for Weather Underground station 'IG******' will be imported.The following options will be used: config=/home/sylvain/weewx-data/weewx.conf, import-config=/home/sylvain/weewx-data/util/import/wu-example.conf station=IG******, date=2023-10-27 apiKey=****************************0b80 dry-run=None, calc_missing=True, ignore_invalid_data=True slice=250, interval=1, wind_direction=[0.0, 360.0]Using database binding 'wx_binding', which is bound to database 'weewx'Destination table 'archive' unit system is '0x11' (METRICWX).Missing derived observations will be calculated.Observations timestamped after 2023-10-27 00:00:00 CEST (1698357600) and up to andincluding 2023-10-28 00:00:00 CEST (1698444000) will be imported.Starting import...Records covering multiple periods have been identified for import.Period 1...Obtaining raw import data for period 1 ...Raw import data read successfully for period 1.Mapping raw import data for period 1 ...Mapped 155 records.Raw import data mapped successfully for period 1.Saving mapped data to archive for period 1 ...Proceeding will save all imported records in the WeeWX archive.Are you sure you want to proceed (y/n)? yUnique records processed: 155; Last timestamp: 2023-10-27 12:54:52 CEST (1698404092)Mapped data saved to archive successfully ""for period 1.Period 2...Obtaining raw import data for period 2 ...**** Unable to load source data for period 2.**** Possibly a bad station ID, an invalid date or data does not exist for this period.****Period 2 will be skipped. Proceeding to next period.Calculating missing derived observations ...Processing record: 155; Last record: 2023-10-28 00:00:00 CEST (1698444000)Recalculating daily summaries...Finished recalculating daily summariesFinished calculating missing derived observationsFinished importTraceback (most recent call last): File "/home/sylvain/weewx-venv/lib/python3.11/site-packages/wee_import.py", line 899, in <module> hand() File "/home/sylvain/weewx-venv/lib/python3.11/site-packages/wee_import.py", line 829, in main source_obj.run() File "/home/sylvain/weewx-venv/lib/python3.11/site-packages/weeimport/weeimport.py", line 509, in run _msg = "%d records were processed and %d unique records " \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^TypeError: must be real number, not NoneType* *______________________________________________________________________________________________________________________________________________________________________________________________* I had the same errors with the --from=xxxx --to=xxxxx, it adds an extra day to the import. Do you have a tip for not exceeding the date limit? I wanted to edit a cron to automatically update every 15 minutes or according to the last date recorded with a bash script but this could be complicated for me especially since these errors are annoying and I have to deal with a datetime in the format of the Epoch. Le vendredi 27 octobre 2023 à 00:36:25 UTC+2, gjr80 a écrit : > I've patched wee_import to handle the changed WU API response. The > patched code will appear in the v5.0.0 (when released). In the meantime if > you wish to use it now with any other the v5.0.0 beta releases you can by > downloading the patched wuimport.py and replacing your existing version. > The location of the v5 WeeWX files is significantly different to that for > v4. You indicated you installed v5 via pip, so the following instructions > should work in this case (or until I have a better approach for installing > patched file): > > 1. download the patched wuimport.py: > > $ wget -P /var/tmp > https://raw.githubusercontent.com/weewx/weewx/V5/src/weeimport/wuimport.py > > 2. locate your existing wuimport.py. If you followed the v5 install > instructions and installed via pip you may find wuimport.py in something > like ~/weewx-venv/lib/python3.9/site-packages/weeimport . If you used a > different virtual environment name to weewx-venv or are running a > different python version change the path accordingly, or depending on your > system the path may be different again. Either way you need to find > wuimport.py. > > 3. If you cannot find wuimport.py on your system or are not sure do not > proceed or guess - stop and seek clarification. Once you have found > wuimport.py copy the downloaded version in /var/tmp over the existing > version. > > 4. you can now use wee_import to import from WU. > > Gary > > On Wednesday, 25 October 2023 at 09:22:06 UTC+10 gjr80 wrote: > >> It's is not your config, it looks like WU has changed the header content >> in one (or more) of their API responses. I should be able to have a look >> tonite and provide a fix. >> >> Gary >> >> On Wednesday, 25 October 2023 at 08:06:04 UTC+10 weewx-user wrote: >> >> The most annoying thing is that I have errors with wee_import to feed my >> mysql database: >> >> *sylvain@gandalf:~/weewx-venv/lib/python3.11/site-packages>** python3 >> wee_import.py --config=/home/sylvain/weewx-data/weewx.conf >> --import-config=/home/sylvain /weewx-data/util/import/wu-example.conf >> --dry-run --from=2022-7-12 --to=2023-10-24 --verbose* >> Using WeeWX configuration file /home/sylvain/weewx-data/weewx.conf >> Starting wee_import... >> Observation history for Weather Underground station 'I********' will be >> imported. >> The following options will be used: >> config=/home/sylvain/weewx-data/weewx.conf, >> import-config=/home/sylvain/weewx-data/util/import/wu-example.conf >> station=I*******, from=2022-7-12, to=2023-10-24 >> apiKey=****************************0b80 >> dry-run=True, calc_missing=True, ignore_invalid_data=True >> slice=250, interval=x, wind_direction=[0.0, 360.0] >> Using database binding 'wx_binding', which is bound to database 'weewx' >> Destination table 'archive' unit system is '0x11' (METRICWX). >> Missing derived observations will be calculated. >> Observations timestamped after 2022-07-12 00:00:00 CEST (1657576800) and >> up to and >> including 2023-10-25 00:00:00 CEST (1698184800) will be imported. >> This is a dry run, imported data will not be saved to archive. >> Starting dry run import... >> Records covering multiple periods have been identified for import. >> Period 1... >> Obtaining raw import data for period 1 ... >> Traceback (most recent call last): >> File >> "/home/sylvain/weewx-venv/lib/python3.11/site-packages/wee_import.py", line >> 899, in <module> >> hand() >> File >> "/home/sylvain/weewx-venv/lib/python3.11/site-packages/wee_import.py", line >> 829, in main >> source_obj.run() >> File >> "/home/sylvain/weewx-venv/lib/python3.11/site-packages/weeimport/weeimport.py", >> >> line 375, in run >> _raw_data = self.getRawData(period) >> ^^^^^^^^^^^^^^^^^^^^^ >> File >> "/home/sylvain/weewx-venv/lib/python3.11/site-packages/weeimport/wuimport.py", >> >> line 249, in getRawData >> _raw_data = f.read().decode(char_set) >> ^^^^^^^^^^^^^^^^^^^^^^^ >> TypeError: decode() argument 'encoding' must be str, not None >> >> >> What did I configure incorrectly? >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/063eaf28-e81e-4f9e-bb5c-6412434dfcecn%40googlegroups.com.
