Re: [weewx-user] FilePile help please
Hi Thomas,
Done :) and it's been stable overnight, thanks again for the advice.
Colin
On Wed, May 8, 2019 at 1:59 AM Thomas Keffer wrote:
> You should terminate your lines with '\n', not '\r' when you write your
> file.
>
> while True:
> data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
> print "Received message:", data
> with open(fname, 'w') as f:
> for ds in data.split(','):
> f.write(ds + "\n")
>
> -tk
>
>
> On Tue, May 7, 2019 at 3:35 AM Colin Larsen
> wrote:
>
>> Thanks Gary appreciate the thoughts, I'll try those out tomorrow
>>
>> Colin
>>
>> On Tue, 7 May 2019, 20:23 gjr80, wrote:
>>
>>> A few thoughts. If it works sometimes and not others there must be a
>>> problem with either the consistency of the format of the data or the
>>> algorithm being used to parse the data. Can't say I am a big fan of slicing
>>> in a situation like this, I think it can be a bit rigid at times. Maybe
>>> consider using split. To to obtain your name, value pair maybe something
>>> like:
>>>
>>> ...
>>> for line in fd:
>>> data = line.split("=")
>>> if len(data) != 2:
>>> continue
>>> name,value = data
>>> new_record_data[self.label_map.get(name, name)] =
>>> to_float(value)
>>> ...
>>>
>>> or cut out the middle man if you like:
>>>
>>> ...
>>> for line in fd:
>>> data = line.split("=")
>>> if len(data) != 2:
>>> continue
>>> new_record_data[self.label_map.get(data[0], data[0])] =
>>> to_float(data[1])
>>> ...
>>>
>>> Gary
>>>
>>> On Tuesday, 7 May 2019 17:05:27 UTC+10, Colin Larsen wrote:
And this is the code the writes the txt file - capturing a string via
UDP
*#!/usr/bin/env python*
*import* socket
*from* shutil *import* copyfile
UDP_IP = *"0.0.0.0"*
UDP_PORT = 9886
fname = *"/home/pi/AQIData.txt"*
sock = socket.socket(socket.AF_INET,* # Internet*
socket.SOCK_DGRAM)* # UDP*
sock.bind((UDP_IP, UDP_PORT))
*while* True:
data, addr = sock.recvfrom(1024)* # buffer size is 1024 bytes*
*print* *"Received message:"*, data
dataSplit = data.split(*','*)
f = open(fname,*'w'*)
*for* ds *in* dataSplit:
f.write(ds + *"\r"*)
f.write(*'\n'*)
f.close()
On Tue, May 7, 2019 at 7:01 PM Colin Larsen wrote:
> Apparently #015 is a numeric representation of \r or 'return' to we
> can rule that out?
>
> On Tue, May 7, 2019 at 6:56 PM Colin Larsen
> wrote:
>
>> Well this is a little strange. This went fine for a while then
>> stopped again with pretty much the same error. It appears to be missing
>> the
>> beginning of the first line in the file which is AQI25 = then gets the
>> reading which is 4.55
>>
>> invalid literal for float(): 4.55#015AQI100 = 9.09#015AQIIndex =
>> 18#015AQICO2 = 863
>>
>> Is the #015 a clue of any sorts?
>>
>> The file looks ok
>>
>> AQI25 = 4.55
>>
>> AQI100 = 9.09
>>
>> AQIIndex = 18
>>
>> AQICO2 = 863
>>
>>
>> Error;
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: engine: Main loop exiting.
>> Shutting engine down.
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: engine: Caught
>> unrecoverable exception in engine:
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: invalid literal
>> for float(): 4.55#015AQI100 = 9.09#015AQIIndex = 18#015AQICO2 = 863
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: Traceback (most
>> recent call last):
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: File
>> "/usr/share/weewx/weewx/engine.py", line 890, in main
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: engine.run()
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: File
>> "/usr/share/weewx/weewx/engine.py", line 202, in run
>>
>> May 7 18:50:16 raspberrypi weewx[11960]:
>> self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: File
>> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>>
>> May 7 18:50:16 raspberrypi weewx[11960]:
>> callback(event)
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: File
>> "/usr/share/weewx/weewx/engine.py", line 574, in post_loop
>>
>> May 7 18:50:16 raspberrypi weewx[11960]:
>> self._software_catchup()
>>
>> May 7 18:50:16 raspberrypi weewx[11960]: File
>> "/usr/share/weewx/weewx/engine.py",
Re: [weewx-user] FilePile help please
You should terminate your lines with '\n', not '\r' when you write your
file.
while True:
data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
print "Received message:", data
with open(fname, 'w') as f:
for ds in data.split(','):
f.write(ds + "\n")
-tk
On Tue, May 7, 2019 at 3:35 AM Colin Larsen wrote:
> Thanks Gary appreciate the thoughts, I'll try those out tomorrow
>
> Colin
>
> On Tue, 7 May 2019, 20:23 gjr80, wrote:
>
>> A few thoughts. If it works sometimes and not others there must be a
>> problem with either the consistency of the format of the data or the
>> algorithm being used to parse the data. Can't say I am a big fan of slicing
>> in a situation like this, I think it can be a bit rigid at times. Maybe
>> consider using split. To to obtain your name, value pair maybe something
>> like:
>>
>> ...
>> for line in fd:
>> data = line.split("=")
>> if len(data) != 2:
>> continue
>> name,value = data
>> new_record_data[self.label_map.get(name, name)] =
>> to_float(value)
>> ...
>>
>> or cut out the middle man if you like:
>>
>> ...
>> for line in fd:
>> data = line.split("=")
>> if len(data) != 2:
>> continue
>> new_record_data[self.label_map.get(data[0], data[0])] =
>> to_float(data[1])
>> ...
>>
>> Gary
>>
>> On Tuesday, 7 May 2019 17:05:27 UTC+10, Colin Larsen wrote:
>>>
>>> And this is the code the writes the txt file - capturing a string via UDP
>>>
>>> *#!/usr/bin/env python*
>>>
>>> *import* socket
>>>
>>> *from* shutil *import* copyfile
>>>
>>>
>>> UDP_IP = *"0.0.0.0"*
>>>
>>> UDP_PORT = 9886
>>>
>>> fname = *"/home/pi/AQIData.txt"*
>>>
>>>
>>> sock = socket.socket(socket.AF_INET,* # Internet*
>>>
>>> socket.SOCK_DGRAM)* # UDP*
>>>
>>> sock.bind((UDP_IP, UDP_PORT))
>>>
>>>
>>>
>>> *while* True:
>>>
>>>
>>> data, addr = sock.recvfrom(1024)* # buffer size is 1024 bytes*
>>>
>>> *print* *"Received message:"*, data
>>>
>>> dataSplit = data.split(*','*)
>>>
>>> f = open(fname,*'w'*)
>>>
>>> *for* ds *in* dataSplit:
>>>
>>> f.write(ds + *"\r"*)
>>>
>>> f.write(*'\n'*)
>>>
>>> f.close()
>>>
>>> On Tue, May 7, 2019 at 7:01 PM Colin Larsen wrote:
>>>
Apparently #015 is a numeric representation of \r or 'return' to we
can rule that out?
On Tue, May 7, 2019 at 6:56 PM Colin Larsen wrote:
> Well this is a little strange. This went fine for a while then stopped
> again with pretty much the same error. It appears to be missing the
> beginning of the first line in the file which is AQI25 = then gets the
> reading which is 4.55
>
> invalid literal for float(): 4.55#015AQI100 = 9.09#015AQIIndex =
> 18#015AQICO2 = 863
>
> Is the #015 a clue of any sorts?
>
> The file looks ok
>
> AQI25 = 4.55
>
> AQI100 = 9.09
>
> AQIIndex = 18
>
> AQICO2 = 863
>
>
> Error;
>
> May 7 18:50:16 raspberrypi weewx[11960]: engine: Main loop exiting.
> Shutting engine down.
>
> May 7 18:50:16 raspberrypi weewx[11960]: engine: Caught
> unrecoverable exception in engine:
>
> May 7 18:50:16 raspberrypi weewx[11960]: invalid literal
> for float(): 4.55#015AQI100 = 9.09#015AQIIndex = 18#015AQICO2 = 863
>
> May 7 18:50:16 raspberrypi weewx[11960]: Traceback (most
> recent call last):
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 890, in main
>
> May 7 18:50:16 raspberrypi weewx[11960]: engine.run()
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 202, in run
>
> May 7 18:50:16 raspberrypi weewx[11960]:
> self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>
> May 7 18:50:16 raspberrypi weewx[11960]:
> callback(event)
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 574, in post_loop
>
> May 7 18:50:16 raspberrypi weewx[11960]:
> self._software_catchup()
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 646, in _software_catchup
>
> May 7 18:50:16 raspberrypi weewx[11960]:
> self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD,
> record=record, origin='software'))
>
> May 7 18:50:16 raspberrypi weewx[11960]: File
> "/usr/share/weewx/weewx/engine.py", line 224, in
Re: [weewx-user] FilePile help please
Thanks Gary appreciate the thoughts, I'll try those out tomorrow
Colin
On Tue, 7 May 2019, 20:23 gjr80, wrote:
> A few thoughts. If it works sometimes and not others there must be a
> problem with either the consistency of the format of the data or the
> algorithm being used to parse the data. Can't say I am a big fan of slicing
> in a situation like this, I think it can be a bit rigid at times. Maybe
> consider using split. To to obtain your name, value pair maybe something
> like:
>
> ...
> for line in fd:
> data = line.split("=")
> if len(data) != 2:
> continue
> name,value = data
> new_record_data[self.label_map.get(name, name)] = to_float
> (value)
> ...
>
> or cut out the middle man if you like:
>
> ...
> for line in fd:
> data = line.split("=")
> if len(data) != 2:
> continue
> new_record_data[self.label_map.get(data[0], data[0])] =
> to_float(data[1])
> ...
>
> Gary
>
> On Tuesday, 7 May 2019 17:05:27 UTC+10, Colin Larsen wrote:
>>
>> And this is the code the writes the txt file - capturing a string via UDP
>>
>> *#!/usr/bin/env python*
>>
>> *import* socket
>>
>> *from* shutil *import* copyfile
>>
>>
>> UDP_IP = *"0.0.0.0"*
>>
>> UDP_PORT = 9886
>>
>> fname = *"/home/pi/AQIData.txt"*
>>
>>
>> sock = socket.socket(socket.AF_INET,* # Internet*
>>
>> socket.SOCK_DGRAM)* # UDP*
>>
>> sock.bind((UDP_IP, UDP_PORT))
>>
>>
>>
>> *while* True:
>>
>>
>> data, addr = sock.recvfrom(1024)* # buffer size is 1024 bytes*
>>
>> *print* *"Received message:"*, data
>>
>> dataSplit = data.split(*','*)
>>
>> f = open(fname,*'w'*)
>>
>> *for* ds *in* dataSplit:
>>
>> f.write(ds + *"\r"*)
>>
>> f.write(*'\n'*)
>>
>> f.close()
>>
>> On Tue, May 7, 2019 at 7:01 PM Colin Larsen wrote:
>>
>>> Apparently #015 is a numeric representation of \r or 'return' to we can
>>> rule that out?
>>>
>>> On Tue, May 7, 2019 at 6:56 PM Colin Larsen wrote:
>>>
Well this is a little strange. This went fine for a while then stopped
again with pretty much the same error. It appears to be missing the
beginning of the first line in the file which is AQI25 = then gets the
reading which is 4.55
invalid literal for float(): 4.55#015AQI100 = 9.09#015AQIIndex =
18#015AQICO2 = 863
Is the #015 a clue of any sorts?
The file looks ok
AQI25 = 4.55
AQI100 = 9.09
AQIIndex = 18
AQICO2 = 863
Error;
May 7 18:50:16 raspberrypi weewx[11960]: engine: Main loop exiting.
Shutting engine down.
May 7 18:50:16 raspberrypi weewx[11960]: engine: Caught unrecoverable
exception in engine:
May 7 18:50:16 raspberrypi weewx[11960]: invalid literal
for float(): 4.55#015AQI100 = 9.09#015AQIIndex = 18#015AQICO2 = 863
May 7 18:50:16 raspberrypi weewx[11960]: Traceback (most
recent call last):
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 890, in main
May 7 18:50:16 raspberrypi weewx[11960]: engine.run()
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 202, in run
May 7 18:50:16 raspberrypi weewx[11960]:
self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
May 7 18:50:16 raspberrypi weewx[11960]: callback(event)
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 574, in post_loop
May 7 18:50:16 raspberrypi weewx[11960]:
self._software_catchup()
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 646, in _software_catchup
May 7 18:50:16 raspberrypi weewx[11960]:
self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD,
record=record, origin='software'))
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
May 7 18:50:16 raspberrypi weewx[11960]: callback(event)
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/user/filepile.py", line 91, in new_archive_record
May 7 18:50:16 raspberrypi weewx[11960]:
new_record_data[self.label_map.get(name,
name)] = to_float(value)
May 7 18:50:16 raspberrypi weewx[11960]: File
"/usr/share/weewx/weeutil/weeutil.py", line 1280, in
Re: [weewx-user] FilePile help please
A few thoughts. If it works sometimes and not others there must be a
problem with either the consistency of the format of the data or the
algorithm being used to parse the data. Can't say I am a big fan of slicing
in a situation like this, I think it can be a bit rigid at times. Maybe
consider using split. To to obtain your name, value pair maybe something
like:
...
for line in fd:
data = line.split("=")
if len(data) != 2:
continue
name,value = data
new_record_data[self.label_map.get(name, name)] = to_float(
value)
...
or cut out the middle man if you like:
...
for line in fd:
data = line.split("=")
if len(data) != 2:
continue
new_record_data[self.label_map.get(data[0], data[0])] =
to_float(data[1])
...
Gary
On Tuesday, 7 May 2019 17:05:27 UTC+10, Colin Larsen wrote:
>
> And this is the code the writes the txt file - capturing a string via UDP
>
> *#!/usr/bin/env python*
>
> *import* socket
>
> *from* shutil *import* copyfile
>
>
> UDP_IP = *"0.0.0.0"*
>
> UDP_PORT = 9886
>
> fname = *"/home/pi/AQIData.txt"*
>
>
> sock = socket.socket(socket.AF_INET,* # Internet*
>
> socket.SOCK_DGRAM)* # UDP*
>
> sock.bind((UDP_IP, UDP_PORT))
>
>
>
> *while* True:
>
>
> data, addr = sock.recvfrom(1024)* # buffer size is 1024 bytes*
>
> *print* *"Received message:"*, data
>
> dataSplit = data.split(*','*)
>
> f = open(fname,*'w'*)
>
> *for* ds *in* dataSplit:
>
> f.write(ds + *"\r"*)
>
> f.write(*'\n'*)
>
> f.close()
>
> On Tue, May 7, 2019 at 7:01 PM Colin Larsen > wrote:
>
>> Apparently #015 is a numeric representation of \r or 'return' to we can
>> rule that out?
>>
>> On Tue, May 7, 2019 at 6:56 PM Colin Larsen > > wrote:
>>
>>> Well this is a little strange. This went fine for a while then stopped
>>> again with pretty much the same error. It appears to be missing the
>>> beginning of the first line in the file which is AQI25 = then gets the
>>> reading which is 4.55
>>>
>>> invalid literal for float(): 4.55#015AQI100 = 9.09#015AQIIndex =
>>> 18#015AQICO2 = 863
>>>
>>> Is the #015 a clue of any sorts?
>>>
>>> The file looks ok
>>>
>>> AQI25 = 4.55
>>>
>>> AQI100 = 9.09
>>>
>>> AQIIndex = 18
>>>
>>> AQICO2 = 863
>>>
>>>
>>> Error;
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: engine: Main loop exiting.
>>> Shutting engine down.
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: engine: Caught unrecoverable
>>> exception in engine:
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: invalid literal for
>>> float(): 4.55#015AQI100 = 9.09#015AQIIndex = 18#015AQICO2 = 863
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: Traceback (most
>>> recent call last):
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 890, in main
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: engine.run()
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 202, in run
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]:
>>> self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: callback(event)
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 574, in post_loop
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]:
>>> self._software_catchup()
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 646, in _software_catchup
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]:
>>> self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD,
>>> record=record, origin='software'))
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: callback(event)
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/user/filepile.py", line 91, in new_archive_record
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]:
>>> new_record_data[self.label_map.get(name,
>>> name)] = to_float(value)
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: File
>>> "/usr/share/weewx/weeutil/weeutil.py", line 1280, in to_float
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: return float(x)
>>> if x is not None else None
>>>
>>> May 7 18:50:16 raspberrypi weewx[11960]: ValueError: invalid
>>> literal for float(): 4.55#015AQI100 = 9.09#015AQIIndex
Re: [weewx-user] FilePile help please
Hi Thomas
Thanks for the quick reply. It was just the format, it needs the spaces so
each line in the text file is
AQI25 = 1.60
AQI100 = 3.30
AQIIndex = 6
AQICO2 = 683
Once I did that and checked the database after an archive period the fields
were populated
Does my section with the units look valid? I've borrowed that from a
similar service but I can't wrap my head around how the relationship
between those and the data in the txt file get joined together
Thanks again
Colin
On Tue, May 7, 2019 at 3:13 PM Thomas Keffer wrote:
> Hi, Colin
>
> Does your text file use newlines ('\n') as line delineators? Perhaps it is
> using the MS-DOS standard of '\n\r'? Examine the file carefully and make
> sure it contains what you think it contains.
>
> -tk
>
> On Mon, May 6, 2019 at 8:05 PM Colin Larsen
> wrote:
>
>> Hi all
>>
>> I'm trying to get Filepile working but have come across an error that I
>> don't understand. Any help appreciated. The database has been extended with
>> fields to match those below in the data file - am I just missing spaces in
>> the layout? Should it be AQI25 = 1.60 etc etc
>>
>> Many thanks
>>
>> This is my data file;
>>
>> AQI25=1.60
>>
>> AQI100=3.30
>>
>> AQIIndex=6
>>
>> AQICO2=683
>>
>>
>> This is my stanza in weewx.conf
>>
>>
>> ##
>>
>>
>> # This section is for FilePile
>>
>>
>> [FilePile]
>>
>>
>> filename = /home/pi/AQIData.txt
>>
>> unit_system = METRIC
>>
>>
>>
>> ##
>>
>>
>> This is my filepile.py (modified from original to suit)
>>
>>
>> *import* syslog
>>
>> *import* weewx
>>
>> *import* weewx.units
>>
>> *from* weewx.wxengine *import* StdService
>>
>> *from* weeutil.weeutil *import* to_float
>>
>>
>> weewx.units.USUnits[*'group_gas_concentration'*] = *'ppm'*
>>
>> weewx.units.MetricUnits[*'group_gas_concentration'*] = *'ppm'*
>>
>> weewx.units.MetricWXUnits[*'group_gas_concentration'*] = *'ppm'*
>>
>> weewx.units.default_unit_format_dict[*'ppm'*] = *'%.0f'*
>>
>> weewx.units.default_unit_label_dict[*'ppm'*] = *' ppm'*
>>
>>
>> weewx.units.USUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
>>
>> weewx.units.MetricUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
>>
>> weewx.units.MetricWXUnits[*'group_dust'*] =
>> *'microgramm_per_meter_cubic'*
>>
>> weewx.units.default_unit_format_dict[*'microgramm_per_meter_cubic'*] =
>> *'%.1f'*
>>
>> weewx.units.default_unit_label_dict[*'microgramm_per_meter_cubic'*] = *'
>> \xce\xbcg/m\xc2\xb3'*
>>
>>
>> *class* FilePile(StdService):
>>
>> *"""WeeWX service for augmenting a record with data parsed from a
>> file."""*
>>
>>
>> *def** __init__*(self, engine, config_dict):
>>
>>* # Initialize my superclass:*
>>
>> super(FilePile, self).__init__(engine, config_dict)
>>
>> *# Extract our stanza from the configuration dicdtionary*
>>
>> filepile_dict = config_dict.get(*'FilePile'*, {})
>>
>>* # Get the location of the file ...*
>>
>> self.filename = filepile_dict.get(*'filename'*,
>> *'/var/tmp/filepile.txt'*)
>>
>>* # ... and the unit system it will use*
>>
>> unit_system_name = filepile_dict.get(*'unit_system'*,
>> *'METRICWX'*).strip().upper()
>>
>>* # Make sure we know about the unit system. If not, raise an
>> exception.*
>>
>> *if* unit_system_name *not* *in* weewx.units.unit_constants:
>>
>> *raise* ValueError(*"FilePile: Unknown unit system: %s"* %
>> unit_system_name)
>>
>>* # Use the numeric code for the unit system*
>>
>> self.unit_system = weewx.units.unit_constants[unit_system_name]
>>
>>
>>* # Mapping from variable names to weewx names*
>>
>> self.label_map = filepile_dict.get(*'label_map'*, {})
>>
>> syslog.syslog(syslog.LOG_INFO, *"filepile: Using %s with the
>> '%s' unit system"*
>>
>> % (self.filename, unit_system_name))
>>
>> syslog.syslog(syslog.LOG_INFO, *"filepile: Label map is %s"* %
>> self.label_map)
>>
>>
>>* # Bind to the NEW_ARCHIVE_RECORD event*
>>
>> self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
>>
>>
>> *def** new_archive_record*(self, event):
>>
>> new_record_data = {}
>>
>> *try*:
>>
>> *with* open(self.filename, *'r'*) *as* fd:
>>
>> *for* line *in* fd:
>>
>> eq_index = line.find(*'='*)
>>
>> *# Ignore all lines that do not have an equal sign*
>>
>> *if* eq_index == -1:
>>
>> *continue*
>>
>> name = line[:eq_index].strip()
>>
>> value = line[eq_index + 1:].strip()
>>
>> new_record_data[self.label_map.get(name, name)] =
>> to_float(value)
>>
>>* # Supply a unit system if one wasn't included in the
>> file*
>>
>>
Re: [weewx-user] FilePile help please
Hi, Colin
Does your text file use newlines ('\n') as line delineators? Perhaps it is
using the MS-DOS standard of '\n\r'? Examine the file carefully and make
sure it contains what you think it contains.
-tk
On Mon, May 6, 2019 at 8:05 PM Colin Larsen wrote:
> Hi all
>
> I'm trying to get Filepile working but have come across an error that I
> don't understand. Any help appreciated. The database has been extended with
> fields to match those below in the data file - am I just missing spaces in
> the layout? Should it be AQI25 = 1.60 etc etc
>
> Many thanks
>
> This is my data file;
>
> AQI25=1.60
>
> AQI100=3.30
>
> AQIIndex=6
>
> AQICO2=683
>
>
> This is my stanza in weewx.conf
>
>
> ##
>
>
> # This section is for FilePile
>
>
> [FilePile]
>
>
> filename = /home/pi/AQIData.txt
>
> unit_system = METRIC
>
>
>
> ##
>
>
> This is my filepile.py (modified from original to suit)
>
>
> *import* syslog
>
> *import* weewx
>
> *import* weewx.units
>
> *from* weewx.wxengine *import* StdService
>
> *from* weeutil.weeutil *import* to_float
>
>
> weewx.units.USUnits[*'group_gas_concentration'*] = *'ppm'*
>
> weewx.units.MetricUnits[*'group_gas_concentration'*] = *'ppm'*
>
> weewx.units.MetricWXUnits[*'group_gas_concentration'*] = *'ppm'*
>
> weewx.units.default_unit_format_dict[*'ppm'*] = *'%.0f'*
>
> weewx.units.default_unit_label_dict[*'ppm'*] = *' ppm'*
>
>
> weewx.units.USUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
>
> weewx.units.MetricUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
>
> weewx.units.MetricWXUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
>
> weewx.units.default_unit_format_dict[*'microgramm_per_meter_cubic'*] =
> *'%.1f'*
>
> weewx.units.default_unit_label_dict[*'microgramm_per_meter_cubic'*] = *'
> \xce\xbcg/m\xc2\xb3'*
>
>
> *class* FilePile(StdService):
>
> *"""WeeWX service for augmenting a record with data parsed from a
> file."""*
>
>
> *def** __init__*(self, engine, config_dict):
>
>* # Initialize my superclass:*
>
> super(FilePile, self).__init__(engine, config_dict)
>
> *# Extract our stanza from the configuration dicdtionary*
>
> filepile_dict = config_dict.get(*'FilePile'*, {})
>
>* # Get the location of the file ...*
>
> self.filename = filepile_dict.get(*'filename'*,
> *'/var/tmp/filepile.txt'*)
>
>* # ... and the unit system it will use*
>
> unit_system_name = filepile_dict.get(*'unit_system'*, *'METRICWX'*
> ).strip().upper()
>
>* # Make sure we know about the unit system. If not, raise an
> exception.*
>
> *if* unit_system_name *not* *in* weewx.units.unit_constants:
>
> *raise* ValueError(*"FilePile: Unknown unit system: %s"* %
> unit_system_name)
>
>* # Use the numeric code for the unit system*
>
> self.unit_system = weewx.units.unit_constants[unit_system_name]
>
>
>* # Mapping from variable names to weewx names*
>
> self.label_map = filepile_dict.get(*'label_map'*, {})
>
> syslog.syslog(syslog.LOG_INFO, *"filepile: Using %s with the '%s'
> unit system"*
>
> % (self.filename, unit_system_name))
>
> syslog.syslog(syslog.LOG_INFO, *"filepile: Label map is %s"* %
> self.label_map)
>
>
>* # Bind to the NEW_ARCHIVE_RECORD event*
>
> self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
>
>
> *def** new_archive_record*(self, event):
>
> new_record_data = {}
>
> *try*:
>
> *with* open(self.filename, *'r'*) *as* fd:
>
> *for* line *in* fd:
>
> eq_index = line.find(*'='*)
>
> *# Ignore all lines that do not have an equal sign*
>
> *if* eq_index == -1:
>
> *continue*
>
> name = line[:eq_index].strip()
>
> value = line[eq_index + 1:].strip()
>
> new_record_data[self.label_map.get(name, name)] =
> to_float(value)
>
>* # Supply a unit system if one wasn't included in the
> file*
>
> *if* *'usUnits'* *not* *in* new_record_data:
>
> new_record_data[*'usUnits'*] = self.unit_system
>
>* # Convert the new values to the same unit system as the
> record*
>
> target_data = weewx.units.to_std_system(new_record_data,
> event.record[*'usUnits'*])
>
>* # Add the converted values to the record:*
>
> event.record.update(target_data)
>
> *except* IOError *as* e:
>
>syslog.syslog(syslog.LOG_ERR, *"FilePile: Cannot open file.
> Reason: %s"* % e)
>
> This is the syslog error;
>
> May 7 14:50:16 raspberrypi weewx[6605]: engine: Caught unrecoverable
> exception in engine:
>
> May 7 14:50:16
[weewx-user] FilePile help please
Hi all
I'm trying to get Filepile working but have come across an error that I
don't understand. Any help appreciated. The database has been extended with
fields to match those below in the data file - am I just missing spaces in
the layout? Should it be AQI25 = 1.60 etc etc
Many thanks
This is my data file;
AQI25=1.60
AQI100=3.30
AQIIndex=6
AQICO2=683
This is my stanza in weewx.conf
##
# This section is for FilePile
[FilePile]
filename = /home/pi/AQIData.txt
unit_system = METRIC
##
This is my filepile.py (modified from original to suit)
*import* syslog
*import* weewx
*import* weewx.units
*from* weewx.wxengine *import* StdService
*from* weeutil.weeutil *import* to_float
weewx.units.USUnits[*'group_gas_concentration'*] = *'ppm'*
weewx.units.MetricUnits[*'group_gas_concentration'*] = *'ppm'*
weewx.units.MetricWXUnits[*'group_gas_concentration'*] = *'ppm'*
weewx.units.default_unit_format_dict[*'ppm'*] = *'%.0f'*
weewx.units.default_unit_label_dict[*'ppm'*] = *' ppm'*
weewx.units.USUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
weewx.units.MetricUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
weewx.units.MetricWXUnits[*'group_dust'*] = *'microgramm_per_meter_cubic'*
weewx.units.default_unit_format_dict[*'microgramm_per_meter_cubic'*] =
*'%.1f'*
weewx.units.default_unit_label_dict[*'microgramm_per_meter_cubic'*] = *'
\xce\xbcg/m\xc2\xb3'*
*class* FilePile(StdService):
*"""WeeWX service for augmenting a record with data parsed from a
file."""*
*def** __init__*(self, engine, config_dict):
* # Initialize my superclass:*
super(FilePile, self).__init__(engine, config_dict)
*# Extract our stanza from the configuration dicdtionary*
filepile_dict = config_dict.get(*'FilePile'*, {})
* # Get the location of the file ...*
self.filename = filepile_dict.get(*'filename'*,
*'/var/tmp/filepile.txt'*)
* # ... and the unit system it will use*
unit_system_name = filepile_dict.get(*'unit_system'*, *'METRICWX'*
).strip().upper()
* # Make sure we know about the unit system. If not, raise an
exception.*
*if* unit_system_name *not* *in* weewx.units.unit_constants:
*raise* ValueError(*"FilePile: Unknown unit system: %s"* %
unit_system_name)
* # Use the numeric code for the unit system*
self.unit_system = weewx.units.unit_constants[unit_system_name]
* # Mapping from variable names to weewx names*
self.label_map = filepile_dict.get(*'label_map'*, {})
syslog.syslog(syslog.LOG_INFO, *"filepile: Using %s with the '%s'
unit system"*
% (self.filename, unit_system_name))
syslog.syslog(syslog.LOG_INFO, *"filepile: Label map is %s"* %
self.label_map)
* # Bind to the NEW_ARCHIVE_RECORD event*
self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)
*def** new_archive_record*(self, event):
new_record_data = {}
*try*:
*with* open(self.filename, *'r'*) *as* fd:
*for* line *in* fd:
eq_index = line.find(*'='*)
*# Ignore all lines that do not have an equal sign*
*if* eq_index == -1:
*continue*
name = line[:eq_index].strip()
value = line[eq_index + 1:].strip()
new_record_data[self.label_map.get(name, name)] =
to_float(value)
* # Supply a unit system if one wasn't included in the file*
*if* *'usUnits'* *not* *in* new_record_data:
new_record_data[*'usUnits'*] = self.unit_system
* # Convert the new values to the same unit system as the
record*
target_data = weewx.units.to_std_system(new_record_data,
event.record[*'usUnits'*])
* # Add the converted values to the record:*
event.record.update(target_data)
*except* IOError *as* e:
syslog.syslog(syslog.LOG_ERR, *"FilePile: Cannot open file. Reason:
%s"* % e)
This is the syslog error;
May 7 14:50:16 raspberrypi weewx[6605]: engine: Caught unrecoverable
exception in engine:
May 7 14:50:16 raspberrypi weewx[6605]: invalid literal for
float(): 1.60#015AQI100=3.30#015AQIIndex=6#015AQICO2=683
May 7 14:50:16 raspberrypi weewx[6605]: Traceback (most recent
call last):
May 7 14:50:16 raspberrypi weewx[6605]: File
"/usr/share/weewx/weewx/engine.py", line 890, in main
May 7 14:50:16 raspberrypi weewx[6605]: engine.run()
May 7 14:50:16 raspberrypi weewx[6605]: File
"/usr/share/weewx/weewx/engine.py", line 202, in run
May 7 14:50:16 raspberrypi weewx[6605]:
