Hello Gary,
thanks fot your efforts. You are right with your description how my code should work. Let's say my main problem is that I know how I construct software in principle, but my python knowledgde is not very good, so in order to safe the last cumulative rain value my only solution was to write it into a file. Otherwise I tought the value might get lost between executing the data service I will try to 'buffer' my last rain value in a variable as you describe it and give you feedback. But I still think WeeWx does not accumulate correctly in my case, because im nearly 100% sure, that I have checked that my (indeed strange) code emits the values correctly. My schema looks like this - I think it is fine. > sqlite> .schema archive > CREATE TABLE archive (`dateTime` INTEGER NOT NULL UNIQUE PRIMARY KEY, > `usUnits` INTEGER NOT NULL, `interval` INTEGER NOT NULL, `barometer` REAL, > `pressure` REAL, `altimeter` REAL, `inTemp` REAL, `outTemp` REAL, > `inHumidity` REAL, `outHumidity` REAL, `windSpeed` REAL, `windDir` REAL, > `windGust` REAL, `windGustDir` REAL, `rainRate` REAL, `rain` REAL, > `dewpoint` REAL, `windchill` REAL, `heatindex` REAL, `ET` REAL, `radiation` > REAL, `UV` REAL, `extraTemp1` REAL, `extraTemp2` REAL, `extraTemp3` REAL, > `soilTemp1` REAL, `soilTemp2` REAL, `soilTemp3` REAL, `soilTemp4` REAL, > `leafTemp1` REAL, `leafTemp2` REAL, `extraHumid1` REAL, `extraHumid2` REAL, > `soilMoist1` REAL, `soilMoist2` REAL, `soilMoist3` REAL, `soilMoist4` REAL, > `leafWet1` REAL, `leafWet2` REAL, `rxCheckPercent` REAL, `txBatteryStatus` > REAL, `consBatteryVoltage` REAL, `hail` REAL, `hailRate` REAL, > `heatingTemp` REAL, `heatingVoltage` REAL, `supplyVoltage` REAL, > `referenceVoltage` REAL, `windBatteryStatus` REAL, `rainBatteryStatus` > REAL, `outTempBatteryStatus` REAL, `inTempBatteryStatus` REAL, > `AVG_Rcv_RX0` REAL, `AVG_Rcv_RX1` REAL, `AVG_Rcv_RX2` REAL, `BatVolt_0` > REAL, `SysTemp_0` REAL, `Rsfan_0` REAL, `PacketsSentPerHour_0` REAL, > `Snow_Height` REAL, `BatVolt_1` REAL, `SysTemp_1` REAL, `Rsfan_1` REAL, > `PacketsSentPerHour_1` REAL, `BatVolt_2` REAL, `SysTemp_2` REAL, `Rsfan_2` > REAL, `PacketsSentPerHour_2` REAL, `Soil_Temp_Full1` REAL, > `Soil_Temp_Full2` REAL, `Soil_Temp_Full3` REAL, `Soil_Temp_Full4` REAL, > `AQI_PM1_0` REAL, `AQI_PM2_5` REAL, `AQI_PM10_0` REAL, `AQI_Index` REAL, > `AQI_Temp` REAL, `AQI_Hum` REAL, `CO2` REAL, `GAS_2` REAL, `WiFi_T0` REAL, > `WiFi_H0` REAL, `Signal_Quality_TX0` REAL, `Signal_Quality_TX1` REAL, > `Signal_Quality_TX2` REAL, `Rain_RG11` REAL, `Rain_Rate_RG11` REAL, > `Rain_TX2` REAL, `Rain_Rate_TX2` REAL); > Thank you, engolling Am Samstag, 16. März 2019 07:44:10 UTC+1 schrieb gjr80: > > Hi, > > I had a look through your code and I think you have made things overly > complex for yourself. As I understand your setup you have a file, > /home/pi/WeatherDuino/WeeWx_Exp.txt, that contains your WeatherDuino > data, there are three rows of semicolon separated data, the rows being > labels/names, units and values. One of those fields is a cumulative rain > value. You also use the file /home/pi/WeatherDuino/Rain_tmp.txt that you > use in an elaborate arrangement to read and save the last cumulative rain > value and you obtain the delta rain value by taking the difference between > the current cumulative rain value and the value read from > /home/pi/WeatherDuino/Rain_tmp.txt. > > The logic appears to be sound but you are going about it in a very complex > manner and I suspect that is tripping you up. The need to determine delta > rain from a stream of cumulative rain values is common in many WeeWX > drivers. the approach taken in theses drivers is to create a property, > let's call it last_rain that is initialised to None in the drivers > __init__(). Then when the driver obtains a cumulative rain value, let's > call it cumulative_rain, there is a simple check whether last_rain is None, > if it is then delta_rain is set to None, if last_rain is not None the > delta_rain is set to cumulative_rain - last_rain. Finally last_rain is > set to cumulative_rain. In code it might look like this: > > > class SomeDriver(): > > def __init__(): > .... > self.last_rain = None > .... > > def get_delta_rain(cumulative_rain): > > if self.last_rain is not None: > delta_rain = cumulative_rain - self.last_rain > else: > delta_rain =None > self.last_rain = cumulative_rain > return delta_rain > > I have left a lot of things out there but the point was to illustrate the > overall logic and structure. By using the last_rain property there is no > need to write temporary values to file; your code is a lot neater, simpler > to understand and faster. The example I have used is for a driver, your > service, whilst not a driver, operates in the same manner in many respects > so you should be able to apply a similar construct in your service. You > don' need a separate get_delta_rain method if you don't want, you can put > it all in your read_file method. > > As for data not appearing in your database there are two conditions that > need to be met (1) your data needs ot appear in the archive record > generated by WeeWX and (2) the archive field name containing your data > needs to be included in the archive table schema. In your case your service > should take care of (1). At the moment I am not convinced that it is > operating as intended, but if you simplify/restructure your code as > suggested we will get there. As for (2) I am not convinced that your code > for setting the modified schema is doing as you think. Have you actually > looked in your database to see what schema has been implemented? You can > easily check the schema of a SQLite database table using the sqlite3 > utility (note you may need to install it on your machine using $ sudo > apt-get install sqlite3). To check the schema (assuming your database is > in /home/weewx/archive - it may be in /var/lib/weewx): > > $ sqlite3 /home/weewx/archive/weewx.sdb > sqlite> .schema archive > <schema will be listed here> > sqlite> .q > > What does the above show you, is field Rain_RG11 in the schema? > > Gary > > On Saturday, 16 March 2019 09:09:32 UTC+10, engolling wrote: >> >> Hi Gary, >> >> thanks for your very interesting reply. I tried to augment my extra data >> to the NEW_LOOP_PACKET, whichs works in priciple for all my data except the >> rain delta. >> As I mentioned it seems that the rain delta values are getting sorted out >> somehow. >> >> You can find the code here: >> https://github.com/menachers/WeatherDuino/tree/master/WeeWx_Plugin >> and an excerpt of my weewx.conf in an earlier post. >> >> So if I hardcode a deltarain of 0.1[in] it sums up correctly with each >> archive record - so after 5 minutes I get a total rain of 0.5[in]. If it is >> augmented to one loop packet (beeing emitted aproximatly every 3 seconds) i >> can also see it in the live display of the loop packet, but it can not be >> found in the archive record. So it seems that is is not accumulated >> internaly. >> >> Best regards, >> engolling >> >> Am Freitag, 15. März 2019 06:09:09 UTC+1 schrieb gjr80: >>> >>> Hi, >>> >>> When using a data service to add data to WeeWX you have two choices, you >>> either add your data to the archive records or you add your data to the >>> loop packets. Either way will work. If you decide that your data service is >>> to augment archive records with your rainfall data then your data service >>> must (1) bind to the event NEW_ARCHIVE_RECORD and (2) add the total >>> rainfall seen by your sensor during that archive period (in your case that >>> is the last 1 minute) to the archive record. If you decide your data >>> service is to augment the loop packets then your service must (1) bind to >>> the NEW_LOOP_PACKET event and (2) add the total rainfall seen since the >>> last loop packet you augmented to the loop packet concerned. Note that you >>> do not need to add your data to every loop packet, you can add it to >>> whichever loop packets you wish, but the value you add must be the total >>> rainfall since the last loop packet you augmented. The WeeWX internal >>> machinery will then take care of accumulating the loop data and your >>> accumulated data will appear in the archive record generated by WeeWX. >>> >>> Which way you decide to go is up to you. If you decide to augment the >>> archive record then your service would need to (in your case) obtain the >>> total rainfall for each 1 minute archive period, this may or may not be >>> easy depending on your hardware. The key here is you need to stick to the >>> archive period. If you decide to augment loop packets you can augment >>> whenever you want, you can skip loop packets if you want as long as you >>> stick to the rule of providing total rainfall since you last augmented a >>> loop packet. The loop packet approach is probably simpler, you just add >>> data when ever you can/want; you are not forced to rigidly stick to a fixed >>> interval as you are with the archive approach. >>> >>> In terms of saving data to the archive there are two conditions that >>> must be met. Firstly, the field/data you wish to save must appear in the >>> WeeWX generated archive record (you can see the WeeWX generated loop >>> packets and archive records by running weeWX directly >>> <http://weewx.com/docs/usersguide.htm#Running_directly>). Secondly, >>> your archive table schema must contain a field with the same name as the >>> field of interest in the archive record. So if your service added a field >>> named 'rain2' to the archive records (or loop packets) and the archive >>> table your WeeWX database had a field named 'rain2', then WeeWX would >>> automatically save your rain2 data to the archive. The steps involved in >>> changing your schema are covered in the Customization Guide >>> <http://weewx.com/docs/customizing.htm> under Adding a new type to the >>> database <http://weewx.com/docs/customizing.htm#add_archive_type>. Note >>> there are other approaches to saving new data to archive (eg creating a >>> second database) but these approaches are usually more complex or more >>> involved so I have ignored them in this case. >>> >>> You might also find that posting your code may help as that way we can >>> see exactly what you are/are not doing. >>> >>> Gary >>> >>> On Friday, 15 March 2019 08:45:03 UTC+10, engolling wrote: >>>> >>>> Short update: >>>> I have found out that my plugin is executed via data_services each time >>>> a loop package is generated and this is approximatly every 3 seconds. >>>> An archive dataset is written every 60 seconds, leading to chance of >>>> 1/20 that the raindelta is saved correctly - thats too less :-D >>>> >>>> So I see two possible solutions: >>>> 1. The data service is only executed once before the archiv dataset is >>>> written. Has anybody a idea how this could be done? >>>> >>>> 2. I have access to some kind of software flag saying that a archive >>>> record has been written. >>>> >>>> I had also a look into the vantage driver and I think this piece of >>>> code does the magic of calculating the data: >>>> # Because the Davis stations do not offer bucket tips in LOOP >>>> data, we >>>> # must calculate it by looking for changes in rain totals. >>>> This won't >>>> # work for the very first rain packet. >>>> if self.save_monthRain is None: >>>> delta = None >>>> else: >>>> delta = loop_packet['monthRain'] - self.save_monthRain >>>> # If the difference is negative, we're at the beginning of >>>> a month. >>>> if delta < 0: delta = None >>>> loop_packet['rain'] = delta >>>> self.save_monthRain = loop_packet['monthRain'] >>>> >>>> return loop_packet >>>> >>>> But I do not understand how overwriting the delta is prevented here. >>>> >>>> Hoping for some replys. >>>> >>>> Best wishes, >>>> engolling >>>> >>>> >>>> Am Sonntag, 3. März 2019 13:16:49 UTC+1 schrieb engolling: >>>>> >>>>> Hello, >>>>> I tried to implement a driver providing the rainfall in intervall but >>>>> until now it does not work as expected. >>>>> >>>>> I'am able to handle the correct data to WeeWx with this command: >>>>> syslog.syslog(syslog.LOG_DEBUG, "WeatherDuino: >>>>> " + str(names[n+1]) + ": " + str(deltarain)) >>>>> event.record[str(names[n+1])] = float( >>>>> deltarain) >>>>> The debug log output says: >>>>> >>>>>> Mar 3 11:56:16 WeatherDuinoPi weewx[1366]: WeatherDuino: Rain_RG11: >>>>>> 0.17716535433 >>>>>> >>>>> >>>>> But in the database there is a "0" logged. >>>>> >>>>> If i change the code hardcoding the rain intervall to 10 it works fine. >>>>> syslog.syslog(syslog.LOG_DEBUG, "WeatherDuino: >>>>> " + str(names[n+1]) + ": " + str(10)) >>>>> event.record[str(names[n+1])] = 10 >>>>> >>>>> So I think my driver is executed more often then my one minute logging >>>>> intervall and so the value of the event.record is overwritten with a zero >>>>> again - because the driver thinks the value is already in the WeeWx >>>>> database. >>>>> >>>>> You can find my code here: >>>>> https://github.com/menachers/WeatherDuino/tree/master/WeeWx_Plugin >>>>> >>>>> It is embedded in the weewx.conf: >>>>> # This section configures the internal weewx engine. >>>>> >>>>> [Engine] >>>>> >>>>> [[Services]] >>>>> # This section specifies the services that should be run. >>>>> They are >>>>> # grouped by type, and the order of services within each group >>>>> # determines the order in which the services will be run. >>>>> prep_services = weewx.engine.StdTimeSynch >>>>> data_services = user.WeeWx_WeatherDuino_Logger_plugin. >>>>> WeeWxService, >>>>> process_services = weewx.engine.StdConvert, weewx.engine. >>>>> StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate >>>>> archive_services = weewx.engine.StdArchive >>>>> restful_services = weewx.restx.StdStationRegistry, weewx.restx >>>>> .StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, >>>>> weewx.restx.StdWOW, weewx.restx.StdAWEKAS >>>>> report_services = weewx.engine.StdPrint, weewx.engine. >>>>> StdReport >>>>> >>>>> Regards, >>>>> engolling >>>>> >>>>> >>>>> Am Samstag, 23. Februar 2019 14:10:36 UTC+1 schrieb Andrew Milner: >>>>>> >>>>>> weewx requires rain during archive interval for storing in the >>>>>> database archive records. A driver may have to convert whatever it >>>>>> receives (eg running total) so as to obtain the value for the interval. >>>>>> Daily is accumulated by weewx from the archive interval values and >>>>>> weekly >>>>>> and monthly are derived form the daily totals. This is possibly an over >>>>>> simplification - but should give the idea. >>>>>> >>>>>> so if the second source gives a value for rainfall in interval it >>>>>> should be enough for weewx to derive the remainder. >>>>>> >>>>>> >>>>>> >>>>>> On Saturday, 23 February 2019 12:30:38 UTC+2, engolling wrote: >>>>>>> >>>>>>> Hello community, >>>>>>> >>>>>>> I would like to add an additional rain gauge as additional source >>>>>>> described in the customization guide. >>>>>>> Can you give me any hints how to do this the easiest way, to get the >>>>>>> daily, weekly and monthly percipation. >>>>>>> Can I use any modules or calculations that are already done inside >>>>>>> the system or is this normally done by the corresponding driver, so I >>>>>>> have >>>>>>> to handle all the signals precalculated. >>>>>>> >>>>>>> I hope you understand what I mean. >>>>>>> >>>>>>> Best Regards, >>>>>>> engolling >>>>>>> >>>>>> >>> >>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
