Re: [weewx-user] Re: Wunderground import
SOLVED (Hopefully) Three steps to get this working. 1) As Gary pointed out. Use the cumulative rain (since midnight) to compute the rain since last update.So that the total rain _packet['rain'] is correct for the update period (in my case this is 5min). 2) Only send this information to weewx, ignore all the other rain parameters, i.e. let weewx handle that. 3) as per step 2) set in weewx.conf calculations section "rainRate" and "dayRain" to "software". Initial signs are looking good. Although after 24hrs of heavy rain here in West London we now have blue sky! Thanks for all you help and guidance Gary. Just need to extract my correct rain data from Wunderground and update weewx db... Pete. F.Y.I. here is a link to my PWS on wundergound. I have a webcam attached, note I have overlain with imagemagik output from weewx (info created using cheetah) on the webcam image. Hope you like it. https://www.wunderground.com/personal-weather-station/dashboard?ID=ILONDON542 On Fri, May 19, 2017 at 9:00 AM, Pete Nevill wrote: > Gary, > > wow... now I feel stupid... The answer was staring me in the face...(even > had a warning) > > def check_rain(self, daily_rain_counter): > # *** DO NOT use the &rainin= data! *** > # Handle the rain accum by taking the &dailyrainin= > reading ONLY. > # Then submit those minor increments of daily rain to > weewx. > rain = 0.0 > current_rain = float(daily_rain_counter) > if self.lastrain is not None: > if (current_rain >= self.lastrain): > rain = float(current_rain) - > float(self.lastrain) > self.lastrain = current_rain > return rain > > Although looking at this, is there an issue with flip after midnight? > Assuming it is raining and the bucket tipped just after midnight but before > the packet is sent we will have a situation where current_rain < > self.lastrain (but not zero). self.lastrain will be updated but rain will > be zero. Next packet will be the difference, and so miss the first bucket > tip of the day. Now on a 5 minute interval that I have set up I am sure > this is that significant but just thought I would raise it as a potential > problem. > > As such I added an else (where current_rain < self.lastrain, simply passes > total since midnight): > > rain = float(current_rain) > > I appreciate your comments regarding careful use of drivers etc as well as > space/indents in my python script. As you can tell I am enthusiastic > amateur...! > > I am still a little lost as to why my daily rain value reads zero after > forcing (faking) the rainrate (constant 2mm) for over 24hrs. > > Thanks again, > > Pete. > > On Wed, May 17, 2017 at 2:13 PM, gjr80 wrote: > >> Hi Pete, >> >> A few thoughts for you as you continue... >> >> I know you have just posted a small extract from your weewx.conf so we >> cannot see your complete setup, but be careful with your 'drivers'. The >> weeWX architecture basically supports one 'driver' and multiple 'services'. >> So for example some of the folks that have a PWS and one or more one-wire >> sensors might be running the driver for their PWS and a one-wire service to >> pick up data from their (separate to the PWS) one-wire sensors. It might be >> just semantics but something to keep in mind. If you are just using a >> socketlogger.py-based driver then I would expect that woudl be your driver >> and thought should be all you need weeWX wise. >> >> As mentioned previously, the weeWX rain field holds the total rainfall >> that ocurred over the period concerned. For loop data (ie data that comes >> in regualrly from your PWS, typically several times each archive period) >> this is the total rainfall since the last loop packet. For archive data (ie >> the accumulated data over an archive period, say 5 minutes) this is the >> total rainfall over the archive period. WeeWX can calculate rain rate from >> rain data but cannot calculate rain from rain rate data. If you look in the >> socketlogger driver you will see that each data packet it receives has a >> running total, the socketlogger keeps track of the previous running total >> value and calculates the difference (refer check_rain() method). This >> difference is what is yielded by the driver. >> >> 'Getting data into weeWX'. In its simplest form the driver decodes data >> from the data source (nominally a PWS), does any processing required (eg >> calculates rain from a running total) and the
Re: [weewx-user] Re: Wunderground import
> posted has a tabs in it. Python has some strict indentation rules and > whilst tabs can be made to work, it is likely that you will eventually trip > yourself up as spaces get mixed in with the tabs. Better to get yourself a > decent editor and set it to uses spaces only (when you hit the tab key). > > If you have not already read them you might find the following give some > insight into how weeWX works and how to customise it: > > - Customization guide - Introduction > <http://weewx.com/docs/customizing.htm#introduction> > - Customization Guide - Porting to new hardware > <http://weewx.com/docs/customizing.htm#porting> > - Notes for Developers <http://weewx.com/docs/devnotes.htm> > - various drivers, fileparse.py (examples/fileparse/bin/user/fileparse.py) > is a basic one others are in /home/weewx/bin/weewx/drivers, file size is > probably a good indicator of the simpler v more complex drivers. > > Hope this helps. > > Gary > > > On Wednesday, 17 May 2017 19:38:10 UTC+10, Pete Nevill wrote: >> >> Gary, >> >> Many thanks for your reply. This is very helpful. >> >> First issue fixed. units from Wunderground url as you say are in in/hr so >> I fixed that, i.e. /25.4. >> >> I have written my own driver based on one called socketlogge from here >> https://github.com/poblabs/weewx-socketlogger >> >> >> >> in my weewx.conf file i specify the driver and parameters. >> >> [W8681] >> # This section is for the W8681 wifi series of weather stations. >> >> # The driver to use: >> driver = weewx.drivers.w8681 >> >> [SocketLogger] >> # This section is for the SocketLogger driver >> >> # Station info: >> hardware = Generic intercept for wunderground urls >> >> # IP address, port number and timeout in seconds of the socket to connect to >> mode = sniff # driver mode either sniff of listen >> host_ip = XX.XX.XX.XX # in sniff mode ip of weather station else should be >> localhost for listen >> host_iface = eth1 # only required in sniff mode >> host_port = 80 # port to listen on or port to sniff >> timeout = 320 # timeout in seconds >> # this should be greater than the interval set on your pws >> >> # The driver to use >> driver = user.socketlogger >> >> >> I have attached the py script (still work in progress so use at own risk). >> Passes data to weewex with (for example) _packet['temp']. Not sure if this >> simply inserts data into the database or whether weewx actually "processes" >> the data. >> >> I am just trying to figure out how the rainrate is handled. >> Is weewx smart enough to know if I provide a rain rate of mm/hr every 5 >> minutes what the cumulative rainfall is? Just started running my script with >> a constant rainrate for 24hrs, will see what happens. So far the rain rate >> on the web interface is correct. Although the "Today's Rain" still says >> zero. Will let it run for a few days (we should have a total of 48mm >> tomorrow). >> >> >> Pete. >> >> >> >> >> >> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/weewx-user/4MWfKEqNO2A/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > weewx-user+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[weewx-user] Re: Wunderground import
Gary,
Many thanks for your reply. This is very helpful.
First issue fixed. units from Wunderground url as you say are in in/hr so I
fixed that, i.e. /25.4.
I have written my own driver based on one called socketlogge from here
https://github.com/poblabs/weewx-socketlogger
in my weewx.conf file i specify the driver and parameters.
[W8681]
# This section is for the W8681 wifi series of weather stations.
# The driver to use:
driver = weewx.drivers.w8681
[SocketLogger]
# This section is for the SocketLogger driver
# Station info:
hardware = Generic intercept for wunderground urls
# IP address, port number and timeout in seconds of the socket to connect to
mode = sniff # driver mode either sniff of listen
host_ip = XX.XX.XX.XX # in sniff mode ip of weather station else should be
localhost for listen
host_iface = eth1 # only required in sniff mode
host_port = 80 # port to listen on or port to sniff
timeout = 320 # timeout in seconds
# this should be greater than the interval set on your pws
# The driver to use
driver = user.socketlogger
I have attached the py script (still work in progress so use at own risk).
Passes data to weewex with (for example) _packet['temp']. Not sure if this
simply inserts data into the database or whether weewx actually "processes" the
data.
I am just trying to figure out how the rainrate is handled.
Is weewx smart enough to know if I provide a rain rate of mm/hr every 5 minutes
what the cumulative rainfall is? Just started running my script with a constant
rainrate for 24hrs, will see what happens. So far the rain rate on the web
interface is correct. Although the "Today's Rain" still says zero. Will let it
run for a few days (we should have a total of 48mm tomorrow).
Pete.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
### Instruction for packet sniffing
##
## You need set your router to send copies of wunderground packets to the server running weewx
## I did this on my router that runs dd-wrt by running the following under Administration->Commands
##
## iptables -A PREROUTING -t mangle -s 192.168.1.13 -j ROUTE --gw 192.168.1.24 --tee
##
## this tells the router to make a copy of any packets sent from the PWS (first ip) to the
## machine running weewx (second ip)
##
import logging
logging.getLogger("scapy.runtime").setLevel(logging.ERROR)
import socket
import syslog
from time import gmtime, strftime
import time
import urlparse
from scapy.all import IP, sniff
from scapy.layers import http
import string
import weedb
import weewx.drivers
import weeutil.weeutil
import weewx.wxformulas
def logmsg(dst, msg):
syslog.syslog(dst, 'SocketLogger: %s' % msg)
def loginf(msg):
logmsg(syslog.LOG_INFO, msg)
def logerror(msg):
logmsg(syslog.LOG_ERR, msg)
def logdebug(msg):
logmsg(syslog.LOG_DEBUG, msg)
def loader(config_dict, engine):
station = SocketLogger(**config_dict['SocketLogger'])
return station
def _fmt_bytes(data):
return ' '.join(['%02x' % ord(x) for x in data])
def makeDic():
dictionary = {
'tempf' : 'outTemp',
'outtemp' : 'outTemp',
'humidity' : 'outHumidity',
'outhumi' : 'outHumidity',
'inhumi': 'inHumidity',
'dewpoint' : 'dewpoint',
'dewptf': 'dewpoint',
'winddir' : 'windDir',
'windspeed' : 'WindSpeed',
'windspeedmph' : 'windSpeed',
'windgustmph' : 'windGust',
'windgust' : 'windGust',
'windgustdir' : 'windGustDir',
'windchill' : 'windchill',
'windchillf': 'windchill',
#'rainin': 'rainRate',
#'rainrate' : 'rainRate',
#'dailyrainin' : 'dayRain',
#'dailyrain' : 'dayRain',
#'yearlyrainin' : 'yearRain',
#'monthlyrainin' : 'monthRain',
#'monthlyrain' : 'monthlyrain',
'solarradiation' : 'radiation',
'UV': 'UV',
'indoortempf' : 'inTemp',
'intemp': 'InTemp',
'indoorhumidity' : 'inHumidity',
'baromin' : 'barometer',
'absbaromin': 'pressure',
'absbaro' : 'pressure',
'relbaro' : 'barometer',
'soiltempf' : 'soilTemp1',
'soiltemp2f': 'soilTemp2',
'soiltemp3f': 'soilTemp3',
'soiltemp4f': 'soilTemp4',
'soilmoisture' : 'soilMoist1',
'soilmoisture2' : 'soilMoist2',
'soilmoisture3' : 'soilMoist3',
'soilmoisture3' : 'soilMoist3',
'soilmoisture4' : 'soilMoist4',
'leafwetness' : 'leafWet1',
'leafwetness2' : 'leafWet2',
'lowbatt' : 'txBatteryStatus' }
return dictionary
class SocketLogger(weewx.drivers.AbstractDevice):
#def archive_interval
[weewx-user] Wunderground import
Hi, I have a WATSON W-8681 weather station which I have set up to send data direct to Wunderground every 5 minutes. In order to bring that data into my weewx database I use my router to redirect a copy of the wunderground url string to my server. I have written a user script to listen on a port for the wundergorund txt string. (This PWS is actually made by Fine Offset, just re-branded) http://www.foshk.com/Weather_Professional/HP1003.html The string looks like this... ID=X&PASSWORD=X&indoortempf=78.8&tempf=58.3&dewptf=55.0&windchillf=58.3&indoorhumidity=39&humidity=89&windspeedmph=6.5&windgustmph=7.4&winddir=179&absbaromin=30.17&baromin=30.36&rainin=0.00&dailyrainin=0.02&weeklyrainin=0.04&monthlyrainin=0.39&yearlyrainin=3.56&solarradiation=259.80&UV=2 I extract each of the values and map them to the associated weewx parameters. The trouble I am having is getting the right values for rain into weewx. I have mapped the following: Wunderground urlweewx 'rainin' to 'rainRate' dailyrain to 'dayRain' weeklyrainto * no weewx equivalent.* monthlyrainin to'monthRain' yearlyrainin to'yearRain' As far as I can tell rainin (wunderground) is current rain rate in mm/hr and daily rain is total rain since 12 mindnight. I am guessing either my mapping is wrong or the way these values are calculated within weewx is an issue. As you can see from the URL posted above dailyrain is 0.02 but my weewx is showing zero. I am guessing there are a few possible issues here. 1) correct mapping of parameters. 2) correct unit mapping/conversion 3) method for computation of daily 3) method for computation of rain rate 4) hardware versus software computation. Any guidance of how to get the correct rain data into weewx would be most appreciated. Thanks, Pete. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
