Hey, WeeWx'ers.
Even my workaround of setting pws_url to the value of the rf_url is
still getting 404's, even if I explicitly set a Host header. :-(((((
This is what I have tried... I guess WU is just a steaming pile as per
usual. :-/
Here is my attempted fix, which is part hack.
The "fix" is adding the header. The "hack" is forcing the code to use
the rtupdate URL (because I couldn't spot why the PWS URL was being used
for RF).
$ diff restx.py restx.py_getting404
100d99
< from urllib.parse import urlparse
454,458c453
< _parseduri = urlparse(url)
< _parsedhost = '{uri.netloc}'.format(uri=_parseduri)
< _request.add_header("Host", "%s" % _parsedhost)
< # log.debug("%s: Added Header: 'Host: %s'"
< # % (self.protocol_name, _parsedhost))
---
> _request.add_header("User-Agent", "weewx/%s" %
weewx.__version__)
610,611c605
< #pws_url =
"https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php";
< pws_url =
"https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php";
---
> pws_url =
"https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php";
Or in case your line numbers don't match mine (likely), here it is in
context:
...
# Python 2/3 compatiblity shims
import six
from six.moves import http_client
from six.moves import queue
from six.moves import urllib
from urllib.parse import urlparse
...
def get_request(self, url):
"""Get a request object. This can be overridden to add any
special headers."""
_request = urllib.request.Request(url)
_parseduri = urlparse(url)
_parsedhost = '{uri.netloc}'.format(uri=_parseduri)
_request.add_header("Host", "%s" % _parsedhost)
# log.debug("%s: Added Header: 'Host: %s'"
# % (self.protocol_name, _parsedhost))
_request.add_header("User-Agent", "weewx/%s" %
weewx.__version__)
return _request
...
# the rapidfire URL:
rf_url =
"https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php";
# the personal weather station URL:
#pws_url =
"https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php";
pws_url =
"https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php";
l...@isylum.org - Dearborn, Michigan
On 2/3/20 2:43 PM, Leon Shaner wrote:
Tom,
Looking at the code, under the do_rapidfire_post, we clearly see it
uses rf_url. In my debug output I referenced protocol_name and it's
showing Wunderground-RF as the protocol, but the hostname is from pws_url.
I have temporarily set pws_url to the rtupdate URL and my 404's have
stopped.
Looks like the rtupdate URL will take regular posts, but the
weatherstation URL won't take rf posts. :-/
My debug with pws_url replaced with the rf_url value now shows:
Feb 3 14:36:30 nixie weewx[8747] DEBUG weewx.restx:
Wunderground-RF: Added Header: 'Host: rtupdate.wunderground.com'
That's what it should have shown if rf_url was getting used for
do_rapidfire_post.
I'm stumped. :-(
Regards,
\Leon
l...@isylum.org - Dearborn, Michigan
On 2/3/20 2:23 PM, Leon Shaner wrote:
Brice,
Thanks! That's interesting... I just added an explicit Host header
to the rtupdate request and put debug output there.
This was unexpected:
Feb 3 14:16:34 nixie weewx[8584] DEBUG weewx.restx:
Wunderground-RF: Added Header: 'Host:
/*weatherstation.wunderground.com*/'
That isn't the host that should be used for rapidfire, but it's taken
from the URL being used. :-(
We have:
$ grep wunderground.com restx.py
rf_url =
"https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php";
pws_url =
"https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php";
It will take me some time to figure out why restx.py is using the
wrong URL.
Must have been doing this for a long time, but seems that due to
changes on the WU side, they no longer accept rapidfire updates to
the weatherstation.wunderground.com host.
Regards,
\Leon
l...@isylum.org - Dearborn, Michigan
On 2/3/20 12:59 PM, Brice Ruth wrote:
header would be something like `Host: rtupdate.wunderground.com
<http://rtupdate.wunderground.com>` ... basically, whatever you're
using as the hostname in the URL, needs to be sent as the `Host`
header ... that allows load-balancers to route the request to the
correct backend.
Brice Ruth, FCD
Software Engineer, Madison WI
On Mon, Feb 3, 2020 at 11:44 AM Leon Shaner <l...@isylum.org
<mailto:l...@isylum.org>> wrote:
Hey, WeeWx'ers,
I am still getting 404 errors from the rtupdate URL site.
There is a reply over here about it.
I'm now looking into what the headers need to be.
Does anyone know?
https://apicommunity.wunderground.com/weatherapi/topics/is-the-rtupdate-wunderground-com-address-no-longer-valid
For requests to route properly, the
host header must be set correctly
path must be correct
Regards,
\Leon
--
Leon Shaner :: Dearborn, Michigan (iPad)
On Feb 1, 2020, at 12:39 PM, Leon Shaner <l...@isylum.org
<mailto:l...@isylum.org>> wrote:
Hey, WeeWx'ers.
I took a different approach to working around the WU
certificate issue -- keep SSL but don't verify the CERT.
Scroll for that "solution."
But meanwhile, using Wunderground-RF (rapid fire) I am getting
an HTTP Error 404: Not Found.
Any ideas on that? Did they change the URL, or is it just down?
I noticed that even their own WunderStation App on iOS/iPadOS
is subject to the certificate issue and reports all stations as
"not reporting" -- even ones that are, like mine, which I can
verify via the WU website has the data and has reported within
minutes ago.
From the logs, re: RapidFire HTTP 404:
Feb 1 12:31:51 nixie weewx[13184] DEBUG weewx.restx:
Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Feb 1 12:31:51 nixie weewx[13184] ERROR weewx.restx:
Wunderground-RF: Failed to publish record 2020-02-01 12:31:51
EST (1580578311): Failed upload after 1 tries
Feb 1 12:31:56 nixie weewx[13184] DEBUG weewx.restx:
Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Feb 1 12:31:56 nixie weewx[13184] ERROR weewx.restx:
Wunderground-RF: Failed to publish record 2020-02-01 12:31:56
EST (1580578316): Failed upload after 1 tries
Feb 1 12:32:04 nixie weewx[13184] DEBUG weewx.restx:
Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Feb 1 12:32:04 nixie weewx[13184] ERROR weewx.restx:
Wunderground-RF: Failed to publish record 2020-02-01 12:32:04
EST (1580578324): Failed upload after 1 tries
Feb 1 12:32:05 nixie weewx[13184] DEBUG weewx.restx:
Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Feb 1 12:32:05 nixie weewx[13184] ERROR weewx.restx:
Wunderground-RF: Failed to publish record 2020-02-01 12:32:05
EST (1580578325): Failed upload after 1 tries
Feb 1 12:32:06 nixie weewx[13184] DEBUG weewx.restx:
Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Feb 1 12:32:06 nixie weewx[13184] ERROR weewx.restx:
Wunderground-RF: Failed to publish record 2020-02-01 12:32:05
EST (1580578325): Failed upload after 1 tries
Here is the diff for my SSL CERT workaround:
$ diff restx.py restx.py.20200201.1
110,115d109
< # SSL certificate hack
< global WUssl
< WUssl = ssl.create_default_context();
< WUssl.check_hostname=False
< WUssl.verify_mode=ssl.CERT_NONE
<
454d447
< _request.add_header("User-Agent", "weewx/%s" % weewx.__version__)
543,544c536
< # _response = urllib.request.urlopen(request,
data=data_bytes, timeout=self.timeout)
< _response = urllib.request.urlopen(request,
data=data_bytes, timeout=self.timeout, context=WUssl)
---
> _response = urllib.request.urlopen(request,
data=data_bytes, timeout=self.timeout)
1051,1052c1043
< # _response = urllib.request.urlopen(request,
timeout=self.timeout)
< _response = urllib.request.urlopen(request,
timeout=self.timeout, context=WUssl)
---
> _response = urllib.request.urlopen(request,
timeout=self.timeout)
Regards,
\Leon
--
Leon Shaner :: Dearborn, Michigan (iPad)
On Jan 31, 2020, at 4:16 PM, Thomas Keffer <tkef...@gmail.com
<mailto:tkef...@gmail.com>> wrote:
Travis: what is the error? If it's a certificate error,
Version 4 will do a retry after an hour. Unfortunately,
Version 3 does not.
On Fri, Jan 31, 2020 at 1:04 PM Travis Bully <tbu...@gmail.com
<mailto:tbu...@gmail.com>> wrote:
Not here yet. It'll work for awhile and then will get the
random cert error I sent earlier. A restart of weewx will
get it going again. I wish restx would just try again
after xx seconds.
On Friday, January 31, 2020 at 2:48:50 PM UTC-5, J B wrote:
Working here too. I had to restart Weewx.
On Friday, January 31, 2020 at 11:34:19 AM UTC-8,
Brice Ruth wrote:
Looking like it's working to me, too.
Brice Ruth, FCD
Software Engineer, Madison WI
On Fri, Jan 31, 2020 at 1:31 PM Travis Bully
<tbu...@gmail.com> wrote:
Agreed. Seems like I still get one cert error
every now and then. Likely
still propagating changes through their
infrastructure?
Jan 31 14:29:25 homeauto03 weewx[3784]: restx:
Wunderground-RF: Published record 2020-01-31
14:29:24 EST (1580498964)
Jan 31 14:29:26 homeauto03 weewx[3784]: restx:
Wunderground-RF: Failed upload attempt 1:
<urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify
failed (_ssl.c:727)>
Jan 31 14:29:31 homeauto03 weewx[3784]: restx:
Wunderground-RF: Failed to publish record
2020-01-31 14:29:26 EST (1580498966): Failed
upload after 1 tries
Jan 31 14:29:31 homeauto03 weewx[3784]: restx:
Wunderground-RF: Published record 2020-01-31
14:29:30 EST (1580498970)
On Fri, Jan 31, 2020 at 2:30 PM Denny Page
<splo...@gmail.com> wrote:
It's fixed now.
On Friday, January 31, 2020 at 8:54:02 AM
UTC-8, Denny Page wrote:
Wunderground just posted a note about
the intermediate certificate issue.
Hopefully they will fix it shortly.
--
You received this message because you are
subscribed to a topic in the Google Groups
"weewx-user" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/weewx-user/mYhw6CSKHHg/unsubscribe.
To unsubscribe from this group and all its
topics, send an email to
weewx...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are
subscribed to the Google Groups "weewx-user"
group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
weewx...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com
<https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to
the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the
Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com
<https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the
Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org
<https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com
<https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org
<https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to weewx-user+unsubscr...@googlegroups.com
<mailto:weewx-user+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org
<https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/45f04b3b-ae67-aaeb-9d25-625ea42b111b%40isylum.org.
