Re: [weewx-user] Belchertown MQTT Issues

[Doug Jenkins]

Matt:

for some reason I did not get the updates for this thread. I am glad the 
setup is now working for you! I checked your site and you do not have any 
javascript errors now :)


Vince : I agree, I will be more than happy to do a write up on how to 
configure this setup (with screenshots and all). I attached a word document 
to this thread a while back that listed what was needed in Cloudflare to 
offload SSL. I can do a formal write up with some commands and 
configuration from Pat O'Briens MQTT documentation. 

I can also post a message on Pat O'Briens discus chat on his weather site 
to see if he wants to add that as a post to his labs site (obrienlabs.net). 
That may help the majority of the skin users in setting this up on the 
first go.

Doug

On Monday, July 26, 2021 at 2:10:15 PM UTC-4 mattjo...@gmail.com wrote:

> Vince,
>
> I'd be happy to pull it all into one doc with some supporting notes so it 
> makes sense. This was a frustrating (at times) yet rewarding experience to 
> figure this all out and I know many others will want to achieve a similar 
> website and control of their data.
>
> Matt
>
> On Monday, July 26, 2021 at 12:27:51 PM UTC-4 vince wrote:
>
>> Given the long back+forth on this one, it would be really helpful if you 
>> could capture your entire setup someplace for the next person/victim who 
>> tries to do the same thing.  You have quite a lot of moving parts and it 
>> would be great to capture everything you did to set it up so it works both 
>> LAN and WAN.
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/d6747cf7-bc8d-4bf9-a5a3-ed88e0c599aen%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Vince,

I'd be happy to pull it all into one doc with some supporting notes so it 
makes sense. This was a frustrating (at times) yet rewarding experience to 
figure this all out and I know many others will want to achieve a similar 
website and control of their data.

Matt

On Monday, July 26, 2021 at 12:27:51 PM UTC-4 vince wrote:

> Given the long back+forth on this one, it would be really helpful if you 
> could capture your entire setup someplace for the next person/victim who 
> tries to do the same thing.  You have quite a lot of moving parts and it 
> would be great to capture everything you did to set it up so it works both 
> LAN and WAN.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/dd10f515-d693-4cf2-813f-bf4e7b3437f2n%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[vince]

Given the long back+forth on this one, it would be really helpful if you 
could capture your entire setup someplace for the next person/victim who 
tries to do the same thing.  You have quite a lot of moving parts and it 
would be great to capture everything you did to set it up so it works both 
LAN and WAN.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/46d4e4cb-1ecb-476b-8383-b5a8c939b23an%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Holy cow, no sooner did I send that did I see the glorious sight of 
"connected to weather station live" !!

Not sure what happened, maybe it just took a few minutes for Cloudflare to 
propagate the changes? In any case it is working and tested on LAN and WAN. 
I am so grateful for your support Doug

On Monday, July 26, 2021 at 10:46:10 AM UTC-4 Matt Johnson wrote:

> Doug,
>
> I went through your doc and updated everything at Cloudflare, my Nginx 
> Proxy Server and conf files and not working still...not sure what else it 
> could be.
>
> On Sunday, July 25, 2021 at 7:05:19 PM UTC-4 ln77 wrote:
>
>> Ah, ok thanks. I’m running the web server unencrypted as well, for the 
>> same reason.  
>>
>>   -Les
>>
>>
>> On Jul 25, 2021, at 4:14 AM, Karen K  wrote:
>>
>> ln77 schrieb am Sonntag, 25. Juli 2021 um 03:35:38 UTC+2:
>>
>>> I don’t see the point of SSL on the websockets connection from the 
>>> browser. The data is all public, there’s nothing sensitive sent by the 
>>> client, and if someone spoofs the mqtt broker, who really cares?
>>>
>>
>> The web browser cares. If it receives the website by HTTPS (SSL 
>> encrypted), then it refuses to connect to an unencrypted websockets server.
>>  
>>
>>>  
>>>
>> -- 
>>
>> You received this message because you are subscribed to the Google Groups 
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to weewx-user+...@googlegroups.com.
>>
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/weewx-user/08aa26d4-d986-4916-9c82-4c895c8b8fddn%40googlegroups.com
>>  
>> 
>> .
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/37e583e6-2cf2-445b-a0e9-9330653368den%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Doug,

I went through your doc and updated everything at Cloudflare, my Nginx 
Proxy Server and conf files and not working still...not sure what else it 
could be.

On Sunday, July 25, 2021 at 7:05:19 PM UTC-4 ln77 wrote:

> Ah, ok thanks. I’m running the web server unencrypted as well, for the 
> same reason.  
>
>   -Les
>
>
> On Jul 25, 2021, at 4:14 AM, Karen K  wrote:
>
> ln77 schrieb am Sonntag, 25. Juli 2021 um 03:35:38 UTC+2:
>
>> I don’t see the point of SSL on the websockets connection from the 
>> browser. The data is all public, there’s nothing sensitive sent by the 
>> client, and if someone spoofs the mqtt broker, who really cares?
>>
>
> The web browser cares. If it receives the website by HTTPS (SSL 
> encrypted), then it refuses to connect to an unencrypted websockets server.
>  
>
>>  
>>
> -- 
>
> You received this message because you are subscribed to the Google Groups 
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to weewx-user+...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/weewx-user/08aa26d4-d986-4916-9c82-4c895c8b8fddn%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/f15908cf-536e-4e0e-ad5c-0e43b72f16cfn%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Les Niles]

Ah, ok thanks. I’m running the web server unencrypted as well, for the same 
reason.  

  -Les


> On Jul 25, 2021, at 4:14 AM, Karen K  wrote:
> 
> ln77 schrieb am Sonntag, 25. Juli 2021 um 03:35:38 UTC+2:
>> I don’t see the point of SSL on the websockets connection from the browser. 
>> The data is all public, there’s nothing sensitive sent by the client, and if 
>> someone spoofs the mqtt broker, who really cares?
> 
> The web browser cares. If it receives the website by HTTPS (SSL encrypted), 
> then it refuses to connect to an unencrypted websockets server.
>  
>>  
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/weewx-user/08aa26d4-d986-4916-9c82-4c895c8b8fddn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/4C1AF58F-A4E6-413C-9724-01F248E382B0%402pi.org.

Re: [weewx-user] Belchertown MQTT Issues

[Karen K]

ln77 schrieb am Sonntag, 25. Juli 2021 um 03:35:38 UTC+2:

> I don’t see the point of SSL on the websockets connection from the 
> browser. The data is all public, there’s nothing sensitive sent by the 
> client, and if someone spoofs the mqtt broker, who really cares?
>

The web browser cares. If it receives the website by HTTPS (SSL encrypted), 
then it refuses to connect to an unencrypted websockets server.
 

>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/08aa26d4-d986-4916-9c82-4c895c8b8fddn%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Les Niles]

I don’t see the point of SSL on the websockets connection from the browser. The 
data is all public, there’s nothing sensitive sent by the client, and if 
someone spoofs the mqtt broker, who really cares?

Neither do I see much need for ssl on the weewx-to-broker connection when 
they’re running on the same machine, or even on an internal network. I can 
imagine scenarios where not using ssl allows a vulnerability, but I think the 
likelihood is low if the internal network itself is well secured, so I haven’t 
bothered with ssl. It would be different with a cloud-hosted or other external 
mqtt broker with the weewx-mqtt connection running over a public network. 

  -Les


> On Jul 24, 2021, at 9:33 AM, Matt Johnson  wrote:
> 
> 
> Doing more trouble shooting. I have narrowed it down to a SSL issue. 
> 
> To confirm this I disabled SSL at cloudflare and disabled force SSL for 
> shakerweather.com with my NGINX reverse proxy server and ran with this setup:
> 
> weewx.config
> [[MQTT]]
> server_url = mqtt://usr:p...@mqtt.shakerweather.com:1883/
> 
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
> 
> skin.conf
># MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "mqtt.shakerweather.com"
> mqtt_websockets_port = 9001
> mqtt_websockets_ssl = 0
> mqtt_websockets_topic = "weather/loop"
> disconnect_live_website_visitor = 180
> 
> mosquitto.conf
> persistence false
> 
> # mqtt
> listener 1883
> protocol mqtt
> 
> # websockets
> listener 9001
> protocol websockets
> 
> allow_anonymous true
> password_file /etc/mosquitto/passwd
> acl_file /etc/mosquitto/acl
> 
> 
> I am able to see the real time websocket updates internally and externally at 
> http://shakerweather.com (I'll leave it up for a short time so if it doesn't 
> work if you click on it that could be why)
> 
> So at least I have narrowed down the issue. Still not sure how to resolve to 
> ensure all is secured with SSL
>> On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:
>> Your setup seems similar to mine so I went for it. Still not working. 
>> 
>> I went ahead and added a mqtt CNAME record to the shakerweather.com 
>> cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy 
>> Server to forward to port 9001 using the same SSL certificate I have for 
>> shakerweather.com that is forwarding to port 80 to serve the page.
>> 
>> Cloudflare:
>> TYPE  NAME CONTENT
>> CNAME  mqttshakerweather.com
>> A shakerweather.com static wan IP
>> 
>> weewx.conf
>> [[MQTT]]
>> server_url = mqtt://user:p...@10.0.0.30:1883/
>> 
>> topic = weather
>> unit_system = US
>> binding = archive, loop 
>> aggregation = aggregate
>> 
>> skin.conf
>>  # MQTT Websockets defaults
>> mqtt_websockets_enabled = 1
>> mqtt_websockets_host = "mqtt.shakerweather.com"
>> mqtt_websockets_port = 443
>> 
>> mqtt_websockets_ssl = 1
>> mqtt_websockets_topic = "weather/loop"
>> disconnect_live_website_visitor = 180
>> 
>> mosquitto.conf
>> persistence false
>> 
>> # mqtt
>> listener 1883
>> protocol mqtt
>> 
>> # websockets
>> listener 9001
>> protocol websockets
>> 
>> 
>> allow_anonymous true
>> password_file /etc/mosquitto/passwd
>> 
>> acl_file /etc/mosquitto/acl
>>> On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 do...@dougjenkins.com wrote:
>>> I had a number of struggles with this when I setup the Belchertown skin on 
>>> my Raspberry Pi hosting my weather site, www.largoweather.com. I think the 
>>> issue is that you need your mqtt_websockets_port to be set to 443 as the 
>>> websocket traffic is getting filtered out by your firewall (9001->8083)
>>> 
>>> Here is what I did to get this to work on largoweather.com:
>>> 
>>> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
>>>- largoweather.com (A Record)
>>>- wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record 
>>> pointing the content to largoweather.com
>>>- Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
>>> processing so essentially all traffic is coming in through 443.
>>>
>>>These are pointed to my public IP address which is dynamic. I use a 
>>> shell script to update cloudflare' content to keep my public IP current for 
>>> their system
>>> 
>>> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
>>> that acts as a reverse proxy manager for my domains:
>>>  2.1 : Setup:
>>> - largoweather.com : Setup to point to my local server ip 
>>> address. The scheme is http. I use the program's lets encrypt function to 
>>> get a SSL certificate and force SSL traffic to my final weewx website.
>>>  - wx.largoweather.com : This handles my sockets setup. I 
>>> forward all traffic to port 9001 and use the same ssl certi

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Thanks for this detail Doug, I will try again. Your analysis is spot on as 
I did disable SSL temporarily to see if the websockets would in fact work 
that way confiming it is an SSL issue. If you go to unsecured http (
http://shakerweather.com) you should see it temporarily working with 
unsecured websocket updates (until I go back in and try to fix it the 
correct way with SSL this afternoon)

Light at the end of the tunnel. I will post later today when I either reach 
success or get stuck again.


On Saturday, July 24, 2021 at 1:05:12 PM UTC-4 do...@dougjenkins.com wrote:

> I just checked your site in edge and pulled up the dev console. I see the 
> main issue is that the client (me) is trying to access a non-secure 
> websocket (your server) while the rest of the content is coming from the 
> https stream:
>
> *paho-mqtt.min.js:37 Mixed Content: The page at 
> 'https://shakerweather.com/ ' was loaded over 
> HTTPS, but attempted to connect to the insecure WebSocket endpoint 
> 'ws://mqtt.shakerweather.com:9001/mqtt 
> '. This request has been blocked; 
> this endpoint must be available over WSS.*
>
> I had this problem before and what I to solve it was to have CloudFlare to 
> translate all my traffic to SSL. 
>
> Here is my Cloudflare and NGINX Proxy Manager setup that I use for 
> largoweather.com. Essentially if you check my SSL Cert, it is provided by 
> Cloudflare. Cloudflare pushes all http traffic as https to my internal 
> server. I have firewall rules to route all incoming 80/443 traffic to my 
> internal server in which that is properly routed to my NGINX Proxy Manager 
> instance.
>
> NGINX Proxy manager proxies that traffic to the correct webserver (in my 
> case container) that hosts my weewx website. I did issue a LetsEncrypt SSL 
> within NGINX Proxy manager to get that to work.
>
> They trick with LetsEncrypt is that you need to expose (just for a second) 
> your public IP Address in Cloudflare (assuming that is your name server of 
> the domain) for the SSL Certificate to be created correctly. Once you do 
> that, you can setup Proxy again to protect your public IP.
>
> Once this is setup, you will immediately see the live updates working. 
>
> On Saturday, July 24, 2021 at 12:36:18 PM UTC-4 Doug Jenkins wrote:
>
>> Matt:
>>
>> I am looking into the issue now. I think you need to offload the SSL to 
>> Cloudflare with NGNIX Proxy. I am capturing screenshots of my setup to see 
>> if that will help you.
>>
>> Doug Jenkins
>>
>> On Saturday, July 24, 2021 at 12:33:22 PM UTC-4 mattjo...@gmail.com 
>> wrote:
>>
>>> Doing more trouble shooting. I have narrowed it down to a SSL issue. 
>>>
>>> To confirm this I disabled SSL at cloudflare and disabled force SSL for 
>>> shakerweather.com with my NGINX reverse proxy server and ran with this 
>>> setup:
>>>
>>> *weewx.config*
>>> [[MQTT]]
>>> server_url = mqtt://usr:p...@mqtt.shakerweather.com:1883/
>>>
>>> topic = weather
>>> unit_system = US
>>> binding = archive, loop 
>>> aggregation = aggregate
>>>
>>> *skin.conf*
>>># MQTT Websockets defaults
>>> mqtt_websockets_enabled = 1
>>> mqtt_websockets_host = "mqtt.shakerweather.com"
>>> mqtt_websockets_port = 9001
>>> mqtt_websockets_ssl = 0
>>> mqtt_websockets_topic = "weather/loop"
>>> disconnect_live_website_visitor = 180
>>>
>>> *mosquitto.conf*
>>> persistence false
>>>
>>> # mqtt
>>> listener 1883
>>> protocol mqtt
>>>
>>> # websockets
>>> listener 9001
>>> protocol websockets
>>>
>>> allow_anonymous true
>>> password_file /etc/mosquitto/passwd
>>> acl_file /etc/mosquitto/acl
>>>
>>>
>>> I am able to see the real time websocket updates internally and 
>>> externally at http://shakerweather.com (I'll leave it up for a short 
>>> time so if it doesn't work if you click on it that could be why)
>>>
>>> So at least I have narrowed down the issue. Still not sure how to 
>>> resolve to ensure all is secured with SSL
>>> On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:
>>>
 Your setup seems similar to mine so I went for it. Still not working. 

 I went ahead and added a mqtt CNAME record to the shakerweather.com 
 cloudflare setup. I also added mqtt.shakerweather.com to my NGINX 
 Proxy Server to forward to port 9001 using the same SSL certificate I have 
 for shakerweather.com that is forwarding to port 80 to serve the page.

 Cloudflare:
 TYPE  NAME CONTENT
 CNAME  mqttshakerweather.com
 A shakerweather.com static wan IP

 *weewx.conf*
 [[MQTT]]
 server_url = mqtt://user:p...@10.0.0.30:1883/ 
 

 topic = weather
 unit_system = US
 binding = archive, loop 
 aggregation = aggregate

 *skin.conf*
  # 

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Thanks for this detail Doug, I will try again. Your analysis is spot on as 
I did disable SSL temporarily to see if the websockets would in fact work 
that way confiming it is an SSL issue. If you go to unsecured http 
(http://shakerweather.com) you should see it all working with websocket 
updates (until I go back in and try to fix it the correct way with SSL this 
afternoon)

Light at the end of the tunnel. I will post later today when I either reach 
success or get stuck again.
On Saturday, July 24, 2021 at 1:05:12 PM UTC-4 do...@dougjenkins.com wrote:

> I just checked your site in edge and pulled up the dev console. I see the 
> main issue is that the client (me) is trying to access a non-secure 
> websocket (your server) while the rest of the content is coming from the 
> https stream:
>
> *paho-mqtt.min.js:37 Mixed Content: The page at 
> 'https://shakerweather.com/ ' was loaded over 
> HTTPS, but attempted to connect to the insecure WebSocket endpoint 
> 'ws://mqtt.shakerweather.com:9001/mqtt 
> '. This request has been blocked; 
> this endpoint must be available over WSS.*
>
> I had this problem before and what I to solve it was to have CloudFlare to 
> translate all my traffic to SSL. 
>
> Here is my Cloudflare and NGINX Proxy Manager setup that I use for 
> largoweather.com. Essentially if you check my SSL Cert, it is provided by 
> Cloudflare. Cloudflare pushes all http traffic as https to my internal 
> server. I have firewall rules to route all incoming 80/443 traffic to my 
> internal server in which that is properly routed to my NGINX Proxy Manager 
> instance.
>
> NGINX Proxy manager proxies that traffic to the correct webserver (in my 
> case container) that hosts my weewx website. I did issue a LetsEncrypt SSL 
> within NGINX Proxy manager to get that to work.
>
> They trick with LetsEncrypt is that you need to expose (just for a second) 
> your public IP Address in Cloudflare (assuming that is your name server of 
> the domain) for the SSL Certificate to be created correctly. Once you do 
> that, you can setup Proxy again to protect your public IP.
>
> Once this is setup, you will immediately see the live updates working. 
>
> On Saturday, July 24, 2021 at 12:36:18 PM UTC-4 Doug Jenkins wrote:
>
>> Matt:
>>
>> I am looking into the issue now. I think you need to offload the SSL to 
>> Cloudflare with NGNIX Proxy. I am capturing screenshots of my setup to see 
>> if that will help you.
>>
>> Doug Jenkins
>>
>> On Saturday, July 24, 2021 at 12:33:22 PM UTC-4 mattjo...@gmail.com 
>> wrote:
>>
>>> Doing more trouble shooting. I have narrowed it down to a SSL issue. 
>>>
>>> To confirm this I disabled SSL at cloudflare and disabled force SSL for 
>>> shakerweather.com with my NGINX reverse proxy server and ran with this 
>>> setup:
>>>
>>> *weewx.config*
>>> [[MQTT]]
>>> server_url = mqtt://usr:p...@mqtt.shakerweather.com:1883/
>>>
>>> topic = weather
>>> unit_system = US
>>> binding = archive, loop 
>>> aggregation = aggregate
>>>
>>> *skin.conf*
>>># MQTT Websockets defaults
>>> mqtt_websockets_enabled = 1
>>> mqtt_websockets_host = "mqtt.shakerweather.com"
>>> mqtt_websockets_port = 9001
>>> mqtt_websockets_ssl = 0
>>> mqtt_websockets_topic = "weather/loop"
>>> disconnect_live_website_visitor = 180
>>>
>>> *mosquitto.conf*
>>> persistence false
>>>
>>> # mqtt
>>> listener 1883
>>> protocol mqtt
>>>
>>> # websockets
>>> listener 9001
>>> protocol websockets
>>>
>>> allow_anonymous true
>>> password_file /etc/mosquitto/passwd
>>> acl_file /etc/mosquitto/acl
>>>
>>>
>>> I am able to see the real time websocket updates internally and 
>>> externally at http://shakerweather.com (I'll leave it up for a short 
>>> time so if it doesn't work if you click on it that could be why)
>>>
>>> So at least I have narrowed down the issue. Still not sure how to 
>>> resolve to ensure all is secured with SSL
>>> On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:
>>>
 Your setup seems similar to mine so I went for it. Still not working. 

 I went ahead and added a mqtt CNAME record to the shakerweather.com 
 cloudflare setup. I also added mqtt.shakerweather.com to my NGINX 
 Proxy Server to forward to port 9001 using the same SSL certificate I have 
 for shakerweather.com that is forwarding to port 80 to serve the page.

 Cloudflare:
 TYPE  NAME CONTENT
 CNAME  mqttshakerweather.com
 A shakerweather.com static wan IP

 *weewx.conf*
 [[MQTT]]
 server_url = mqtt://user:p...@10.0.0.30:1883/ 
 

 topic = weather
 unit_system = US
 binding = archive, loop 
 aggregation = aggregate

 *skin.conf*
  # MQTT Websockets defa

Re: [weewx-user] Belchertown MQTT Issues

[Doug Jenkins]

Matt:

I am looking into the issue now. I think you need to offload the SSL to 
Cloudflare with NGNIX Proxy. I am capturing screenshots of my setup to see 
if that will help you.

Doug Jenkins

On Saturday, July 24, 2021 at 12:33:22 PM UTC-4 mattjo...@gmail.com wrote:

> Doing more trouble shooting. I have narrowed it down to a SSL issue. 
>
> To confirm this I disabled SSL at cloudflare and disabled force SSL for 
> shakerweather.com with my NGINX reverse proxy server and ran with this 
> setup:
>
> *weewx.config*
> [[MQTT]]
> server_url = mqtt://usr:p...@mqtt.shakerweather.com:1883/
>
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
>
> *skin.conf*
># MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "mqtt.shakerweather.com"
> mqtt_websockets_port = 9001
> mqtt_websockets_ssl = 0
> mqtt_websockets_topic = "weather/loop"
> disconnect_live_website_visitor = 180
>
> *mosquitto.conf*
> persistence false
>
> # mqtt
> listener 1883
> protocol mqtt
>
> # websockets
> listener 9001
> protocol websockets
>
> allow_anonymous true
> password_file /etc/mosquitto/passwd
> acl_file /etc/mosquitto/acl
>
>
> I am able to see the real time websocket updates internally and externally 
> at http://shakerweather.com (I'll leave it up for a short time so if it 
> doesn't work if you click on it that could be why)
>
> So at least I have narrowed down the issue. Still not sure how to resolve 
> to ensure all is secured with SSL
> On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:
>
>> Your setup seems similar to mine so I went for it. Still not working. 
>>
>> I went ahead and added a mqtt CNAME record to the shakerweather.com 
>> cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy 
>> Server to forward to port 9001 using the same SSL certificate I have for 
>> shakerweather.com that is forwarding to port 80 to serve the page.
>>
>> Cloudflare:
>> TYPE  NAME CONTENT
>> CNAME  mqttshakerweather.com
>> A shakerweather.com static wan IP
>>
>> *weewx.conf*
>> [[MQTT]]
>> server_url = mqtt://user:p...@10.0.0.30:1883/ 
>> 
>>
>> topic = weather
>> unit_system = US
>> binding = archive, loop 
>> aggregation = aggregate
>>
>> *skin.conf*
>>  # MQTT Websockets defaults
>> mqtt_websockets_enabled = 1
>> mqtt_websockets_host = "mqtt.shakerweather.com"
>> mqtt_websockets_port = 443
>>
>> mqtt_websockets_ssl = 1
>> mqtt_websockets_topic = "weather/loop"
>> disconnect_live_website_visitor = 180
>>
>> *mosquitto.conf*
>> persistence false
>>
>> # mqtt
>> listener 1883
>> protocol mqtt
>>
>> # websockets
>> listener 9001
>> protocol websockets
>>
>>
>> allow_anonymous true
>> password_file /etc/mosquitto/passwd
>>
>> acl_file /etc/mosquitto/acl
>> On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 do...@dougjenkins.com 
>> wrote:
>>
>>> I had a number of struggles with this when I setup the Belchertown skin 
>>> on my Raspberry Pi hosting my weather site, www.largoweather.com. I 
>>> think the issue is that you need your mqtt_websockets_port to be set to 443 
>>> as the websocket traffic is getting filtered out by your firewall 
>>> (9001->8083)
>>>
>>> Here is what I did to get this to work on largoweather.com:
>>>
>>> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
>>>- largoweather.com (A Record)
>>>- wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME 
>>> Record pointing the content to largoweather.com
>>>- Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
>>> processing so essentially all traffic is coming in through 443.
>>>
>>>These are pointed to my public IP address which is dynamic. I use a 
>>> shell script to update cloudflare' content to keep my public IP current for 
>>> their system
>>>
>>> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
>>> that acts as a reverse proxy manager for my domains:
>>>  2.1 : Setup:
>>> - largoweather.com : Setup to point to my local server ip 
>>> address. The scheme is http. I use the program's lets encrypt function to 
>>> get a SSL certificate and force SSL traffic to my final weewx website.
>>>  - wx.largoweather.com : This handles my sockets setup. I 
>>> forward all traffic to port 9001 and use the same ssl certificate issued 
>>> for largoweather. I force SSL on this setup as well.
>>>
>>> 3. mosquitto configuration : I setup mosquitto on the same server as my 
>>> weewx install since everything is running on the pi. here is my 
>>> mosquitto.conf:
>>>pid_file /var/run/mosquitto.pid
>>>persistence true
>>>persistence_location /var/lib/mosquitto/
>>>log_type error
>>>websockets

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Doing more trouble shooting. I have narrowed it down to a SSL issue. 

To confirm this I disabled SSL at cloudflare and disabled force SSL for 
shakerweather.com with my NGINX reverse proxy server and ran with this 
setup:

*weewx.config*
[[MQTT]]
server_url = mqtt://usr:p...@mqtt.shakerweather.com:1883/

topic = weather
unit_system = US
binding = archive, loop 
aggregation = aggregate

*skin.conf*
   # MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.shakerweather.com"
mqtt_websockets_port = 9001
mqtt_websockets_ssl = 0
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 180

*mosquitto.conf*
persistence false

# mqtt
listener 1883
protocol mqtt

# websockets
listener 9001
protocol websockets

allow_anonymous true
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl


I am able to see the real time websocket updates internally and externally 
at http://shakerweather.com (I'll leave it up for a short time so if it 
doesn't work if you click on it that could be why)

So at least I have narrowed down the issue. Still not sure how to resolve 
to ensure all is secured with SSL
On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:

> Your setup seems similar to mine so I went for it. Still not working. 
>
> I went ahead and added a mqtt CNAME record to the shakerweather.com 
> cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy 
> Server to forward to port 9001 using the same SSL certificate I have for 
> shakerweather.com that is forwarding to port 80 to serve the page.
>
> Cloudflare:
> TYPE  NAME CONTENT
> CNAME  mqttshakerweather.com
> A shakerweather.com static wan IP
>
> *weewx.conf*
> [[MQTT]]
> server_url = mqtt://user:p...@10.0.0.30:1883/ 
> 
>
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
>
> *skin.conf*
>  # MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "mqtt.shakerweather.com"
> mqtt_websockets_port = 443
>
> mqtt_websockets_ssl = 1
> mqtt_websockets_topic = "weather/loop"
> disconnect_live_website_visitor = 180
>
> *mosquitto.conf*
> persistence false
>
> # mqtt
> listener 1883
> protocol mqtt
>
> # websockets
> listener 9001
> protocol websockets
>
>
> allow_anonymous true
> password_file /etc/mosquitto/passwd
>
> acl_file /etc/mosquitto/acl
> On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 do...@dougjenkins.com 
> wrote:
>
>> I had a number of struggles with this when I setup the Belchertown skin 
>> on my Raspberry Pi hosting my weather site, www.largoweather.com. I 
>> think the issue is that you need your mqtt_websockets_port to be set to 443 
>> as the websocket traffic is getting filtered out by your firewall 
>> (9001->8083)
>>
>> Here is what I did to get this to work on largoweather.com:
>>
>> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
>>- largoweather.com (A Record)
>>- wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record 
>> pointing the content to largoweather.com
>>- Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
>> processing so essentially all traffic is coming in through 443.
>>
>>These are pointed to my public IP address which is dynamic. I use a 
>> shell script to update cloudflare' content to keep my public IP current for 
>> their system
>>
>> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
>> that acts as a reverse proxy manager for my domains:
>>  2.1 : Setup:
>> - largoweather.com : Setup to point to my local server ip 
>> address. The scheme is http. I use the program's lets encrypt function to 
>> get a SSL certificate and force SSL traffic to my final weewx website.
>>  - wx.largoweather.com : This handles my sockets setup. I 
>> forward all traffic to port 9001 and use the same ssl certificate issued 
>> for largoweather. I force SSL on this setup as well.
>>
>> 3. mosquitto configuration : I setup mosquitto on the same server as my 
>> weewx install since everything is running on the pi. here is my 
>> mosquitto.conf:
>>pid_file /var/run/mosquitto.pid
>>persistence true
>>persistence_location /var/lib/mosquitto/
>>log_type error
>>websockets_log_level 1023
>>connection_messages true
>>log_dest file /mnt/*/weewx/logs/mosquitto.log
>>
>>allow_anonymous true
>>password_file /etc/mosquitto/passwd
>>acl_file /etc/mosquitto/acl
>>
>>listener 9001
>>protocol websockets
>>
>>listener 1883
>>protocol mqtt
>>log_type error
>>
>>
>> 4. weewx/Belchertow

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Your setup seems similar to mine so I went for it. Still not working. 

I went ahead and added a mqtt CNAME record to the shakerweather.com 
cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy 
Server to forward to port 9001 using the same SSL certificate I have for 
shakerweather.com that is forwarding to port 80 to serve the page.

Cloudflare:
TYPE  NAME CONTENT
CNAME  mqttshakerweather.com
A shakerweather.com static wan IP

*weewx.conf*
[[MQTT]]
server_url = mqtt://user:pw@10.0.0.30:1883/
topic = weather
unit_system = US
binding = archive, loop 
aggregation = aggregate

*skin.conf*
 # MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.shakerweather.com"
mqtt_websockets_port = 443
mqtt_websockets_ssl = 1
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 180

*mosquitto.conf*
persistence false

# mqtt
listener 1883
protocol mqtt

# websockets
listener 9001
protocol websockets

allow_anonymous true
password_file /etc/mosquitto/passwd

acl_file /etc/mosquitto/acl
On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 do...@dougjenkins.com wrote:

> I had a number of struggles with this when I setup the Belchertown skin on 
> my Raspberry Pi hosting my weather site, www.largoweather.com. I think 
> the issue is that you need your mqtt_websockets_port to be set to 443 as 
> the websocket traffic is getting filtered out by your firewall (9001->8083)
>
> Here is what I did to get this to work on largoweather.com:
>
> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
>- largoweather.com (A Record)
>- wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record 
> pointing the content to largoweather.com
>- Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
> processing so essentially all traffic is coming in through 443.
>
>These are pointed to my public IP address which is dynamic. I use a 
> shell script to update cloudflare' content to keep my public IP current for 
> their system
>
> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
> that acts as a reverse proxy manager for my domains:
>  2.1 : Setup:
> - largoweather.com : Setup to point to my local server ip 
> address. The scheme is http. I use the program's lets encrypt function to 
> get a SSL certificate and force SSL traffic to my final weewx website.
>  - wx.largoweather.com : This handles my sockets setup. I 
> forward all traffic to port 9001 and use the same ssl certificate issued 
> for largoweather. I force SSL on this setup as well.
>
> 3. mosquitto configuration : I setup mosquitto on the same server as my 
> weewx install since everything is running on the pi. here is my 
> mosquitto.conf:
>pid_file /var/run/mosquitto.pid
>persistence true
>persistence_location /var/lib/mosquitto/
>log_type error
>websockets_log_level 1023
>connection_messages true
>log_dest file /mnt/*/weewx/logs/mosquitto.log
>
>allow_anonymous true
>password_file /etc/mosquitto/passwd
>acl_file /etc/mosquitto/acl
>
>listener 9001
>protocol websockets
>
>listener 1883
>protocol mqtt
>log_type error
>
>
> 4. weewx/Belchertown configuration: Here I setup my MQTT to talk to my 
> local IP address on port 1883. Remember the traffic is all coming in on 
> port 443, so that is the port I need Belchertown skin to essentially 
> connect to resolve the web sockets requests.
>
> [[MQTT]]
> server_url = mqtt://joeuser:xxx@:1883/
> topic = weather
> unit_system = US
> binding = archive, loop
> aggregation = aggregate
> [[Belchertown]]
>  skin = Belchertown
>  enable = True
>
>  [[[Extras]]]
>  site_title = Largo Weather 
>  mqtt_websockets_enabled = 1
>  mqtt_websockets_host = wxsocket.largoweather.com
>  mqtt_websockets_port = 443
>  mqtt_websockets_topic = weather/loop
>  mqtt_websockets_ssl = 1
>  disconnect_live_website_visitor = 180
>
> I hope this helps!
>
> Doug Jenkins
>
>
> On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 mattjo...@gmail.com wrote:
>
>> Thanks Les. I think you have helped clear some things up. For the sake of 
>> clarity and getting to the end state I desire and your suggestion let's 
>> focus on locally hosting (my first setup) over the cloud broker. I have a 
>> robust homelab and really like to keep things in house.
>>
>> *think the first configuration, with the local mqtt broker  isn’t going 
>> to work because mqtt_websockets_host is set to localhost, which will only

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Thank you Doug. I just saw this after I was posted my last message, I'll 
take a look.

On Saturday, July 24, 2021 at 8:33:43 AM UTC-4  wrote:

> I had a number of struggles with this when I setup the Belchertown skin on 
> my Raspberry Pi hosting my weather site, www.largoweather.com. I think 
> the issue is that you need your mqtt_websockets_port to be set to 443 as 
> the websocket traffic is getting filtered out by your firewall (9001->8083)
>
> Here is what I did to get this to work on largoweather.com:
>
> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
>- largoweather.com (A Record)
>- wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record 
> pointing the content to largoweather.com
>- Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
> processing so essentially all traffic is coming in through 443.
>
>These are pointed to my public IP address which is dynamic. I use a 
> shell script to update cloudflare' content to keep my public IP current for 
> their system
>
> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
> that acts as a reverse proxy manager for my domains:
>  2.1 : Setup:
> - largoweather.com : Setup to point to my local server ip 
> address. The scheme is http. I use the program's lets encrypt function to 
> get a SSL certificate and force SSL traffic to my final weewx website.
>  - wx.largoweather.com : This handles my sockets setup. I 
> forward all traffic to port 9001 and use the same ssl certificate issued 
> for largoweather. I force SSL on this setup as well.
>
> 3. mosquitto configuration : I setup mosquitto on the same server as my 
> weewx install since everything is running on the pi. here is my 
> mosquitto.conf:
>pid_file /var/run/mosquitto.pid
>persistence true
>persistence_location /var/lib/mosquitto/
>log_type error
>websockets_log_level 1023
>connection_messages true
>log_dest file /mnt/*/weewx/logs/mosquitto.log
>
>allow_anonymous true
>password_file /etc/mosquitto/passwd
>acl_file /etc/mosquitto/acl
>
>listener 9001
>protocol websockets
>
>listener 1883
>protocol mqtt
>log_type error
>
>
> 4. weewx/Belchertown configuration: Here I setup my MQTT to talk to my 
> local IP address on port 1883. Remember the traffic is all coming in on 
> port 443, so that is the port I need Belchertown skin to essentially 
> connect to resolve the web sockets requests.
>
> [[MQTT]]
> server_url = mqtt://joeuser:xxx@:1883/
> topic = weather
> unit_system = US
> binding = archive, loop
> aggregation = aggregate
> [[Belchertown]]
>  skin = Belchertown
>  enable = True
>
>  [[[Extras]]]
>  site_title = Largo Weather 
>  mqtt_websockets_enabled = 1
>  mqtt_websockets_host = wxsocket.largoweather.com
>  mqtt_websockets_port = 443
>  mqtt_websockets_topic = weather/loop
>  mqtt_websockets_ssl = 1
>  disconnect_live_website_visitor = 180
>
> I hope this helps!
>
> Doug Jenkins
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/c54a5413-88c2-4aea-bf46-b2816ae5d727n%40googlegroups.com.

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Ok, progress with the locally hosted MQTT server.

I changed my DNS of mqtt.beldenserver.com to point to my firewall instead 
of the cloud instance I had setup and added the port forwarding rules to 
OPNSense. I can get the mqtt websocket real time updates to work internally 
on any client by accessing the site by ip address of the weewx/mqtt device 
with the following config:

*weewx.config*
[[MQTT]]
server_url = mqtt://usr:p...@mqtt.beldenserver.com:8883/
topic = weather
unit_system = US
binding = archive, loop 
aggregation = aggregate

*skin.conf*
   # MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.beldenserver.com"
mqtt_websockets_port = 8083
mqtt_websockets_ssl = 0
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 180

This says to me that I successfully have the mqtt.beldenserver.com DNS 
setup and forwarded in my router. I still can't see it externally nor 
locally with the secure website https://shakerweather.com. Perhaps I was 
hoping the NGINX reverse proxy that is handling the SSL cert would just 
return it all over SSL, so I think working on that setup is next.

I initially tried this config but no luck, and it broke everything locally 
accessed by IP as well. 

*weewx.config*
[[MQTT]]
server_url = mqtt://usr:p...@mqtt.beldenserver.com:8883/
topic = weather
unit_system = US
binding = archive, loop 
aggregation = aggregate
[[[tls]]]
tls_version = tlsv1
ca_certs = /etc/ssl/certs/ca-certificates.crt

*skin.conf*
   # MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.beldenserver.com"
mqtt_websockets_port = 8083
mqtt_websockets_ssl = 1
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 180

What next to check?
On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 Matt Johnson wrote:

> Thanks Les. I think you have helped clear some things up. For the sake of 
> clarity and getting to the end state I desire and your suggestion let's 
> focus on locally hosting (my first setup) over the cloud broker. I have a 
> robust homelab and really like to keep things in house.
>
> *think the first configuration, with the local mqtt broker  isn’t going to 
> work because mqtt_websockets_host is set to localhost, which will only 
> resolve to the weewx/mqtt server when the web browser is running on that 
> server.  You need to set something here that will resolve to the weewx/mqtt 
> server from any client that you want to get realtime updates.  *
>
> This makes sense. I initially was on this path and changed the 
> mqtt_websockets host to the ip address of the weewx/mqtt server. This did 
> not work for other clients on the LAN which seemed strange given how I have 
> lots of locally running services on a few different servers in my home lab 
> and I access them all via device IP and port, with a few that are 
> accessible externally via reverse proxy. 
>
>
> *You need a DNS name that will resolve to your firewall and get 
> port-forwarded (for port 8083) to the weewx/mqtt server, 
> for mqtt_websockets_host. That should enable external access.  *
>
> I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare 
> for this. I am running OPNSense for my firewall so I should be able to do 
> anything. Right now I have a firewall rule setup to send port 80/443 
> traffic to my NGINX Reverse Proxy where I have several DNS addresses 
> pointing to different services on different servers. One of these takes 
> shakerweather.com through 80/443 and through the reverse proxy and points 
> to the weewx/mqtt server to return the webpage (which is working fine)
>
> So you are saying I just need to add a rule to forward external port 8083 
> requests to the weewx/mqtt server IP and port?
>
> *And, if your firewall will do hairpinning, it should work internally as 
> well. It may take some magic with forwarding/masquerading rules on the 
> firewall to get hairpinning to work.  (The alternative for internal access 
> is to have an internal DNS server that resolves that hostname directly to 
> the internal IP of the weewx/mqtt server for clients on the internal 
> network.)*
>
> My OPNSense firewall should be able to do hairpinning. I read on that 
> briefly as I have only heard of the term and not too familiar with it. 
>
>
>
>
> On Saturday, July 24, 2021 at 2:24:32 AM UTC-4 ln77 wrote:
>
>> I think the first configuration, with the local mqtt broker  isn’t going 
>> to work because mqtt_websockets_host is set to localhost, which will only 
>> resolve to the weewx/mqtt server when the web browser is running on that 
>> server.  You need to set something here that will resolve to the weewx/mqtt 
>> server from any client that you want to get realtime updates.  
>>
>> Not sure why the second config, with the cloud mqtt broker, isn’t 
>> working. Are you sur

Re: [weewx-user] Belchertown MQTT Issues

[Doug Jenkins]

I had a number of struggles with this when I setup the Belchertown skin on 
my Raspberry Pi hosting my weather site, www.largoweather.com. I think the 
issue is that you need your mqtt_websockets_port to be set to 443 as the 
websocket traffic is getting filtered out by your firewall (9001->8083)

Here is what I did to get this to work on largoweather.com:

1. Setup Cloudflare to manage the DNS proxy on 2 domains: 
   - largoweather.com (A Record)
   - wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record 
pointing the content to largoweather.com
   - Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL 
processing so essentially all traffic is coming in through 443.
   
   These are pointed to my public IP address which is dynamic. I use a 
shell script to update cloudflare' content to keep my public IP current for 
their system

2. NGNIX Proxy Manager : I use this program to manage my NGINX instance 
that acts as a reverse proxy manager for my domains:
 2.1 : Setup:
- largoweather.com : Setup to point to my local server ip 
address. The scheme is http. I use the program's lets encrypt function to 
get a SSL certificate and force SSL traffic to my final weewx website.
 - wx.largoweather.com : This handles my sockets setup. I 
forward all traffic to port 9001 and use the same ssl certificate issued 
for largoweather. I force SSL on this setup as well.

3. mosquitto configuration : I setup mosquitto on the same server as my 
weewx install since everything is running on the pi. here is my 
mosquitto.conf:
   pid_file /var/run/mosquitto.pid
   persistence true
   persistence_location /var/lib/mosquitto/
   log_type error
   websockets_log_level 1023
   connection_messages true
   log_dest file /mnt/*/weewx/logs/mosquitto.log
   
   allow_anonymous true
   password_file /etc/mosquitto/passwd
   acl_file /etc/mosquitto/acl

   listener 9001
   protocol websockets

   listener 1883
   protocol mqtt
   log_type error


4. weewx/Belchertown configuration: Here I setup my MQTT to talk to my 
local IP address on port 1883. Remember the traffic is all coming in on 
port 443, so that is the port I need Belchertown skin to essentially 
connect to resolve the web sockets requests.
   
[[MQTT]]
server_url = mqtt://joeuser:xxx@:1883/
topic = weather
unit_system = US
binding = archive, loop
aggregation = aggregate
[[Belchertown]]
 skin = Belchertown
 enable = True

 [[[Extras]]]
 site_title = Largo Weather 
 mqtt_websockets_enabled = 1
 mqtt_websockets_host = wxsocket.largoweather.com
 mqtt_websockets_port = 443
 mqtt_websockets_topic = weather/loop
 mqtt_websockets_ssl = 1
 disconnect_live_website_visitor = 180

I hope this helps!

Doug Jenkins


On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 mattjo...@gmail.com wrote:

> Thanks Les. I think you have helped clear some things up. For the sake of 
> clarity and getting to the end state I desire and your suggestion let's 
> focus on locally hosting (my first setup) over the cloud broker. I have a 
> robust homelab and really like to keep things in house.
>
> *think the first configuration, with the local mqtt broker  isn’t going to 
> work because mqtt_websockets_host is set to localhost, which will only 
> resolve to the weewx/mqtt server when the web browser is running on that 
> server.  You need to set something here that will resolve to the weewx/mqtt 
> server from any client that you want to get realtime updates.  *
>
> This makes sense. I initially was on this path and changed the 
> mqtt_websockets host to the ip address of the weewx/mqtt server. This did 
> not work for other clients on the LAN which seemed strange given how I have 
> lots of locally running services on a few different servers in my home lab 
> and I access them all via device IP and port, with a few that are 
> accessible externally via reverse proxy. 
>
>
> *You need a DNS name that will resolve to your firewall and get 
> port-forwarded (for port 8083) to the weewx/mqtt server, 
> for mqtt_websockets_host. That should enable external access.  *
>
> I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare 
> for this. I am running OPNSense for my firewall so I should be able to do 
> anything. Right now I have a firewall rule setup to send port 80/443 
> traffic to my NGINX Reverse Proxy where I have several DNS addresses 
> pointing to different services on different servers. One of these takes 
> shakerweather.com through 80/443 and through the reverse proxy and points 
> to the weewx/mqtt server to return the webpage (which is working fine)
>
> So you are saying I just need to add a rule to forward externa

Re: [weewx-user] Belchertown MQTT Issues

[Matt Johnson]

Thanks Les. I think you have helped clear some things up. For the sake of 
clarity and getting to the end state I desire and your suggestion let's 
focus on locally hosting (my first setup) over the cloud broker. I have a 
robust homelab and really like to keep things in house.

*think the first configuration, with the local mqtt broker  isn’t going to 
work because mqtt_websockets_host is set to localhost, which will only 
resolve to the weewx/mqtt server when the web browser is running on that 
server.  You need to set something here that will resolve to the weewx/mqtt 
server from any client that you want to get realtime updates.  *

This makes sense. I initially was on this path and changed the 
mqtt_websockets host to the ip address of the weewx/mqtt server. This did 
not work for other clients on the LAN which seemed strange given how I have 
lots of locally running services on a few different servers in my home lab 
and I access them all via device IP and port, with a few that are 
accessible externally via reverse proxy. 


*You need a DNS name that will resolve to your firewall and get 
port-forwarded (for port 8083) to the weewx/mqtt server, 
for mqtt_websockets_host. That should enable external access.  *

I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare for 
this. I am running OPNSense for my firewall so I should be able to do 
anything. Right now I have a firewall rule setup to send port 80/443 
traffic to my NGINX Reverse Proxy where I have several DNS addresses 
pointing to different services on different servers. One of these takes 
shakerweather.com through 80/443 and through the reverse proxy and points 
to the weewx/mqtt server to return the webpage (which is working fine)

So you are saying I just need to add a rule to forward external port 8083 
requests to the weewx/mqtt server IP and port?

*And, if your firewall will do hairpinning, it should work internally as 
well. It may take some magic with forwarding/masquerading rules on the 
firewall to get hairpinning to work.  (The alternative for internal access 
is to have an internal DNS server that resolves that hostname directly to 
the internal IP of the weewx/mqtt server for clients on the internal 
network.)*

My OPNSense firewall should be able to do hairpinning. I read on that 
briefly as I have only heard of the term and not too familiar with it. 




On Saturday, July 24, 2021 at 2:24:32 AM UTC-4 ln77 wrote:

> I think the first configuration, with the local mqtt broker  isn’t going 
> to work because mqtt_websockets_host is set to localhost, which will only 
> resolve to the weewx/mqtt server when the web browser is running on that 
> server.  You need to set something here that will resolve to the weewx/mqtt 
> server from any client that you want to get realtime updates.  
>
> Not sure why the second config, with the cloud mqtt broker, isn’t working. 
> Are you sure the mqtt broker is configured for SSL on port 8883?  You might 
> put “log_success = true” in the weewx [[MQTT]] config and see if the log 
> messages tell you anything useful.  
>
> Or forget about the cloud server and go back to getting the first config 
> working.  You need a DNS name that will resolve to your firewall and get 
> port-forwarded (for port 8083) to the weewx/mqtt server, 
> for mqtt_websockets_host. That should enable external access.  And, if your 
> firewall will do hairpinning, it should work internally as well. It may 
> take some magic with forwarding/masquerading rules on the firewall to get 
> hairpinning to work.  (The alternative for internal access is to have an 
> internal DNS server that resolves that hostname directly to the internal IP 
> of the weewx/mqtt server for clients on the internal network.)
>
>   -Les
>
>
>
> On 23 Jul 2021, at 21:05, Matt Johnson  wrote:
>
> I've been trouble shooting getting the Belchertown skin MQTT Websocket 
> real time updates to work on my site shakerweather.com for a lot of this 
> week. 
>
> I have WeeWx installed on a dedicated thin client on Ubuntu 20.04 LTS and 
> have Mosquitto and NGINX installed on the same machine. Running bare metal, 
> no docker or VMs here.
>
> External access to WeeWx website is handled via NGINX reverse proxy 
> manager with SSL certs on a different server via docker. Requests to 
> shakerweather.com are sent to the proxy server and then to the WeeWx 
> machine.
>
> With this setup, the site is served up fine internally and externally with 
> the updates at archive intervals every 5 minutes. 
>
> I know that the weewx-mqtt extension is installed correctly as I have been 
> able to test it locally and get the websocket updates to work perfectly 
> with the following configs:
>
> *weewx.conf*
> [[MQTT]]
> server_url = mqtt://user:pw@localhost:8883/
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
>
> *skin.conf*
>  # MQTT Websockets defaults
> mqtt_websockets_enabl

Re: [weewx-user] Belchertown MQTT Issues

[Les Niles]

I think the first configuration, with the local mqtt broker  isn’t going to 
work because mqtt_websockets_host is set to localhost, which will only resolve 
to the weewx/mqtt server when the web browser is running on that server.  You 
need to set something here that will resolve to the weewx/mqtt server from any 
client that you want to get realtime updates.  

Not sure why the second config, with the cloud mqtt broker, isn’t working. Are 
you sure the mqtt broker is configured for SSL on port 8883?  You might put 
“log_success = true” in the weewx [[MQTT]] config and see if the log messages 
tell you anything useful.  

Or forget about the cloud server and go back to getting the first config 
working.  You need a DNS name that will resolve to your firewall and get 
port-forwarded (for port 8083) to the weewx/mqtt server, for 
mqtt_websockets_host. That should enable external access.  And, if your 
firewall will do hairpinning, it should work internally as well. It may take 
some magic with forwarding/masquerading rules on the firewall to get 
hairpinning to work.  (The alternative for internal access is to have an 
internal DNS server that resolves that hostname directly to the internal IP of 
the weewx/mqtt server for clients on the internal network.)

  -Les



> On 23 Jul 2021, at 21:05, Matt Johnson  wrote:
> 
> I've been trouble shooting getting the Belchertown skin MQTT Websocket real 
> time updates to work on my site shakerweather.com for a lot of this week. 
> 
> I have WeeWx installed on a dedicated thin client on Ubuntu 20.04 LTS and 
> have Mosquitto and NGINX installed on the same machine. Running bare metal, 
> no docker or VMs here.
> 
> External access to WeeWx website is handled via NGINX reverse proxy manager 
> with SSL certs on a different server via docker. Requests to 
> shakerweather.com are sent to the proxy server and then to the WeeWx machine.
> 
> With this setup, the site is served up fine internally and externally with 
> the updates at archive intervals every 5 minutes. 
> 
> I know that the weewx-mqtt extension is installed correctly as I have been 
> able to test it locally and get the websocket updates to work perfectly with 
> the following configs:
> 
> weewx.conf
> [[MQTT]]
> server_url = mqtt://user:pw@localhost:8883/
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
> 
> skin.conf
>  # MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "localhost"
> mqtt_websockets_port = 8083
> mqtt_websockets_ssl = 0
> mqtt_websockets_topic = "weather/loop"
> disconnect_live_website_visitor = 180
> 
> I am only able to see the the real time updates on the local machine only 
> with WeeWx and Mosquitto. If I try to access it by IP address elsewhere on my 
> LAN on other clients it does not connect and eventually fails. No luck 
> externally either - despite my NGINX Reverse Proxy Manger handling serving 
> the page and SSL certs the websocket real time updates don't pass through. 
> That was my original thought of how it would work.
> 
> After much trial and error, and reading every thread imaginable on this along 
> with many messages and some correspondence with Pat O'Brien I decided to go 
> ahead and setup a Digital Ocean Ubuntu VM and install Mosquito there to serve 
> as a cloud broker. I followed Pat's instructions exactly as he outlines in 
> setting up the cloud broker: 
> https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/ 
> 
> 
> I have the cloud MQTT broker installed correctly at Digital Ocean with Let's 
> Encrypt, and ran tests on it. Messages can be sent when authenticated, ports 
> are open, etc. However, I can get no further with the websockets real time 
> updates than "Connected. Waiting for data". If I reboot the cloud MQTT broker 
> I immediately get a disconnected message on the website so it does appear to 
> be connecting and waiting for data. Somehow the data is simply not 
> transferring from my WeeWx client to the cloud MQTT broker at Digital Ocean. 
> The other weird thing is if I try to access shakerweather.com or the website 
> by local IP address on the machine that hosts WeeWx I always get a failed 
> message, won't even connect to the server. However, any other client on my 
> LAN and external on WAN does not have this issue.
> 
> Here are my current configs:
> 
> weewx.conf
> [[MQTT]]
> server_url = mqtt://user:p...@mqtt.beldenserver.com:8883/
> topic = weather
> unit_system = US
> binding = archive, loop 
> aggregation = aggregate
>  [[[tls]]]
> tls_version = tlsv1
> ca_certs = /etc/ssl/certs/ca-certificates.crt
> 
> skin.conf
> # MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "mqtt.beldenserver.com"
> mqtt_websockets_port = 8083
>