Re: [weewx-user] Re: Rsync Help or User Help :o)

[cric...@pobox.com]

Important detail:

What is important with the slashes and the path are related to the SOURCE 
path.  It doesn't matter if
the destination does or doesn't have a trailing slash.  It does matter if 
the destination specifies a file rather than
a directory, though.

On Tuesday, November 15, 2022 at 9:51:30 AM UTC-7 FIA Ranch wrote:

> Okay.. Thanks all for the thoughts and comments.
>
> For the sake of anyone fighting this same issue somewhere in the future, 
> let me get this refocused, clarified, up to date.
>
> Baseline Info
> --
> 1) Host Machine is a hosting company
> 2) The host machine is a shared Linux based machine
> 3) I own the domain and rent the hosting
> 4) In the absolute path... home3/fiaranch/ - everything under and 
> past 'fiaranch' I have full control of
>
> 5) Client Machine is mine and is a private machine to be used and abused 
> as I see fit. :o)
>
> Users Question - Closed
> With regards to this issue with WeeWX, all activity will be with my root 
> hat on.
>
> SSH Keys Issue - Closed
> Thanks to Vince I learned that user and root keys are independant.  Now 
> both root and user have their own keys and they have been tested.
>
> SSH Port Question - Closed
> The hosting company states two ports as follows:
> SSH - 22
> SSH Shared/Reseller Servers - 
> NOTE - Command line Rsync actually works with both
>
>
> Question of "/" placement - Closed
>
> Thank you cric...@pobox.com for the definition of differentiation.
>
> File placement on the host is absolute path 
> /home3/fiaranch/public_html/weather/*.*
>
> Using command line, Rsync will accept both, which makes sense based on 
> what cric...@pobox.com stated.
>
> Command (WITH trailing '/') - root@weather:~# rsync -avhze ssh --progress 
> --update --stats  /var/www/html/ --delete 
> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
>
> Result (without all the superfulous data)
> sent 595.67K bytes  received 17.85K bytes  31.46K bytes/sec
> total size is 1.79M  speedup is 2.91
>
> Command (WITHOUT trailing '/') - root@weather:~# rsync -avhze ssh 
> --progress --update --stats  /var/www/html/ --delete 
> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather
>
> Result (without all the superfulous data)
> sent 309.53K bytes  received 12.20K bytes  71.49K bytes/sec
> total size is 1.86M  speedup is 5.77
>
> Note - The result difference is mainly a time difference.  However, based 
> on differentiation cric...@pobox.com stated, I am going to go forward 
> without the trailing '/'.
>
> So that brings me to WeeWX config discussion.
>
> The WeeWX Config is now as follows:
> skin = Rsync
> enable = true
> server = fiaranch.com
> user = fiaranch
> path = /home3/fiaranch/public_html/weather
> port = 
> log_success = true
> log_failure = true
> HTML_ROOT = /var/www/html
> delete = 1
>
> Notes:
> 1) The two log entries I added from what Vince posted.  Just to see if 
> that provides any additional information.
> 2) Removed the trailing '/' from the path.
>
> Results
> Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Running 
> report 'RSYNC'
> Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Found 
> configuration file /etc/weewx/skins/Rsync/skin.conf for report 'RSYNC'
> Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Cannot read 
> localization file /etc/weewx/skins/Rsync/lang/en.conf for report 'RSYNC': 
> Config file not found: "/etc/weewx/skins/Rsync/lang/en.conf".
> Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine:  Using 
> defaults instead.
> Nov 15 09:34:23 weather weewx[5274] DEBUG weeutil.rsyncupload: 
> rsyncupload: cmd: [['rsync', '--archive', '--stats', '--delete', '-e', 'ssh 
> -p ', '/var/www/html/', 
> 'fiar...@fiaranch.com:/home3/fiaranch/public_html/weather']]
> Nov 15 09:34:26 weather weewx[5274] INFO weeutil.rsyncupload: rsync'd 56 
> files (1,093,360 bytes) in 2.42 seconds
>
> WHAT !
>
> All said and done the failures can be attributed to two things:
> 1) SSH Keys must be paired under root, not user - Thank You Vince
> 2) No trailing '/' in the path.  Even though it will work in a command 
> line Rsync, it will not work in WeeWX.
>
> Discussion Closed
>
> Thanks to all for the comments, ideas, assistance, and sharing your 
> experience!
>
> Now going to play with Rsync options to see if I can break it again!! 
>  LMFAO
>
>
>
>
>
>
>
> On Tuesday, November 15, 2022 at 9:17:55 AM UTC-7 jo...@johnkline.com 
> wrote:
>
>> A trailing slash means copy the contents of the directory.
>>
>> Note that WeeWX code will add the slash if it is a directory.  See code 
>> snippet below.
>>
>> [image: weewx.png]
>>
>> weewx/rsyncupload.py at master · weewx/weewx 
>> 
>> github.com 
>> 
>> 
>>
>> # If the source path ends wi

Re: [weewx-user] Re: Rsync Help or User Help :o)

[FIA Ranch]

Okay.. Thanks all for the thoughts and comments.

For the sake of anyone fighting this same issue somewhere in the future, 
let me get this refocused, clarified, up to date.

Baseline Info
--
1) Host Machine is a hosting company
2) The host machine is a shared Linux based machine
3) I own the domain and rent the hosting
4) In the absolute path... home3/fiaranch/ - everything under and 
past 'fiaranch' I have full control of

5) Client Machine is mine and is a private machine to be used and abused as 
I see fit. :o)

Users Question - Closed
With regards to this issue with WeeWX, all activity will be with my root 
hat on.

SSH Keys Issue - Closed
Thanks to Vince I learned that user and root keys are independant.  Now 
both root and user have their own keys and they have been tested.

SSH Port Question - Closed
The hosting company states two ports as follows:
SSH - 22
SSH Shared/Reseller Servers - 
NOTE - Command line Rsync actually works with both


Question of "/" placement - Closed

Thank you cric...@pobox.com for the definition of differentiation.

File placement on the host is absolute path 
/home3/fiaranch/public_html/weather/*.*

Using command line, Rsync will accept both, which makes sense based on what 
cric...@pobox.com stated.

Command (WITH trailing '/') - root@weather:~# rsync -avhze ssh --progress 
--update --stats  /var/www/html/ --delete 
fiara...@fiaranch.com:/home3/fiaranch/public_html/weather/

Result (without all the superfulous data)
sent 595.67K bytes  received 17.85K bytes  31.46K bytes/sec
total size is 1.79M  speedup is 2.91

Command (WITHOUT trailing '/') - root@weather:~# rsync -avhze ssh 
--progress --update --stats  /var/www/html/ --delete 
fiara...@fiaranch.com:/home3/fiaranch/public_html/weather

Result (without all the superfulous data)
sent 309.53K bytes  received 12.20K bytes  71.49K bytes/sec
total size is 1.86M  speedup is 5.77

Note - The result difference is mainly a time difference.  However, based 
on differentiation cric...@pobox.com stated, I am going to go forward 
without the trailing '/'.

So that brings me to WeeWX config discussion.

The WeeWX Config is now as follows:
skin = Rsync
enable = true
server = fiaranch.com
user = fiaranch
path = /home3/fiaranch/public_html/weather
port = 
log_success = true
log_failure = true
HTML_ROOT = /var/www/html
delete = 1

Notes:
1) The two log entries I added from what Vince posted.  Just to see if that 
provides any additional information.
2) Removed the trailing '/' from the path.

Results
Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Running 
report 'RSYNC'
Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Found 
configuration file /etc/weewx/skins/Rsync/skin.conf for report 'RSYNC'
Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine: Cannot read 
localization file /etc/weewx/skins/Rsync/lang/en.conf for report 'RSYNC': 
Config file not found: "/etc/weewx/skins/Rsync/lang/en.conf".
Nov 15 09:34:23 weather weewx[5274] DEBUG weewx.reportengine:  Using 
defaults instead.
Nov 15 09:34:23 weather weewx[5274] DEBUG weeutil.rsyncupload: rsyncupload: 
cmd: [['rsync', '--archive', '--stats', '--delete', '-e', 'ssh -p ', 
'/var/www/html/', 
'fiara...@fiaranch.com:/home3/fiaranch/public_html/weather']]
Nov 15 09:34:26 weather weewx[5274] INFO weeutil.rsyncupload: rsync'd 56 
files (1,093,360 bytes) in 2.42 seconds

WHAT !

All said and done the failures can be attributed to two things:
1) SSH Keys must be paired under root, not user - Thank You Vince
2) No trailing '/' in the path.  Even though it will work in a command line 
Rsync, it will not work in WeeWX.

Discussion Closed

Thanks to all for the comments, ideas, assistance, and sharing your 
experience!

Now going to play with Rsync options to see if I can break it again!!  LMFAO







On Tuesday, November 15, 2022 at 9:17:55 AM UTC-7 jo...@johnkline.com wrote:

> A trailing slash means copy the contents of the directory.
>
> Note that WeeWX code will add the slash if it is a directory.  See code 
> snippet below.
>
> [image: weewx.png]
>
> weewx/rsyncupload.py at master · weewx/weewx 
> 
> github.com 
> 
> 
>
> # If the source path ends with a slash, rsync interprets
> # that as a request to copy all the directory's *contents*,
> # whereas if it doesn't, it copies the entire directory.
> # We want the former, so make it end with a slash.
> # Note: Don't add the slash if local_root isn't a directory
> if self.local_root.endswith(os.sep) or not 
> os.path.isdir(self.local_root):
> rsynclocalspec = self.local_root
> else:
> rsynclocalspec = self.local_root + os.sep
>
>
> On Nov 15, 2022, at 7:33 AM, cric...@pobox.c

Re: [weewx-user] Re: Rsync Help or User Help :o)

['John Kline' via weewx-user]

A trailing slash means copy the contents of the directory.Note that WeeWX code will add the slash if it is a directory.  See code snippet below.weewx/rsyncupload.py at master · weewx/weewxgithub.com        # If the source path ends with a slash, rsync interprets        # that as a request to copy all the directory's *contents*,        # whereas if it doesn't, it copies the entire directory.        # We want the former, so make it end with a slash.        # Note: Don't add the slash if local_root isn't a directory        if self.local_root.endswith(os.sep) or not os.path.isdir(self.local_root):            rsynclocalspec = self.local_root        else:            rsynclocalspec = self.local_root + os.sepOn Nov 15, 2022, at 7:33 AM, cric...@pobox.com  wrote:A couple of points about the trailing slashes.  A trailing slash means copy the contents of the directory.  The --archivemeans copy everything below the specified directory.  Not including a trailing slash means copy this directory and with--archive, everything below it.  It's not a matter of rsync not working because of the slashes, it just does two similar, butdifferent things.  The man pages for rsync are pretty good.As for the ssh keys, I'm pretty sure you can just copy the ~fiaranch/.ssh/id_rsa to ~root/.ssh.  No need to create root-specifickey pair and copy the public server's .ssh directory.  Oh, one other thing that's really important:The .ssh directory permissions are critical.  The directory itself must be chmod 0700.  The files inside it should be chmod 0600.This is true for all the .ssh directories on both ends of the connection.My setup is a little different; I rsync to another local machine that runs a web server.  Then I've got a separate script forpublishing the weewx (and a few other sites) to a public host, since the various href's need to be updated.On Tuesday, November 15, 2022 at 5:25:11 AM UTC-7 Greg Troxel wrote:
"gszla...@gmail.com"  writes:

> 1. Vince is correct. WeeWX runs as root and RSA generation must be done as 
> root.

I was sure Vince was correct to start with :-) but I think running weewx
as root is the wrong thing to do.   I run it as a non-root user, and
really there should be a dedicated weewx user that has permissions to do
what it needs (read the serial port, write the db, access credentials
for upload) and no more.

But this is really "how best to configure weewx on operating system X",
which is orthogonal to "how should I made weewx on system X already
configured as Y work".




-- 
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a69561f5-048a-4e2f-bc8f-e009a9cc2911n%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/E14937D0-EDC4-4C7B-9500-D716F41C09B4%40johnkline.com.

Re: [weewx-user] Re: Rsync Help or User Help :o)

[cric...@pobox.com]

A couple of points about the trailing slashes.  A trailing slash means copy 
the contents of the directory.  The --archive
means copy everything below the specified directory.  Not including a 
trailing slash means copy this directory and with
--archive, everything below it.  It's not a matter of rsync not working 
because of the slashes, it just does two similar, but
different things.  The man pages for rsync are pretty good.
As for the ssh keys, I'm pretty sure you can just copy the 
~fiaranch/.ssh/id_rsa to ~root/.ssh.  No need to create root-specific
key pair and copy the public server's .ssh directory.  Oh, one other thing 
that's really important:

The .ssh directory permissions are critical.  The directory itself must be 
chmod 0700.  The files inside it should be chmod 0600.
This is true for all the .ssh directories on both ends of the connection.

My setup is a little different; I rsync to another local machine that runs 
a web server.  Then I've got a separate script for
publishing the weewx (and a few other sites) to a public host, since the 
various href's need to be updated.

On Tuesday, November 15, 2022 at 5:25:11 AM UTC-7 Greg Troxel wrote:

>
> "gszla...@gmail.com"  writes:
>
> > 1. Vince is correct. WeeWX runs as root and RSA generation must be done 
> as 
> > root.
>
> I was sure Vince was correct to start with :-) but I think running weewx
> as root is the wrong thing to do. I run it as a non-root user, and
> really there should be a dedicated weewx user that has permissions to do
> what it needs (read the serial port, write the db, access credentials
> for upload) and no more.
>
> But this is really "how best to configure weewx on operating system X",
> which is orthogonal to "how should I made weewx on system X already
> configured as Y work".
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/a69561f5-048a-4e2f-bc8f-e009a9cc2911n%40googlegroups.com.

Re: [weewx-user] Re: Rsync Help or User Help :o)

[Greg Troxel]

"gszla...@gmail.com"  writes:

> 1. Vince is correct. WeeWX runs as root and RSA generation must be done as 
> root.

I was sure Vince was correct to start with :-) but I think running weewx
as root is the wrong thing to do.   I run it as a non-root user, and
really there should be a dedicated weewx user that has permissions to do
what it needs (read the serial port, write the db, access credentials
for upload) and no more.

But this is really "how best to configure weewx on operating system X",
which is orthogonal to "how should I made weewx on system X already
configured as Y work".

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/rmisfikflr5.fsf%40s1.lexort.com.


signature.asc
Description: PGP signature

[weewx-user] Re: Rsync Help or User Help :o)

[gszla...@gmail.com]

Technical support at the web host was required to set things up.

1. Vince is correct. WeeWX runs as root and RSA generation must be done as 
root.
2. slahes matter. If you are missing a slash, rsync will not work. Note the 
preceding and trailing slash in HTML_ROOT
3 Port numbers are important. Some web hosts change the default port 22 to 
something else. See example below. Call them to verify.
4. no .ssh config was needed in my case.

After many days of hair pulling, this worked for me. Web hosts can be 
different. If it still doesn't work it is worth calling them..
In my case I had tto ry a number of times until I found a "senior" tech who 
knew his stuff.

*enable = true*
   * server = my_domain_name.com or or the IP address of the web host*
*port = 2002*
*user = remote_user_name*   # user name at web host
*path = public_html/belchertown* # example target path on web host
   *HTML_ROOT = /var/www/html/weewx/belchertown/* # source path on your 
home computer 

On Monday, November 14, 2022 at 8:24:05 PM UTC-5 vince wrote:

> path = the absolute path to where you want things to go.  The parent 
> directory must already be there and be writable.  I do not provide a 
> trailing slash nor a HTML_ROOT item.  Here's mine.
>
> [[RSYNC]]
> skin = Rsync
> enable = true
> server = 1.2.3.4
> user = remoteaccounthere
> path = /home/remoteaccounthere/public_html/weewx
> log_success = true 
> log_failure = true
> delete = 0
>
> You're getting permission denied errors so I'm not certain the account 
> you're logging into on the remote side has permission to log in on the 
> remote side.   The port  looks odd to me.  It should be 22.  I have no 
> idea where that  is coming from.  My syslog does not show a -p flag.
>
> The only thing I can think of is 'perhaps' you are specifying a port 
> number in your /root/.ssh/config for your remote host, but that's just a 
> guess.
>
> On Monday, November 14, 2022 at 4:01:40 PM UTC-8 FIA Ranch wrote:
>
>> First thank you very much for you time and expertise... I do 
>> appreciate it.
>>
>> I have got things running from root.
>> Can get in, out, etc with the server.
>> Can do long Rsync command at the top of this novel from root.
>>
>> However.. it did not fix the command content/format error.  
>>
>> My guess is the missing / after the 'weather' directory???  But it did 
>> get in, and no key issues reported.  Here is the tail.  Same as previous 
>> regarding the command error.
>>
>> Nov 14 16:26:27 weather weewx[80341] DEBUG weewx.reportengine:  Using 
>> defaults instead.
>> Nov 14 16:26:27 weather weewx[80341] DEBUG weeutil.rsyncupload: 
>> rsyncupload: cmd: [['rsync', '--archive', '--stats', '--delete', '-e', 'ssh 
>> -p ', '/var/www/html/', 
>> 'fiar...@fiaranch.com:/home3/fiaranch/public_html/weather']]
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload: rsync 
>> reported errors. Original command: ['rsync', '--archive', '--stats', 
>> '--delete', '-e', 'ssh -p ', '/var/www/html/', 
>> 'fiar...@fiaranch.com:/home3/fiaranch/public_html/weather']
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
>> Permission denied, please try again.
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
>> Received disconnect from 108.167.172.147 port :2: Too many 
>> authentication failures
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
>> Disconnected from 108.167.172.147 port 
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
>> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
>> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
>> rsync error: error in rsync protocol data stream (code 12) at io.c(235) 
>> [sender=3.1.3]
>>
>> I tried one thing went into the config and added a second / after the 
>> directory "weather//" think it might pick up the first one.  No deal
>>
>> Any thoughts / ideas on that one??
>>
>> Again, thanks for you time.
>>
>> Have a good evening/day/morning, or whatever it is in your corner of the 
>> world. Early evening high in the Rockies and I am outa here until tomorrow.
>>
>> David
>>
>> On Monday, November 14, 2022 at 2:26:32 PM UTC-7 vince wrote:
>>
>>> It's doesn't matter what the fiaranch account is.
>>>
>>> Unless you did something special, weewx runs as root so you have to set 
>>> up passwordless ssh in the 'root' account for weewx's rsync uploader to 
>>> work.   All your steps looked correct to me.  You just didn't do it under 
>>> the root account.
>>>
>>> Note - do "sudo bash" to open a bash shell as root when you set things 
>>> up.  You should be able to do "ssh fiaranch.com" and have it log in 
>>> successfully if you have things right.
>>>
>>> FWIW, I have never needed to set delete=1 nor set the ServerWhatever 
>>> items in .ssh/config in many years of runni

[weewx-user] Re: Rsync Help or User Help :o)

[vince]

path = the absolute path to where you want things to go.  The parent 
directory must already be there and be writable.  I do not provide a 
trailing slash nor a HTML_ROOT item.  Here's mine.

[[RSYNC]]
skin = Rsync
enable = true
server = 1.2.3.4
user = remoteaccounthere
path = /home/remoteaccounthere/public_html/weewx
log_success = true 
log_failure = true
delete = 0

You're getting permission denied errors so I'm not certain the account 
you're logging into on the remote side has permission to log in on the 
remote side.   The port  looks odd to me.  It should be 22.  I have no 
idea where that  is coming from.  My syslog does not show a -p flag.

The only thing I can think of is 'perhaps' you are specifying a port number 
in your /root/.ssh/config for your remote host, but that's just a guess.

On Monday, November 14, 2022 at 4:01:40 PM UTC-8 FIA Ranch wrote:

> First thank you very much for you time and expertise... I do 
> appreciate it.
>
> I have got things running from root.
> Can get in, out, etc with the server.
> Can do long Rsync command at the top of this novel from root.
>
> However.. it did not fix the command content/format error.  
>
> My guess is the missing / after the 'weather' directory???  But it did get 
> in, and no key issues reported.  Here is the tail.  Same as previous 
> regarding the command error.
>
> Nov 14 16:26:27 weather weewx[80341] DEBUG weewx.reportengine:  Using 
> defaults instead.
> Nov 14 16:26:27 weather weewx[80341] DEBUG weeutil.rsyncupload: 
> rsyncupload: cmd: [['rsync', '--archive', '--stats', '--delete', '-e', 'ssh 
> -p ', '/var/www/html/', 
> 'fiar...@fiaranch.com:/home3/fiaranch/public_html/weather']]
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload: rsync 
> reported errors. Original command: ['rsync', '--archive', '--stats', 
> '--delete', '-e', 'ssh -p ', '/var/www/html/', 
> 'fiar...@fiaranch.com:/home3/fiaranch/public_html/weather']
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
> Permission denied, please try again.
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
> Received disconnect from 108.167.172.147 port :2: Too many 
> authentication failures
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
> Disconnected from 108.167.172.147 port 
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
> Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  rsync 
> error: error in rsync protocol data stream (code 12) at io.c(235) 
> [sender=3.1.3]
>
> I tried one thing went into the config and added a second / after the 
> directory "weather//" think it might pick up the first one.  No deal
>
> Any thoughts / ideas on that one??
>
> Again, thanks for you time.
>
> Have a good evening/day/morning, or whatever it is in your corner of the 
> world. Early evening high in the Rockies and I am outa here until tomorrow.
>
> David
>
> On Monday, November 14, 2022 at 2:26:32 PM UTC-7 vince wrote:
>
>> It's doesn't matter what the fiaranch account is.
>>
>> Unless you did something special, weewx runs as root so you have to set 
>> up passwordless ssh in the 'root' account for weewx's rsync uploader to 
>> work.   All your steps looked correct to me.  You just didn't do it under 
>> the root account.
>>
>> Note - do "sudo bash" to open a bash shell as root when you set things 
>> up.  You should be able to do "ssh fiaranch.com" and have it log in 
>> successfully if you have things right.
>>
>> FWIW, I have never needed to set delete=1 nor set the ServerWhatever 
>> items in .ssh/config in many years of running the rsync uploader, but that 
>> isn't relevant to getting you set up.  Just passing it along.
>>
>> On Monday, November 14, 2022 at 12:16:44 PM UTC-8 FIA Ranch wrote:
>>
>>> Thanks.  Am not a Linux expert, but
>>> 1) fiaranch is an admin account. I understand that is not root.
>>> 2) I did all the get, install, etc, under the user account. I hear you 
>>> saying it runs under root.
>>> 3) I will create a key pair under root. Upload the public one to the 
>>> server.  See if that impacts Line 7.
>>> 4) Not seeing how that will impact the Line 6 error. But... we will 
>>> see
>>>
>>> Thanks.  
>>>
>>>
>>> On Monday, November 14, 2022 at 12:29:08 PM UTC-7 vince wrote:
>>>
 Two things.  One is you need to do your steps as the account weewx runs 
 as, which is likely root.   While doing your steps from the unprivileged 
 account 'fiaranch' does prove the remote side works and the key works, it 
 has nothing to do with weewx at all.  To get weewx rsync working you need 
 to set up the passwordless ssh for the root account almost certainly.

 Second thing is the ERROR message.  It's likely you either have no 
 .ssh/known_hosts 

[weewx-user] Re: Rsync Help or User Help :o)

[FIA Ranch]

First thank you very much for you time and expertise... I do appreciate 
it.

I have got things running from root.
Can get in, out, etc with the server.
Can do long Rsync command at the top of this novel from root.

However.. it did not fix the command content/format error.  

My guess is the missing / after the 'weather' directory???  But it did get 
in, and no key issues reported.  Here is the tail.  Same as previous 
regarding the command error.

Nov 14 16:26:27 weather weewx[80341] DEBUG weewx.reportengine:  Using 
defaults instead.
Nov 14 16:26:27 weather weewx[80341] DEBUG weeutil.rsyncupload: 
rsyncupload: cmd: [['rsync', '--archive', '--stats', '--delete', '-e', 'ssh 
-p ', '/var/www/html/', 
'fiara...@fiaranch.com:/home3/fiaranch/public_html/weather']]
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload: rsync 
reported errors. Original command: ['rsync', '--archive', '--stats', 
'--delete', '-e', 'ssh -p ', '/var/www/html/', 
'fiara...@fiaranch.com:/home3/fiaranch/public_html/weather']
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
Permission denied, please try again.
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
Received disconnect from 108.167.172.147 port :2: Too many 
authentication failures
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  
Disconnected from 108.167.172.147 port 
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  rsync: 
connection unexpectedly closed (0 bytes received so far) [sender]
Nov 14 16:26:28 weather weewx[80341] ERROR weeutil.rsyncupload:  rsync 
error: error in rsync protocol data stream (code 12) at io.c(235) 
[sender=3.1.3]

I tried one thing went into the config and added a second / after the 
directory "weather//" think it might pick up the first one.  No deal

Any thoughts / ideas on that one??

Again, thanks for you time.

Have a good evening/day/morning, or whatever it is in your corner of the 
world. Early evening high in the Rockies and I am outa here until tomorrow.

David

On Monday, November 14, 2022 at 2:26:32 PM UTC-7 vince wrote:

> It's doesn't matter what the fiaranch account is.
>
> Unless you did something special, weewx runs as root so you have to set up 
> passwordless ssh in the 'root' account for weewx's rsync uploader to work. 
>   All your steps looked correct to me.  You just didn't do it under the 
> root account.
>
> Note - do "sudo bash" to open a bash shell as root when you set things up. 
>  You should be able to do "ssh fiaranch.com" and have it log in 
> successfully if you have things right.
>
> FWIW, I have never needed to set delete=1 nor set the ServerWhatever items 
> in .ssh/config in many years of running the rsync uploader, but that isn't 
> relevant to getting you set up.  Just passing it along.
>
> On Monday, November 14, 2022 at 12:16:44 PM UTC-8 FIA Ranch wrote:
>
>> Thanks.  Am not a Linux expert, but
>> 1) fiaranch is an admin account. I understand that is not root.
>> 2) I did all the get, install, etc, under the user account. I hear you 
>> saying it runs under root.
>> 3) I will create a key pair under root. Upload the public one to the 
>> server.  See if that impacts Line 7.
>> 4) Not seeing how that will impact the Line 6 error. But... we will 
>> see
>>
>> Thanks.  
>>
>>
>> On Monday, November 14, 2022 at 12:29:08 PM UTC-7 vince wrote:
>>
>>> Two things.  One is you need to do your steps as the account weewx runs 
>>> as, which is likely root.   While doing your steps from the unprivileged 
>>> account 'fiaranch' does prove the remote side works and the key works, it 
>>> has nothing to do with weewx at all.  To get weewx rsync working you need 
>>> to set up the passwordless ssh for the root account almost certainly.
>>>
>>> Second thing is the ERROR message.  It's likely you either have no 
>>> .ssh/known_hosts entry for your remote system or an old one that doesn't 
>>> match the remote system's current host key (which is typical if you rebuilt 
>>> the remote system, os's autogenerate new keys).  That's likely why you're 
>>> getting the host key verification message.
>>>
>>> Shorter answer is - repeat your steps as root, assuming weewx also runs 
>>> as root.  Sounds like you just tested with the wrong account on your client 
>>> side.
>>>
>>> On Monday, November 14, 2022 at 11:07:06 AM UTC-8 FIA Ranch wrote:
>>>
 Is WeeWX Rsync buggy or am I!!  Likely me, and I just haven't found the 
 right bug spray to kill it!!  :o)

 About ready to just run a crontab job! Yes I understand WeeWX and 
 crontab being/getting out of sync.

 I have been fighting this for a couple of days.  So now I am looking 
 for hints.  I have run out!

 I apologize for this being a long post!  I hope I have covered all the 
 "have you tried.."

 What DOES work.

 I can sign into the server, move around, add, delete files, etc.

>>>

[weewx-user] Re: Rsync Help or User Help :o)

[vince]

It's doesn't matter what the fiaranch account is.

Unless you did something special, weewx runs as root so you have to set up 
passwordless ssh in the 'root' account for weewx's rsync uploader to work. 
  All your steps looked correct to me.  You just didn't do it under the 
root account.

Note - do "sudo bash" to open a bash shell as root when you set things up. 
 You should be able to do "ssh fiaranch.com" and have it log in 
successfully if you have things right.

FWIW, I have never needed to set delete=1 nor set the ServerWhatever items 
in .ssh/config in many years of running the rsync uploader, but that isn't 
relevant to getting you set up.  Just passing it along.

On Monday, November 14, 2022 at 12:16:44 PM UTC-8 FIA Ranch wrote:

> Thanks.  Am not a Linux expert, but
> 1) fiaranch is an admin account. I understand that is not root.
> 2) I did all the get, install, etc, under the user account. I hear you 
> saying it runs under root.
> 3) I will create a key pair under root. Upload the public one to the 
> server.  See if that impacts Line 7.
> 4) Not seeing how that will impact the Line 6 error. But... we will see
>
> Thanks.  
>
>
> On Monday, November 14, 2022 at 12:29:08 PM UTC-7 vince wrote:
>
>> Two things.  One is you need to do your steps as the account weewx runs 
>> as, which is likely root.   While doing your steps from the unprivileged 
>> account 'fiaranch' does prove the remote side works and the key works, it 
>> has nothing to do with weewx at all.  To get weewx rsync working you need 
>> to set up the passwordless ssh for the root account almost certainly.
>>
>> Second thing is the ERROR message.  It's likely you either have no 
>> .ssh/known_hosts entry for your remote system or an old one that doesn't 
>> match the remote system's current host key (which is typical if you rebuilt 
>> the remote system, os's autogenerate new keys).  That's likely why you're 
>> getting the host key verification message.
>>
>> Shorter answer is - repeat your steps as root, assuming weewx also runs 
>> as root.  Sounds like you just tested with the wrong account on your client 
>> side.
>>
>> On Monday, November 14, 2022 at 11:07:06 AM UTC-8 FIA Ranch wrote:
>>
>>> Is WeeWX Rsync buggy or am I!!  Likely me, and I just haven't found the 
>>> right bug spray to kill it!!  :o)
>>>
>>> About ready to just run a crontab job! Yes I understand WeeWX and 
>>> crontab being/getting out of sync.
>>>
>>> I have been fighting this for a couple of days.  So now I am looking for 
>>> hints.  I have run out!
>>>
>>> I apologize for this being a long post!  I hope I have covered all the 
>>> "have you tried.."
>>>
>>> What DOES work.
>>>
>>> I can sign into the server, move around, add, delete files, etc.
>>>
>>> NOTE:  It is a shared server with a shared IP address.  All of the 
>>> following have been tried both ways.  Using the domain and the shared IP.
>>>
>>> 
>>> fiaranch@weather:~$ ssh fiar...@fiaranch.com (Enter)
>>> Last login: Mon Nov 14 07:52:14 2022 from 166.sub-174-212-228.myvzw.com
>>> fiar...@fiaranch.com [~]# exit (Hit Enter)
>>> logout
>>> Connection to fiaranch.com closed.
>>> fiaranch@weather:~$ 
>>> 
>>>
>>> I can rsync from the command line
>>> -
>>> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>>>  /var/www/html/ --delete 
>>> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
>>> (Hit Enter)
>>> sending incremental file list
>>> ./
>>> #FTP.last
>>>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
>>> belchertown-dark.min.css
>>> |
>>> |
>>> blah-blah-blah
>>> |
>>> Number of files: 202 (reg: 185, dir: 17)
>>> Number of created files: 0
>>> Number of deleted files: 0
>>> Number of regular files transferred: 57
>>> Total file size: 1.91M bytes
>>> Total transferred file size: 1.15M bytes
>>> Literal data: 280.85K bytes
>>> Matched data: 872.30K bytes
>>> File list size: 0
>>> File list generation time: 0.001 seconds
>>> File list transfer time: 0.000 seconds
>>> Total bytes sent: 171.97K
>>> Total bytes received: 11.16K
>>>
>>> sent 171.97K bytes  received 11.16K bytes  73.25K bytes/sec
>>> total size is 1.91M  speedup is 10.44
>>> -
>>>
>>> Immediately run it again..
>>> --
>>> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>>>  /var/www/html/ --delete 
>>> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
>>> (Hit Enter)
>>> sending incremental file list
>>> ./
>>> #FTP.last
>>>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
>>> belchertown-dark.min.css
>>> |
>>> |
>>> blah-blah-blah
>>> |
>>> Number of files: 202 (reg: 185, dir: 17)
>>> Number of created files: 0
>>> Number of deleted files: 0
>>> Number of regular files transferred: 0
>>> Total file size: 1.91M bytes
>>> Total transferred file size: 0 bytes
>>> Literal data: 0 bytes
>>> Matched data: 0 bytes
>>> File list size: 0
>>> File list generation time: 0.001 seconds
>>> File list transfer time: 0.000 second

[weewx-user] Re: Rsync Help or User Help :o)

[FIA Ranch]

Thanks.  Am not a Linux expert, but
1) fiaranch is an admin account. I understand that is not root.
2) I did all the get, install, etc, under the user account. I hear you 
saying it runs under root.
3) I will create a key pair under root. Upload the public one to the 
server.  See if that impacts Line 7.
4) Not seeing how that will impact the Line 6 error. But... we will see

Thanks.  


On Monday, November 14, 2022 at 12:29:08 PM UTC-7 vince wrote:

> Two things.  One is you need to do your steps as the account weewx runs 
> as, which is likely root.   While doing your steps from the unprivileged 
> account 'fiaranch' does prove the remote side works and the key works, it 
> has nothing to do with weewx at all.  To get weewx rsync working you need 
> to set up the passwordless ssh for the root account almost certainly.
>
> Second thing is the ERROR message.  It's likely you either have no 
> .ssh/known_hosts entry for your remote system or an old one that doesn't 
> match the remote system's current host key (which is typical if you rebuilt 
> the remote system, os's autogenerate new keys).  That's likely why you're 
> getting the host key verification message.
>
> Shorter answer is - repeat your steps as root, assuming weewx also runs as 
> root.  Sounds like you just tested with the wrong account on your client 
> side.
>
> On Monday, November 14, 2022 at 11:07:06 AM UTC-8 FIA Ranch wrote:
>
>> Is WeeWX Rsync buggy or am I!!  Likely me, and I just haven't found the 
>> right bug spray to kill it!!  :o)
>>
>> About ready to just run a crontab job! Yes I understand WeeWX and crontab 
>> being/getting out of sync.
>>
>> I have been fighting this for a couple of days.  So now I am looking for 
>> hints.  I have run out!
>>
>> I apologize for this being a long post!  I hope I have covered all the 
>> "have you tried.."
>>
>> What DOES work.
>>
>> I can sign into the server, move around, add, delete files, etc.
>>
>> NOTE:  It is a shared server with a shared IP address.  All of the 
>> following have been tried both ways.  Using the domain and the shared IP.
>>
>> 
>> fiaranch@weather:~$ ssh fiar...@fiaranch.com (Enter)
>> Last login: Mon Nov 14 07:52:14 2022 from 166.sub-174-212-228.myvzw.com
>> fiar...@fiaranch.com [~]# exit (Hit Enter)
>> logout
>> Connection to fiaranch.com closed.
>> fiaranch@weather:~$ 
>> 
>>
>> I can rsync from the command line
>> -
>> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>>  /var/www/html/ --delete 
>> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
>> (Hit Enter)
>> sending incremental file list
>> ./
>> #FTP.last
>>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
>> belchertown-dark.min.css
>> |
>> |
>> blah-blah-blah
>> |
>> Number of files: 202 (reg: 185, dir: 17)
>> Number of created files: 0
>> Number of deleted files: 0
>> Number of regular files transferred: 57
>> Total file size: 1.91M bytes
>> Total transferred file size: 1.15M bytes
>> Literal data: 280.85K bytes
>> Matched data: 872.30K bytes
>> File list size: 0
>> File list generation time: 0.001 seconds
>> File list transfer time: 0.000 seconds
>> Total bytes sent: 171.97K
>> Total bytes received: 11.16K
>>
>> sent 171.97K bytes  received 11.16K bytes  73.25K bytes/sec
>> total size is 1.91M  speedup is 10.44
>> -
>>
>> Immediately run it again..
>> --
>> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>>  /var/www/html/ --delete 
>> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
>> (Hit Enter)
>> sending incremental file list
>> ./
>> #FTP.last
>>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
>> belchertown-dark.min.css
>> |
>> |
>> blah-blah-blah
>> |
>> Number of files: 202 (reg: 185, dir: 17)
>> Number of created files: 0
>> Number of deleted files: 0
>> Number of regular files transferred: 0
>> Total file size: 1.91M bytes
>> Total transferred file size: 0 bytes
>> Literal data: 0 bytes
>> Matched data: 0 bytes
>> File list size: 0
>> File list generation time: 0.001 seconds
>> File list transfer time: 0.000 seconds
>> Total bytes sent: 5.31K
>> Total bytes received: 35
>>
>> sent 5.31K bytes  received 35 bytes  2.14K bytes/sec
>> total size is 1.91M  speedup is 357.60
>> ---
>>
>> All of the above tells me that the ssh keys are set and working.  Also 
>> says that Rsync is working between client and server. 
>>
>> Now we go to WeeWX !!!
>>
>> Following are the file setups as they currently exist.
>>
>> First the external files
>>
>> The .ssh directory
>> fiaranch@weather:~$ dir ~/.ssh (Hit Enter)
>>
>> configid_rsaknown_hosts
>>
>> Three files - config, key file, known hosts
>>
>> Config File Content
>> --
>> HostName fiaranch.com
>> IdentityFile ~/.ssh/id_rsa
>> StrictHostKeyChecking no
>> User fiaranch
>> ServerAliveInterval 30
>> ServerAliveCountMax 6
>> -
>> NOTE: The ServerAlive entries are from researching one of the debug 
>> errors.  Same with t

[weewx-user] Re: Rsync Help or User Help :o)

[vince]

Two things.  One is you need to do your steps as the account weewx runs as, 
which is likely root.   While doing your steps from the unprivileged 
account 'fiaranch' does prove the remote side works and the key works, it 
has nothing to do with weewx at all.  To get weewx rsync working you need 
to set up the passwordless ssh for the root account almost certainly.

Second thing is the ERROR message.  It's likely you either have no 
.ssh/known_hosts entry for your remote system or an old one that doesn't 
match the remote system's current host key (which is typical if you rebuilt 
the remote system, os's autogenerate new keys).  That's likely why you're 
getting the host key verification message.

Shorter answer is - repeat your steps as root, assuming weewx also runs as 
root.  Sounds like you just tested with the wrong account on your client 
side.

On Monday, November 14, 2022 at 11:07:06 AM UTC-8 FIA Ranch wrote:

> Is WeeWX Rsync buggy or am I!!  Likely me, and I just haven't found the 
> right bug spray to kill it!!  :o)
>
> About ready to just run a crontab job! Yes I understand WeeWX and crontab 
> being/getting out of sync.
>
> I have been fighting this for a couple of days.  So now I am looking for 
> hints.  I have run out!
>
> I apologize for this being a long post!  I hope I have covered all the 
> "have you tried.."
>
> What DOES work.
>
> I can sign into the server, move around, add, delete files, etc.
>
> NOTE:  It is a shared server with a shared IP address.  All of the 
> following have been tried both ways.  Using the domain and the shared IP.
>
> 
> fiaranch@weather:~$ ssh fiar...@fiaranch.com (Enter)
> Last login: Mon Nov 14 07:52:14 2022 from 166.sub-174-212-228.myvzw.com
> fiar...@fiaranch.com [~]# exit (Hit Enter)
> logout
> Connection to fiaranch.com closed.
> fiaranch@weather:~$ 
> 
>
> I can rsync from the command line
> -
> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>  /var/www/html/ --delete 
> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
> (Hit Enter)
> sending incremental file list
> ./
> #FTP.last
>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
> belchertown-dark.min.css
> |
> |
> blah-blah-blah
> |
> Number of files: 202 (reg: 185, dir: 17)
> Number of created files: 0
> Number of deleted files: 0
> Number of regular files transferred: 57
> Total file size: 1.91M bytes
> Total transferred file size: 1.15M bytes
> Literal data: 280.85K bytes
> Matched data: 872.30K bytes
> File list size: 0
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 171.97K
> Total bytes received: 11.16K
>
> sent 171.97K bytes  received 11.16K bytes  73.25K bytes/sec
> total size is 1.91M  speedup is 10.44
> -
>
> Immediately run it again..
> --
> fiaranch@weather:~$ rsync -avhze ssh --progress --update --stats 
>  /var/www/html/ --delete 
> fiar...@fiaranch.com:/home3/fiaranch/public_html/weather/
> (Hit Enter)
> sending incremental file list
> ./
> #FTP.last
>  26.05K 100%   23.51MB/s0:00:00 (xfr#1, to-chk=200/202)
> belchertown-dark.min.css
> |
> |
> blah-blah-blah
> |
> Number of files: 202 (reg: 185, dir: 17)
> Number of created files: 0
> Number of deleted files: 0
> Number of regular files transferred: 0
> Total file size: 1.91M bytes
> Total transferred file size: 0 bytes
> Literal data: 0 bytes
> Matched data: 0 bytes
> File list size: 0
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 5.31K
> Total bytes received: 35
>
> sent 5.31K bytes  received 35 bytes  2.14K bytes/sec
> total size is 1.91M  speedup is 357.60
> ---
>
> All of the above tells me that the ssh keys are set and working.  Also 
> says that Rsync is working between client and server. 
>
> Now we go to WeeWX !!!
>
> Following are the file setups as they currently exist.
>
> First the external files
>
> The .ssh directory
> fiaranch@weather:~$ dir ~/.ssh (Hit Enter)
>
> configid_rsaknown_hosts
>
> Three files - config, key file, known hosts
>
> Config File Content
> --
> HostName fiaranch.com
> IdentityFile ~/.ssh/id_rsa
> StrictHostKeyChecking no
> User fiaranch
> ServerAliveInterval 30
> ServerAliveCountMax 6
> -
> NOTE: The ServerAlive entries are from researching one of the debug 
> errors.  Same with the StrictHost That info coming up.
>
> The known_hosts file (without all the key stuff
> 
> |1|lz75l---
> |1|lMA4Y..
> fiaranch.com
> 
> NOTE: Added the domain from another research finding
>
> Now the WeeWX config file - Again omitting everything except relevant parts
>
>
> -
> [[RSYNC]]
> skin = Rsync
> enable = true
> server = fiaranch.com
> user = fiaranch
> path = /home3/fiaranch/public_html/weather/
> HTML_ROOT = /var/www/html
> delete = 1
> ---
> NOTE:  Yes, HTML_ROOT is a different directory