Re: Alternatives to rsync

2016-10-22 Thread Julian H. Stacey 
Hi, Reference:
> From: "reko.turja--- via freebsd-ports" 
> Reply-to: reko.tu...@liukuma.net
> Date: Thu, 13 Oct 2016 07:39:45 +0300

"reko.turja--- via freebsd-ports" wrote:
> One of my users is needing rsync like functionality to transfer changed 
> contents of some directories between couple of machines. As rsync 3 isn't 
> open source, but GPL3 it's out of question in order to keep the system 
> untainted.
> 
> The software should be relatively lightweight - no fullblown 
> mirroring/backup is needed. Also hints how to achieve similar ends using 
> maybe tar/ssh might do.
> 
> -Reko 

I've used use /usr/ports/net/rdist6 for decades, similar to but a big
improvement on rdist, rdist was in BSD src/ back I recall in UCB-4.2.

Samples:
http://www.berklix.com/~jhs/Distfile(also called from ./Makefile)
http://www.berklix.com/~jhs/home/Distfile
http://www.berklix.com/~jhs/scanjet/data/Distfile

Cheers,
Julian
--
Julian Stacey, BSD Linux Unix Sys Eng Consultant Munich
 Reply below, Prefix '> '. Plain text, No .doc, base64, HTML, quoted-printable.
 http://berklix.eu/brexit/#stolen_votes
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-17 Thread Greg 'groggy' Lehey 
On Monday, 17 October 2016 at  6:51:05 -0700, Freddie Cash wrote:
> On Oct 17, 2016 1:21 AM, "Lars Engels"  wrote:
>>
>> rsync 2 has a different algorithm for checking if a file changed. The
>> new one is much faster.
>
> Rsync 2 also spent a long time building a list of changes files first,
> before transferring any data.
>
> Rsync 3 starts building the file list, then starts transferring data while
> it continues to build the file list. This drops the total time spent by a
> large factor. Our backup times dropped by about an hour per server
> switching from 2 to 3.
>
> Something else to consider when looking at resurrecting version 2.

I can see no good reason to use rsync 2 in favour of 3.  But reko is
concerned by the GPL number, not rsync number.

The other thing to consider is whether rsync 2 is faster than the
other alternative software.  I suspect that it is.

Greg
--
Sent from my desktop computer.
Finger g...@freebsd.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA


signature.asc
Description: PGP signature

Re: Alternatives to rsync

2016-10-17 Thread Freddie Cash 
On Oct 17, 2016 1:21 AM, "Lars Engels"  wrote:
>
> On Mon, Oct 17, 2016 at 09:46:31AM +1100, Greg 'groggy' Lehey wrote:
> > On Friday, 14 October 2016 at  8:01:51 +0300, reko.turja--- via
freebsd-ports wrote:
> > >
> > > Greg, I've actually put some thought in making a local port of
> > > rsync2. I've done some research on it and it seems to be fairly
> > > usable and security patched still.
> >
> > Even simpler, use the old version of the current rsync port.  Check
> > out with svn, which (svn log) can also tell you when the last rsync 2
> > version was.
>
> rsync 2 has a different algorithm for checking if a file changed. The
> new one is much faster.

Rsync 2 also spent a long time building a list of changes files first,
before transferring any data.

Rsync 3 starts building the file list, then starts transferring data while
it continues to build the file list. This drops the total time spent by a
large factor. Our backup times dropped by about an hour per server
switching from 2 to 3.

Something else to consider when looking at resurrecting version 2.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-17 Thread Lars Engels 
On Mon, Oct 17, 2016 at 09:46:31AM +1100, Greg 'groggy' Lehey wrote:
> On Friday, 14 October 2016 at  8:01:51 +0300, reko.turja--- via freebsd-ports 
> wrote:
> >
> > Greg, I've actually put some thought in making a local port of
> > rsync2. I've done some research on it and it seems to be fairly
> > usable and security patched still.
> 
> Even simpler, use the old version of the current rsync port.  Check
> out with svn, which (svn log) can also tell you when the last rsync 2
> version was.

rsync 2 has a different algorithm for checking if a file changed. The
new one is much faster.


pgp1PMubTZN7A.pgp
Description: PGP signature

Re: Alternatives to rsync

2016-10-17 Thread Matthias Andree 
Am 17.10.2016 um 00:46 schrieb Greg 'groggy' Lehey:
> On Friday, 14 October 2016 at  8:01:51 +0300, reko.turja--- via freebsd-ports 
> wrote:
>> Greg, I've actually put some thought in making a local port of
>> rsync2. I've done some research on it and it seems to be fairly
>> usable and security patched still.
> Even simpler, use the old version of the current rsync port.  Check
> out with svn, which (svn log) can also tell you when the last rsync 2
> version was.
How many unfixed and critical/security bugs will that bring back?

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-16 Thread Greg 'groggy' Lehey 
On Friday, 14 October 2016 at  8:01:51 +0300, reko.turja--- via freebsd-ports 
wrote:
>
> Greg, I've actually put some thought in making a local port of
> rsync2. I've done some research on it and it seems to be fairly
> usable and security patched still.

Even simpler, use the old version of the current rsync port.  Check
out with svn, which (svn log) can also tell you when the last rsync 2
version was.

Greg
--
Sent from my desktop computer.
Finger g...@freebsd.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA


signature.asc
Description: PGP signature

Re: Alternatives to rsync

2016-10-14 Thread Dewayne Geraghty 
On 14 October 2016 at 17:01, Franco Fichtner  wrote:

>
> > On 14 Oct 2016, at 7:54 AM, Eduardo Morras via freebsd-ports <
> freebsd-ports@freebsd.org> wrote:
> >
> >
> > Sorry for commenting on this reply to Greg to answer Shane Ambler, I
> > joined maillist today.
> >
> > On Fri, 14 Oct 2016 09:26:03 +1100
> > Greg 'groggy' Lehey  wrote:
> >
> >> On Thursday, 13 October 2016 at 18:13:39 +1030, Shane Ambler wrote:
> >>> On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:
>  One of my users is needing rsync like functionality to transfer
>  changed contents of some directories between couple of machines.
>  As rsync 3 isn't open source, but GPL3 it's out of question in
>  order to keep the system untainted.
> 
>  The software should be relatively lightweight - no fullblown
>  mirroring/backup is needed. Also hints how to achieve similar ends
>  using maybe tar/ssh might do.
> >>>
> >>> sysutils/cpdup provides similar functionality to rsync and is bsd
> >>> licensed.
> >
> > cpdup in ports comes from old matt dillon pages and is version1.18.
> > DragonflyBSD has version 1.32 at [1][2] and compiles with low effort on
> > FreeBSD.
>
> If there is interest in updating the port we should do it.  I can talk
> to Matt, see if he is willing to release an updated portable version
> with required build fixes (if any).
>
>
> Cheers,
> Franco
> ___
> freebsd-ports@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
>


Franco, That's probably a good idea as the cpdup homepage has source for
1.09.

Could we be so lucky to include extended attributes and MAC labels in cpdup?
Regards, Dewayne
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-14 Thread Franco Fichtner 

> On 14 Oct 2016, at 7:54 AM, Eduardo Morras via freebsd-ports 
>  wrote:
> 
> 
> Sorry for commenting on this reply to Greg to answer Shane Ambler, I
> joined maillist today.
> 
> On Fri, 14 Oct 2016 09:26:03 +1100
> Greg 'groggy' Lehey  wrote:
> 
>> On Thursday, 13 October 2016 at 18:13:39 +1030, Shane Ambler wrote:
>>> On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:
 One of my users is needing rsync like functionality to transfer
 changed contents of some directories between couple of machines.
 As rsync 3 isn't open source, but GPL3 it's out of question in
 order to keep the system untainted.
 
 The software should be relatively lightweight - no fullblown
 mirroring/backup is needed. Also hints how to achieve similar ends
 using maybe tar/ssh might do.
>>> 
>>> sysutils/cpdup provides similar functionality to rsync and is bsd
>>> licensed.
> 
> cpdup in ports comes from old matt dillon pages and is version1.18.
> DragonflyBSD has version 1.32 at [1][2] and compiles with low effort on
> FreeBSD.

If there is interest in updating the port we should do it.  I can talk
to Matt, see if he is willing to release an updated portable version
with required build fixes (if any).


Cheers,
Franco
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread Eduardo Morras via freebsd-ports 

Sorry for commenting on this reply to Greg to answer Shane Ambler, I
joined maillist today.

On Fri, 14 Oct 2016 09:26:03 +1100
Greg 'groggy' Lehey  wrote:

> On Thursday, 13 October 2016 at 18:13:39 +1030, Shane Ambler wrote:
> > On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:
> >> One of my users is needing rsync like functionality to transfer
> >> changed contents of some directories between couple of machines.
> >> As rsync 3 isn't open source, but GPL3 it's out of question in
> >> order to keep the system untainted.
> >>
> >> The software should be relatively lightweight - no fullblown
> >> mirroring/backup is needed. Also hints how to achieve similar ends
> >> using maybe tar/ssh might do.
> >
> > sysutils/cpdup provides similar functionality to rsync and is bsd
> > licensed.

cpdup in ports comes from old matt dillon pages and is version1.18.
DragonflyBSD has version 1.32 at [1][2] and compiles with low effort on
FreeBSD.

> Does anybody have information on how efficient it is in comparison
> with rsync?  Apart from that, I agree with the other comments.  But if
> Reko wants a non-GPL3 package, for whatever reason, what's wrong with
> an older version of rsync?

For me it has better performance, the BSD licence and promote/use BSD
borned tools, for me it's a plus.

> Greg

[1]
http://gitweb.dragonflybsd.org/dragonfly.git/tree/d72200edc8a9934f16e185f29e31ef5fe654c93a:/bin/cpdup
[2]
http://gitweb.dragonflybsd.org/dragonfly.git/blob/d72200edc8a9934f16e185f29e31ef5fe654c93a:/bin/cpdup/cpdup.c

---   ---
Eduardo Morras 
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread Shane Ambler 

On 14/10/2016 08:56, Greg 'groggy' Lehey wrote:

On Thursday, 13 October 2016 at 18:13:39 +1030, Shane Ambler wrote:

On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:

One of my users is needing rsync like functionality to transfer changed
contents of some directories between couple of machines. As rsync 3
isn't open source, but GPL3 it's out of question in order to keep the
system untainted.

The software should be relatively lightweight - no fullblown
mirroring/backup is needed. Also hints how to achieve similar ends using
maybe tar/ssh might do.


sysutils/cpdup provides similar functionality to rsync and is bsd licensed.


Does anybody have information on how efficient it is in comparison
with rsync?  Apart from that, I agree with the other comments.  But if
Reko wants a non-GPL3 package, for whatever reason, what's wrong with
an older version of rsync?


A new port could be created to build an older version - unless there is
a wider interest in maintaining the older version, the drawback will be
that it will use an old unsupported code base that will need to be
maintained by the one person using it and will likely only get limited
testing within the scope of their use case.

There is a chance that others want to stay away from gplv3 so a fork of
rsync 2.6.9 could be the start of a new rsync alternative project - not
just a port to build an old version.

--
FreeBSD - the place to B...Saving Data

Shane Ambler

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread reko.turja--- via freebsd-ports 

Thanks a lot for your suggestions!

Franco and Shane - I will definitely check cpdup out. Georges suggestion is 
neat,
sadly for this usage pattern zfs isn't ideal - lack of memory, files are 
transferred
between freebsd and linux etc. but it's definitely something I'll have to 
remember

for the future.

Greg, I've actually put some thought in making a local port of rsync2. I've 
done
some research on it and it seems to be fairly usable and security patched 
still.


-Reko

-Original Message- 
One of my users is needing rsync like functionality to transfer changed 
contents of some directories between couple of machines. As rsync 3 isn't 
open source, but GPL3 it's out of question in order to keep the system 
untainted.


The software should be relatively lightweight - no fullblown 
mirroring/backup is needed. Also hints how to achieve similar ends using 
maybe tar/ssh might do.


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" 


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread Greg 'groggy' Lehey 
On Thursday, 13 October 2016 at 18:13:39 +1030, Shane Ambler wrote:
> On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:
>> One of my users is needing rsync like functionality to transfer changed
>> contents of some directories between couple of machines. As rsync 3
>> isn't open source, but GPL3 it's out of question in order to keep the
>> system untainted.
>>
>> The software should be relatively lightweight - no fullblown
>> mirroring/backup is needed. Also hints how to achieve similar ends using
>> maybe tar/ssh might do.
>
> sysutils/cpdup provides similar functionality to rsync and is bsd licensed.

Does anybody have information on how efficient it is in comparison
with rsync?  Apart from that, I agree with the other comments.  But if
Reko wants a non-GPL3 package, for whatever reason, what's wrong with
an older version of rsync?

Greg
--
Sent from my desktop computer.
Finger g...@freebsd.org for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA


signature.asc
Description: PGP signature

Re: Alternatives to rsync

2016-10-13 Thread Matthias Andree 
Am 13.10.2016 um 07:20 schrieb Peter Beckman:
> 2. Why is GPL3 out of the question? Is the user going to resell the
> device
>as a service? If the user is simply "using" the software, no
> disclosure
>of other software is necessary. Only if your user is attempting to
> "make
>money" from the sale of the software or service powered by rsync does
>the source need to be disclosed. 
What does the GPLv3 have to do with reselling software as a service?
The _Affero_ GPL variants are what copyleft software-as-a-service, not
the plain GPLv3 itself.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread Shane Ambler 

On 13/10/2016 15:09, reko.turja--- via freebsd-ports wrote:

One of my users is needing rsync like functionality to transfer changed
contents of some directories between couple of machines. As rsync 3
isn't open source, but GPL3 it's out of question in order to keep the
system untainted.

The software should be relatively lightweight - no fullblown
mirroring/backup is needed. Also hints how to achieve similar ends using
maybe tar/ssh might do.

-Reko


sysutils/cpdup provides similar functionality to rsync and is bsd licensed.

--
FreeBSD - the place to B...Software Developing

Shane Ambler

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread George Michaelson 
If you want block efficient, then zfs is your friend

1) make the 'dir' be a distinct zfs filestore in the zpool
2) run zfssnap on some duty cycle
3) profit

seriously: as long as the copy can be maintained readonly, in sync
with the source, the block level copy of zfs snapshots under some
serial/time cycle, does the job.

I ran this over mbuffer to get around ssh insane packet behaviour, I
only stopped when the client wanted to prune the copy and it ceased to
be a zfs snapshot copy.

Its much faster than rsync. Its at the filesystem block level.

On Thu, Oct 13, 2016 at 3:59 PM, Franco Fichtner  wrote:
>
>> On 13 Oct 2016, at 6:39 AM, reko.turja--- via freebsd-ports 
>>  wrote:
>>
>> The software should be relatively lightweight - no fullblown 
>> mirroring/backup is needed. Also hints how to achieve similar ends using 
>> maybe tar/ssh might do.
>
> Try cpdup(1).
>
>
> Cheers,
> Franco
> ___
> freebsd-ports@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-13 Thread Franco Fichtner 

> On 13 Oct 2016, at 6:39 AM, reko.turja--- via freebsd-ports 
>  wrote:
> 
> The software should be relatively lightweight - no fullblown mirroring/backup 
> is needed. Also hints how to achieve similar ends using maybe tar/ssh might 
> do.

Try cpdup(1).


Cheers,
Franco
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Alternatives to rsync

2016-10-12 Thread Peter Beckman 

1. rsync IS open source: https://rsync.samba.org/
"rsync is an open source utility"

2. Why is GPL3 out of the question? Is the user going to resell the device
   as a service? If the user is simply "using" the software, no disclosure
   of other software is necessary. Only if your user is attempting to "make
   money" from the sale of the software or service powered by rsync does
   the source need to be disclosed.

If the user simply wants to use rsync for their own purposes, GPL3 does not
prevent them from doing so nor does it require that the user disclose all
code written.

If the user IS going to make money off of using rsync, you are correct.

Beckman

On Thu, 13 Oct 2016, reko.turja--- via freebsd-ports wrote:

One of my users is needing rsync like functionality to transfer changed 
contents of some directories between couple of machines. As rsync 3 isn't 
open source, but GPL3 it's out of question in order to keep the system 
untainted.


The software should be relatively lightweight - no fullblown mirroring/backup 
is needed. Also hints how to achieve similar ends using maybe tar/ssh might 
do.


-Reko 
___

freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"



---
Peter Beckman  Internet Guy
beck...@angryox.com http://www.angryox.com/
---
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Alternatives to rsync

2016-10-12 Thread reko.turja--- via freebsd-ports 
One of my users is needing rsync like functionality to transfer changed 
contents of some directories between couple of machines. As rsync 3 isn't 
open source, but GPL3 it's out of question in order to keep the system 
untainted.


The software should be relatively lightweight - no fullblown 
mirroring/backup is needed. Also hints how to achieve similar ends using 
maybe tar/ssh might do.


-Reko 


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"