Can anyone explain why rsync is no longer considered an appropriate
method for fetching Packages files? It's the only mechanism I'm aware of
that makes apt-get update over a 56Kb/s connection complete in a
reasonable length of time. Am I missing something?
Begin forwarded message:
Date: Thu, 7
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sun, 17 Oct 2004 19:11:23 -0700
Source: libfile-rsync-perl
Binary: libfile-rsync-perl
Architecture: source all
Version: 0.34-1
Distribution: unstable
Urgency: low
Maintainer: Ivan Kohler [EMAIL PROTECTED]
Changed-By: Ivan Kohler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Mon, 4 Oct 2004 11:58:18 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.3-1
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Tue, 17 Aug 2004 11:16:13 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.2.pre3.1-1
Distribution: experimental
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sat, 14 Aug 2004 14:11:22 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.2-3
Distribution: unstable
Urgency: high
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sat, 7 Aug 2004 20:43:44 -0700
Source: libfile-rsync-perl
Binary: libfile-rsync-perl
Architecture: source all
Version: 0.33-1
Distribution: unstable
Urgency: low
Maintainer: Ivan Kohler [EMAIL PROTECTED]
Changed-By: Ivan Kohler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Tue, 29 Jun 2004 15:20:06 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.2-2
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Mon, 3 May 2004 14:05:15 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.2-1
Distribution: unstable
Urgency: medium
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Thu, 25 Mar 2004 10:58:55 +0100
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.0.99pre1-1
Distribution: experimental
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Wed, 24 Mar 2004 20:23:34 +0100
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.6.0-3
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Mon, 5 Jan 2004 16:00:22 +0100
Source: rsync
Binary: rsync
Architecture: source i386 alpha
Version: 2.6.0-2
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Thu, 1 Jan 2004 21:09:52 +0100
Source: rsync
Binary: rsync
Architecture: source i386 alpha
Version: 2.6.0-1
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Fri, 19 Dec 2003 22:54:33 +0100
Source: rsync
Binary: rsync
Architecture: source i386 alpha
Version: 2.5.99pre1-1
Distribution: experimental
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Thu, 25 Sep 2003 20:48:00 +0200
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.5.6-1
Distribution: unstable
Urgency: low
Maintainer: Paul Slootman [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
From time to time the question arises on different forums whether it is
possible to efficiently use rsync with apt-get. Recently there has been a
thread here on debian-devel and it was also mentioned in Debian Weekly News
June 24th, 2003. However, I only saw different small parts of a huge
On Sun, Jul 06, 2003 at 01:29:06AM +0200, Andrew Suffield wrote:
It should put them in the package in the order they came from
readdir(), which will depend on the filesystem. This is normally the
order in which they were created,
As long as the file system uses an inefficient approach for
Michael Karcher [EMAIL PROTECTED] writes:
On Sun, Jul 06, 2003 at 01:29:06AM +0200, Andrew Suffield wrote:
It should put them in the package in the order they came from
readdir(), which will depend on the filesystem. This is normally the
order in which they were created,
As long as the
On Sun, 2003-07-06 at 09:27, Goswin Brederlow wrote:
4. (and this is the knockout) rsync support for apt-get is NO
WANTED. rsync uses too much resources (cpu and more relevant IO) on
the server side and a widespread use of rsync for apt-get would choke
the rsync mirrors and do more harm than
On Sun, Jul 06, 2003 at 12:37:00PM +1200, Corrin Lakeland wrote:
4. (and this is the knockout) rsync support for apt-get is NO
WANTED. rsync uses too much resources (cpu and more relevant IO) on
the server side and a widespread use of rsync for apt-get would choke
the rsync mirrors and do
On Sun, 6 Jul 2003, Andrew Suffield wrote:
It should put them in the package in the order they came from
readdir(), which will depend on the filesystem. This is normally the
order in which they were created, and should not vary when
rebuilding. As such, sorting the list probably doesn't
On Sun, Jul 06, 2003 at 10:28:07PM +0200, Koblinger Egmont wrote:
On Sun, 6 Jul 2003, Andrew Suffield wrote:
It should put them in the package in the order they came from
readdir(), which will depend on the filesystem. This is normally the
order in which they were created, and should
Hi,
On 6 Jul 2003, Goswin Brederlow wrote:
2. most of the time you have no old file to rsync against. Only
mirrors will have an old file and they already use rsync.
This is definitely true if you install your system from CD's and then
upgrade it. However, if you keep on upgrading from testing
On Sun, Jul 06, 2003 at 10:12:03PM +0100, Andrew Suffield wrote:
On Sun, Jul 06, 2003 at 10:28:07PM +0200, Koblinger Egmont wrote:
Yes, when saying random order I obviously ment in the order readdir()
returns them. It's random for me. :-)))
It can easily be different on different
alphabetically in
the cpio archive.
So I guess we've already seen pros and cons of sorting the files. (One
thing is missing: we still don't know how efficient rsync is if two
rsyncable tar.gz files contain the same files but in different order.)
The decision is clearly not mine but the Debian developers
will be
returned in the same order as they were created.
However, there's no real need to - that was just an example. As long
as the sequence is more or less stable (which it should be, for
btrees; don't know about htree) then rsync won't be perturbed.
On ext2, as an example, stat()ting or open
On Sun, Jul 06, 2003 at 11:36:34PM +0100, Andrew Suffield wrote:
On Sun, Jul 06, 2003 at 05:48:24PM -0400, Theodore Ts'o wrote:
Err, no. If the htree (hash tree) indexing feature is turned on for
ext2 or ext3 filesystems, they will returned sorted by the hash of the
filename ---
On Sun, Jul 06, 2003 at 07:28:09PM -0400, Matt Zimmerman wrote:
On Sun, Jul 06, 2003 at 11:36:34PM +0100, Andrew Suffield wrote:
On Sun, Jul 06, 2003 at 05:48:24PM -0400, Theodore Ts'o wrote:
Err, no. If the htree (hash tree) indexing feature is turned on for
ext2 or ext3 filesystems,
On Sun, Jul 06, 2003 at 11:36:34PM +0100, Andrew Suffield wrote:
I can only presume this is new or obscure, since everything I tried
had the traditional behaviour. Can't see how to turn it on, either.
It's new for 2.5. Backports to 2.4 are available here:
On Mon, Jul 07, 2003 at 01:01:34AM +0100, Andrew Suffield wrote:
I believe htree == dir_index, so tune2fs(8) and mke2fs(8) have the answer.
My /home has that enabled and readdir() returns files in creation order.
Then you don't have a htree-capable kernel or the directory isn't
indexed.
Hi,
From time to time the question arises on different forums whether it is
possible to efficiently use rsync with apt-get. Recently there has been a
thread here on debian-devel and it was also mentioned in Debian Weekly News
June 24th, 2003. However, I only saw different small parts of a huge
], [4b]) which
makes the files appear in the package in alphabetical order. I don't know
how efficient rsync is if you split a file to some dozens or even hundreds
of parts and shuffle them, and then syncronize this one with the original
version. Anyway, I'm sure that sorting the files cannot hurt
Koblinger Egmont [EMAIL PROTECTED] writes:
Hi,
From time to time the question arises on different forums whether it is
possible to efficiently use rsync with apt-get. Recently there has been a
thread here on debian-devel and it was also mentioned in Debian Weekly News
June 24th, 2003
and ftp mirrors. This will be used to start fetching data before connections
have been opened with peers.
Bittorrent calculates a hash for each block of a file very similar to
what rsync needs to work. Via another small extension rolling
checksums for each block could be included in the protocol
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Sunday 06 July 2003 11:27, Goswin Brederlow wrote:
Koblinger Egmont [EMAIL PROTECTED] writes:
Hi,
From time to time the question arises on different forums whether it is
possible to efficiently use rsync with apt-get. Recently there has
But why at the end of http://home.tiscali.cz:8080/~cz210552/aptrsync.html :
# Get anything we missed due to failed rsync's. [EMAIL PROTECTED] 24 Mar
2002.
os.system('apt-get update')
well, it seems for me this just starts apt-get getting everything all
over again, http_proxy or not. So how
Doing apt-get update just seems to start downloading the Packages.gz
even though we just rsynced Packages.
Tim It could easily be a bug.
Radim It writes HIT! message there and skip this file, because it is
Radim up-to-date by rsync.
Next time I will try with http_proxy unset, because I recall
.
And of course commenting out apt-get update means that if some of the
servers in sources.list don't run rsync, then they won't be hit.
? It just downloaded over
^ I can't quite guess your meaning here.
again for me.
It could easily be a bug. The rsync servers I was hitting randomly
rejected connections, so I didn't reliably get current Packages files
unless I did the apt-get update. I couldn't easily test
Re: Package Lists and Size, linux.debian.devel
Cor Some of the servers run rsync, which works well for the Packages
Cor file, but does not work for the packages themselves.
OK, will putting rsync in one's sources.list as you say below just
affect the Packages file fetching, or also
, not are they in separate
files on the mirrors.
Dunno, if it isn't available then it sounds like a good thing for a feature
request. Though at modem speeds, 12MB only takes one hour.
Cor There are technical solutions to precomputing the diffs used by
Cor rsync, as well as solutions for diffing .gz files. E.g. I
in the last year, so I
think we shouldn't wait.
gzip (1.3.5-4) unstable; urgency=low
* merge patch from Rusty Russell that adds --rsyncable option to gzip.
This modifies the output stream to allow rsync to transfer updated .gz
files much more effectively. The resulting .gz files should
retrieved from
http://www.debian.org/security/ via a recursive wget. Would it be
possible, and would it be beneficial to anyone else, if they were made
available individually via rsync?
Andrew
[1]
http://lists.debian.org/debian-devel/2003/debian-devel-200305/msg00468.html
On Mon, May 19, 2003 at 06:28:19PM +1000, Andrew Pollock wrote:
This would be made easier if the DSA's were obtainable in a more parseable
format. Currently they're being retrieved from
http://www.debian.org/security/ via a recursive wget.
Perhaps CVS would be easier?
:pserver:[EMAIL
On Mon, 19 May 2003 18:28:19 +1000
AP == Andrew Pollock [EMAIL PROTECTED] wrote:
AP
AP
AP This would be made easier if the DSA's were obtainable in a more parseable
AP format.
DSA'a are available in RDF. There exists a link on the bottom of
www.debian.org/security to it.
--
Alexander
On Mon, May 19, 2003 at 04:25:20PM +0400, Alexander Kotelnikov wrote:
On Mon, 19 May 2003 18:28:19 +1000
AP == Andrew Pollock [EMAIL PROTECTED] wrote:
AP
AP
AP This would be made easier if the DSA's were obtainable in a more
parseable
AP format.
DSA'a are available in RDF. There
: Filesystem based backup system using rsync
A utility to maintain multiple backups on online storage, each backup is
available as a sort of snapshot directory, where common files are shared
between the different backup generations. It uses rsync to do the actual
copying.
.
Backups can
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Fri, 7 Mar 2003 12:03:45 +0100
Source: rsync
Binary: rsync
Architecture: source i386
Version: 2.5.6-0.1
Distribution: unstable
Urgency: low
Maintainer: Philip Hands [EMAIL PROTECTED]
Changed-By: Paul Slootman [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sat, 7 Sep 2002 05:34:02 -0700
Source: libfile-rsync-perl
Binary: libfile-rsync-perl
Architecture: source all
Version: 0.22-1
Distribution: unstable
Urgency: low
Maintainer: Ivan Kohler [EMAIL PROTECTED]
Changed-By: Ivan Kohler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Tue, 7 May 2002 17:23:01 -0400
Source: rsync
Binary: rsync
Architecture: m68k
Version: 2.5.5-0.2
Distribution: unstable
Urgency: medium
Maintainer: Debian/m68k (crest) buildd [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Tue, 7 May 2002 17:23:01 -0400
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.5-0.2
Distribution: unstable
Urgency: medium
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sat, 6 Apr 2002 20:36:16 -0500
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.5-0.1
Distribution: unstable
Urgency: high
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL
http://rsync.samba.org/rsync-and-debian/
I'd appreciate comments.
This is certainly a very informative page. I'd appreciate if the CPU
load problem could be solved somehow.
IMO the versioning patch from Paul Russell is not the right approach
since this is Debian specific and has
Most of the scripts/methods I've seen which downloads Debian packages
with rsync do only single file transfer. IMO this must be much more
server friendly than a multi file transfer (no filelist).
Is it possible run a rsync server with anonymous login but restricted to
single file transfer next
On 13 Apr 2002, Brian May [EMAIL PROTECTED] wrote:
On Fri, Apr 12, 2002 at 10:19:27PM +1000, Donovan Baarda wrote:
The big problem with rproxy is it's implemented in perl (perl: crypto for
There might be some other unrelated program called rproxy that's in
Perl, but the one I wrote certainly
On Thu, 11 Apr 2002, Martin Pool wrote:
I'd appreciate comments.
Hmm...
As you may know I'm both the APT author, administrator of the top level
debian mirrors and associated mirror network. So,
3.2 rsync is too hard on servers
If it is, then I think we should fix the problems, rather than
On Thu, 2002-04-11 at 01:15, Martin Pool wrote:
There seems to be a thread about rsync and Debian packages every
couple of months. I've written up a document which tries to cover all
of the questions and debates. It's pretty informal, but hopefully
will be useful.
http://rsync.samba.org
more than once in a month is 300, which require about 30K
of wasted bandwidth. Rsync uses a similar amount of bandwidth sending
checksums back and forth, if I'm not mistaken.
Additionally, many packages only change the version number as far as
the Packages file is concerned, which is handled quite
On 12 Apr 2002, Jason Gunthorpe [EMAIL PROTECTED] wrote:
nobody8835 25.7 0.3 22120 1740 ?RN Apr10 525:24 rsync --daemon
nobody 22896 5.0 0.3 22828 1992 ?SN Apr11 21:20 rsync --daemon
nobody3907 7.3 0.5 22336 2820 ?RN Apr11 15:30 rsync --daemon
On 12 Apr 2002, Martin Pool [EMAIL PROTECTED] wrote:
I've changed my opinion on this since we last talked, partly because
of taking over rsync itself,
... what I meant, but people other than jgg probably didn't know, is
that I was looking at installing rproxy and I'd now rather fix rsync
On 11 Apr 2002, Robert Tiberius Johnson [EMAIL PROTECTED] wrote:
Thanks for all your hard work on rsync. I think it is a great tool.
I'm especially excited to hear it is used in Intermezzo. I like your
rsync/debian web page.
I'm glad you like them.[0]
I feel you aren't fair to diff
On Fri, Apr 12, 2002 at 01:28:01PM +1000, Martin Pool wrote:
On 12 Apr 2002, Brian May [EMAIL PROTECTED] wrote:
I think some more details is required regarding rproxy.
[...]
AFAIK, it solves all the problems regarding server load discussed in
rsync, doesn't it???
Why did you think
On Fri, Apr 12, 2002 at 06:19:07PM +1000, Martin Pool wrote:
What I'd really like is to have access to one of these machines and be
able to attach debuggers to rsync and see what it's doing. (In this case,
that would mean being able to ssh in as 'nobody', or something
equivalent.) I realize
% on average or somesuch. It would be nice
if someone could find that email or repeat the experiments.
With the help of an admin from an rsync server I tested the download of
a with gzip --rsyncable compressed Packages.gz versus an uncompressed
Packages. The results are under
http
On Fri, Apr 12, 2002 at 10:19:27PM +1000, Donovan Baarda wrote:
The big problem with rproxy is it's implemented in perl (perl: crypto for
Are we talking about the same code here?
[502] [snoopy:unstable:bam] ~ ldd /usr/sbin/rproxy
libc.so.6 = /lib/libc.so.6 (0x4001e000)
On Fri, Apr 12, 2002 at 01:28:01PM +1000, Martin Pool wrote:
Why did you think that?
rproxy addresses a rather different problem to rsync: for example, it
transfers only one file at a time, not whole directories. No, rproxy
does not have a magic way to do the delta computation in zero time
version of Goswin Brederlow's proposal to use the
reverse rsync algorithm over HTTP Range requests. The code is included
below: please look it over to make sure I got the protocol right.
Here's what I found by running the program on some Packages[.gz] files:
Using gzip -9 --rsyncable Packages
There seems to be a thread about rsync and Debian packages every
couple of months. I've written up a document which tries to cover all
of the questions and debates. It's pretty informal, but hopefully
will be useful.
http://rsync.samba.org/rsync-and-debian/
I'd appreciate comments
On Thu, 11 Apr 2002, Martin Pool wrote:
There seems to be a thread about rsync and Debian packages every
couple of months. I've written up a document which tries to cover all
of the questions and debates. It's pretty informal, but hopefully
will be useful.
http://rsync.samba.org/rsync
On Thu, Apr 11, 2002 at 06:15:43PM +1000, Martin Pool wrote:
There seems to be a thread about rsync and Debian packages every
couple of months. I've written up a document which tries to cover all
of the questions and debates. It's pretty informal, but hopefully
will be useful.
http
On Fri, 12 Apr 2002, Brian May wrote:
I think some more details is required regarding rproxy.
Why is nobody actively developing it?
AFAIK, it solves all the problems regarding server load discussed in
rsync, doesn't it???
No. I tested it out, and it still hits the server hard
of answer is possible for that question, since
it is really
foreach programmer in the world:
why are they not hacking on rproxy?
Nevertheless I have tried to explain more about it on the web site.
AFAIK, it solves all the problems regarding server load discussed in
rsync, doesn't it???
Why did
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Fri, 15 Mar 2002 15:23:06 +
Source: rsync
Binary: rsync
Architecture: m68k
Version: 2.5.4-1
Distribution: unstable
Urgency: high
Maintainer: Debian/m68k Build Daemon [EMAIL PROTECTED]
Changed-By: Philip Hands [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Fri, 15 Mar 2002 15:23:06 +
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.4-1
Distribution: unstable
Urgency: high
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Philip Hands [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sat, 26 Jan 2002 20:40:00 +0100
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.2-0.1
Distribution: unstable
Urgency: high
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Noel Koethe [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Thu, 3 Jan 2002 20:00:01 -0500
Source: rsync
Binary: rsync
Architecture: m68k
Version: 2.5.1-0.1
Distribution: unstable
Urgency: low
Maintainer: Debian/m68k Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Thu, 3 Jan 2002 20:00:01 -0500
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.1-0.1
Distribution: unstable
Urgency: low
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Mon, 31 Dec 2001 03:53:47 -0500
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.0-0.3
Distribution: unstable
Urgency: medium
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sun, 30 Dec 2001 03:46:11 -0500
Source: rsync
Binary: rsync
Architecture: m68k
Version: 2.5.0-0.2
Distribution: unstable
Urgency: medium
Maintainer: Debian/m68k - slam [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL PROTECTED
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Format: 1.7
Date: Sun, 30 Dec 2001 03:46:11 -0500
Source: rsync
Binary: rsync
Architecture: sparc
Version: 2.5.0-0.2
Distribution: unstable
Urgency: medium
Maintainer: Debian/sparc Build Daemon [EMAIL PROTECTED]
Changed-By: Colin Walters [EMAIL
wouldn't it be great if you could just download what has changed
on some package ? for exemple
the maintainer changes something in /etc/init.d/sendmail and you
have to download 1mo .
with rsync you would just download the part that changed ...
well just an idea
from the secret journal of Jean Charles ([EMAIL PROTECTED]):
wouldn't it be great if you could just download what has changed
on some package ? for exemple
There was a HUGE flameware on this issue a few months ago. Check the
archives and be sure you have something positive to add before you
On Fri, 27 Apr 2001, Jean Charles wrote:
wouldn't it be great if you could just download what has changed
on some package ? for exemple
the maintainer changes something in /etc/init.d/sendmail and you
have to download 1mo .
with rsync you would just download the part that changed
No, this won't work with very many compression algorithms. Most
algorithms update their dictionaries/probability tables dynamically based
on input. There isn't just one static table that could be used for
another file, since the table is automatically updated after every (or
near
gzip --compress-like=old-foo foo
gzip creates a dictionary (that gets realy large) of strings that are
used and encodes references to them. At the start the dictionary is
empty, so the first char is pretty much unencoded and inserted into
the dictionary. The next char is encoded using the
the patch it forces gzip to compress the binary
into chunks of 8K. So every 8K theres a break where rsync can try to
match blocks. It seems to help somehow, but I think it handles
movement of data in a file badly (like when a line is inserted).
It's much cleverer than that. The blocks
gzip --compress-like=old-foo foo
where foo will be compressed as old-foo was or as aquivalent as
possible. Gzip does not need to know anything about foo except how it
was compressed. The switch --compress-like could be added to any
compression algorithmus (bzip?) as long as
gzip --compress-like=old-foo foo
AFAIK thats NOT possible with gzip. Same with bzip2.
Why not.
I wish it where that simple.
I'm not saying it's simple, I'm saying it's possible. I'm not a
compression speciallist but from the theory there is nothing which
prevents this
checksum and if it matches it flushes the dictionary. Most likely two
similar files will therefore flush the dictionary at exactly the same
places. If two files are equal after such a flush, the data will be
encoded the same way and rsync can match those blocks.
The author claims that it takes about
== Otto Wyss [EMAIL PROTECTED] writes:
gzip --compress-like=old-foo foo where foo will be
compressed as old-foo was or as aquivalent as possible. Gzip
does not need to know anything about foo except how it was
compressed. The switch --compress-like could be added to
== Jason Gunthorpe [EMAIL PROTECTED] writes:
On 7 Jan 2001, Bdale Garbee wrote:
gzip --rsyncable, aloready implemented, ask Rusty Russell.
I have a copy of Rusty's patch, but have not applied it since I
don't like diverging Debian packages from upstream this way.
as the source?
Are you going to hack at *every* different kind of file format that you
might ever want to rsync, to make it rsync friendly?
Surely it makes more sense to make rsync able to more efficiently deal with
different formats easily.
I think you reach the right conclusion, but for the wrong
a compressed result
with the same (or similar) difference rate as the source?
Are you going to hack at *every* different kind of file format that you
might ever want to rsync, to make it rsync friendly?
Surely it makes more sense to make rsync able to more efficiently deal
to rsync, to make it rsync friendly?
No, I want rsync not even to be mentioned. All I want is something
similar to
gzip --compress-like=old-foo foo
where foo will be compressed as old-foo was or as aquivalent as
possible. Gzip does not need to know anything about foo except how it
was compressed
at *every* different kind of file format that you
might ever want to rsync, to make it rsync friendly?
No, I want rsync not even to be mentioned. All I want is something
similar to
gzip --compress-like=old-foo foo
where foo will be compressed as old-foo was or as aquivalent as
possible
No, I want rsync not even to be mentioned. All I want is something
similar to
gzip --compress-like=old-foo foo
where foo will be compressed as old-foo was or as aquivalent as
possible. Gzip does not need to know anything about foo except how it
was compressed. The switch
at *every* different kind of file format
that you might ever want to rsync, to make it rsync friendly?
No, I want rsync not even to be mentioned. All I want is
something similar to
gzip --compress-like=old-foo foo
AFAIK thats NOT possible with gzip. Same
== Andrew Lenharth [EMAIL PROTECTED] writes:
What is better and easier is to ensure that the compression is
deturministic (gzip by default is not, bzip2 seems to be), so
that rsync can decompress, rsync, compress, and get the exact
file back on the other side.
gzip
There was a few discussions on the rsync mailing lists about how to
handle compressed files, specifically .debs
I'd like to see some way of handling it better, but I don't think
it'll happen at the rsync end. Reasons include higher server cpu load
to (de)compress every file that is transferred
== John O Sullivan [EMAIL PROTECTED] writes:
There was a few discussions on the rsync mailing lists about
how to handle compressed files, specifically .debs I'd like to
see some way of handling it better, but I don't think it'll
happen at the rsync end. Reasons include
It's commonly agreed that compression does prevent rsync from profit of
older versions of packages when synchronizing Debian mirrors. All the
discussion about fixing rsync to solve this, even trough a deb-plugin is
IMHO not the right way. Rsync's task is to synchronize files without
knowing what's
201 - 300 of 341 matches
Mail list logo
