[Bug 8450] --link-dest seems not to work mounted NTFS file systems

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8450

--- Comment #3 from Toralf Förster  2011-09-13 06:54:29 
UTC ---
FWIW : I ofc tried to append a "/" to "target dir" but this issue still do
exist.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Duration

[David Somers-Harris]

A block level tool won't do a differential back up like rsync does will it?

On Tue, Sep 13, 2011 at 11:04, Kevin Korb  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> The rsync is doing a whole lot more than transferring 600MB of data.  It
> is transferring 600MB of differences.  It is going through 27.4 million
> files on one end and possibly that many on the other.  That is a 402MB
> listing of files plus as many as 54.8 million calls to stat().
>
> Rsync is not about minimizing transfer time.  It is about minimizing
> network IO at the expense of CPU and disk IO.  Rsync reduced your
> network IO from 185.39GB to 600MB only 200 of which was actually
> differences in your data.
>
> IOW rsync did its job.
>
> The unfortunate truth is that no file based tool will perform anywhere
> near wire speeds when it has millions of files to deal with.  If you
> want something wire speed then you should look for a block level tool.
>
> On 09/12/11 21:54, David Somers-Harris wrote:
> > No, I don't have checksum based checking enabled.
> >
> > Sorry for neglecting to mention the options I'm using, here they are:
> >
> > *rsync -vah --exclude=".*" --delete --stats*
> >
> > Would anything in there cause it to be unnecessarily slow? It's taking 3
> > hours but shouldn't take more than 5 minutes...
> >
> >
> > On Tue, Sep 13, 2011 at 00:02, Sandon Van Ness  > > wrote:
> >
> > It could definitely be normal if say you have checksum based
> > checking enabled...
> >
> >
> > On 09/12/2011 01:56 AM, David Somers-Harris wrote:
> >
> >
> > Is it normal for rsync to take 3 hours on this transfer?
> >
> > Number of files: 27419348
> > Number of files transferred: 19501
> > Total file size: 185.39G bytes
> > Total transferred file size: 195.92M bytes
> > Literal data: 195.68M bytes
> > Matched data: 241.09K bytes
> > File list size: 402.01M
> > File list generation time: 0.561 seconds
> > File list transfer time: 0.000 seconds
> > Total bytes sent: 600.61M
> > Total bytes received: 2.44M
> >
> > sent 600.61M bytes  received 2.44M bytes  54.42K bytes/sec
> > total size is 185.39G  speedup is 307.42
> >
> >
> > A normal transfer of 600 MB for me between these two servers
> > only takes a few seconds Is there some way I can speed this
> up?
> >
> >
> >
>
> - --
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>Kevin Korb  Phone:(407) 252-6853
>Systems Administrator   Internet:
>FutureQuest, Inc.   ke...@futurequest.net  (work)
>Orlando, Floridak...@sanitarium.net (personal)
>Web page:   http://www.sanitarium.net/
>PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.17 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk5uuhYACgkQVKC1jlbQAQcV1wCeLLN+f45ydJre5UHmrdIj77FK
> OakAoOMdRTPLvdnSFSEX9407LiWuEMH3
> =O2vB
> -END PGP SIGNATURE-
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Duration

[Kevin Korb]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

The rsync is doing a whole lot more than transferring 600MB of data.  It
is transferring 600MB of differences.  It is going through 27.4 million
files on one end and possibly that many on the other.  That is a 402MB
listing of files plus as many as 54.8 million calls to stat().

Rsync is not about minimizing transfer time.  It is about minimizing
network IO at the expense of CPU and disk IO.  Rsync reduced your
network IO from 185.39GB to 600MB only 200 of which was actually
differences in your data.

IOW rsync did its job.

The unfortunate truth is that no file based tool will perform anywhere
near wire speeds when it has millions of files to deal with.  If you
want something wire speed then you should look for a block level tool.

On 09/12/11 21:54, David Somers-Harris wrote:
> No, I don't have checksum based checking enabled.
> 
> Sorry for neglecting to mention the options I'm using, here they are:
> 
> *rsync -vah --exclude=".*" --delete --stats*
> 
> Would anything in there cause it to be unnecessarily slow? It's taking 3
> hours but shouldn't take more than 5 minutes...
> 
> 
> On Tue, Sep 13, 2011 at 00:02, Sandon Van Ness  > wrote:
> 
> It could definitely be normal if say you have checksum based
> checking enabled...
> 
> 
> On 09/12/2011 01:56 AM, David Somers-Harris wrote:
> 
> 
> Is it normal for rsync to take 3 hours on this transfer?
> 
> Number of files: 27419348
> Number of files transferred: 19501
> Total file size: 185.39G bytes
> Total transferred file size: 195.92M bytes
> Literal data: 195.68M bytes
> Matched data: 241.09K bytes
> File list size: 402.01M
> File list generation time: 0.561 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 600.61M
> Total bytes received: 2.44M
> 
> sent 600.61M bytes  received 2.44M bytes  54.42K bytes/sec
> total size is 185.39G  speedup is 307.42
> 
> 
> A normal transfer of 600 MB for me between these two servers
> only takes a few seconds Is there some way I can speed this up?
> 
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb  Phone:(407) 252-6853
Systems Administrator   Internet:
FutureQuest, Inc.   ke...@futurequest.net  (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page:   http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5uuhYACgkQVKC1jlbQAQcV1wCeLLN+f45ydJre5UHmrdIj77FK
OakAoOMdRTPLvdnSFSEX9407LiWuEMH3
=O2vB
-END PGP SIGNATURE-
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Duration

[David Somers-Harris]

No, I don't have checksum based checking enabled.

Sorry for neglecting to mention the options I'm using, here they are:

*rsync -vah --exclude=".*" --delete --stats*

Would anything in there cause it to be unnecessarily slow? It's taking 3
hours but shouldn't take more than 5 minutes...


On Tue, Sep 13, 2011 at 00:02, Sandon Van Ness  wrote:

> It could definitely be normal if say you have checksum based checking
> enabled...
>
>
> On 09/12/2011 01:56 AM, David Somers-Harris wrote:
>
>>
>> Is it normal for rsync to take 3 hours on this transfer?
>>
>> Number of files: 27419348
>> Number of files transferred: 19501
>> Total file size: 185.39G bytes
>> Total transferred file size: 195.92M bytes
>> Literal data: 195.68M bytes
>> Matched data: 241.09K bytes
>> File list size: 402.01M
>> File list generation time: 0.561 seconds
>> File list transfer time: 0.000 seconds
>> Total bytes sent: 600.61M
>> Total bytes received: 2.44M
>>
>> sent 600.61M bytes  received 2.44M bytes  54.42K bytes/sec
>> total size is 185.39G  speedup is 307.42
>>
>>
>> A normal transfer of 600 MB for me between these two servers only takes a
>> few seconds Is there some way I can speed this up?
>>
>>
>
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 8450] --link-dest seems not to work mounted NTFS file systems

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8450

--- Comment #2 from Toralf Förster  2011-09-12 16:45:32 
UTC ---
Ah - now the pciture becomes clear - I changed my script few days ago to use
relative paths - which doesn't work:

When I specify :

target dir: /mnt/media/daily/a
   ref dir:  ../../daily/f

it doesn't work, however this works :

target dir: /mnt/media/daily/a
   ref dir:  /mnt/media/daily/f


OTOH the relative path itself is ok :

n22 ~ # ls -ld /mnt/media/daily/a
drwxr-xr-x 17 root root 4096 Aug 12 18:28 /mnt/media/daily/a

n22 ~ # ls -ld /mnt/media/daily/a/../../daily/f
drwxr-xr-x 17 root root 4096 Sep 12 17:22 /mnt/media/daily/a/../../daily/f

which let me thinkig, that rsync appends the relative path to  in this
way (which is not intended, but I cannot specify an additional slash "/" - then
it would be an absolute path) :

n22 ~ # ls -ld /mnt/media/daily/a../../daily/f
ls: cannot access /mnt/media/daily/a../../daily/f: No such file or directory

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 8450] --link-dest seems not to work mounted NTFS file systems

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8450

--- Comment #1 from Kevin Korb  2011-09-12 16:11:17 UTC 
---
Try running with --itemize-changes.  It will tell you why rsync thinks the file
needs to be updated.

(Sorry for the duplicate reply on the email list.  I replied there before I
realized this was from the bugzilla)

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 8450] New: --link-dest seems not to work mounted NTFS file systems

[Kevin Korb]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

See what --itemize-changes says.

On 09/12/11 12:06, samba-b...@samba.org wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=8450
> 
>Summary: --link-dest seems not to work mounted NTFS file
> systems
>Product: rsync
>Version: 3.0.8
>   Platform: x86
> OS/Version: Linux
> Status: NEW
>   Severity: normal
>   Priority: P5
>  Component: core
> AssignedTo: way...@samba.org
> ReportedBy: toralf.foers...@gmx.de
>  QAContact: rsync...@samba.org
> 
> 
> I use rsync to create daily/weekly/monthly backups to an external USB drive of
> my Gentoo Linux using this command line options :
> 
> --archive --delete --delete-excluded --exclude-from=/exclude.list
> --link-dest= --stats --verbose  
> 
> This works like a charm for all files located on my ext3 Linux partition.
> However for the 2 files of a NTFS drive not.
> 
> Here rsync always transfers an unchanged file too, even if  contains
> that file (from a backup made yesterday) and therefore it should better be
> hard-linked only instead of being transferred twice.
> 
> What is really curious - at the beginning of using this backup solution (some
> weeks ago) the  doesn't contain an older version of that file, and there
> hard-linking worked IIRC.
> 
> The NTFS drive is mounted read-only:
> 
> tfoerste@n22 ~ $ mount | grep C
> /dev/sda1 on /mnt/C type ntfs
> (ro,noexec,nosuid,nodev,noatime,uid=1000,gid=100,umask=0027)
> 
>   is /mnt/C/notes/data/archive/
>  is /mnt/media/daily/f/
>  is /mnt/media/weekly/b (given as ../../weekly/b at the command 
> line)
> 
> Here you can see that different hard links are used :
> 
> tfoerste@n22 ~ $ ls -il /mnt/C/notes/data/archive/l_DE1448.nsf
> /mnt/media/daily/f/archive/l_DE1448.nsf 
> /mnt/media/weekly/b/archive/l_DE1448.nsf 
>   80719 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
> /mnt/C/notes/data/archive/l_DE1448.nsf
> 2293796 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
> /mnt/media/daily/f/archive/l_DE1448.nsf
> 2310874 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
> /mnt/media/weekly/b/archive/l_DE1448.nsf
> 
> md5sum proves identical content, the file attributes shouldn't changed too
> (NTFS is always mounted read-only, never read-write) :
> 
> tfoerste@n22 ~ $ md5sum /mnt/C/notes/data/archive/l_DE1448.nsf
> /mnt/media/daily/f/archive/l_DE1448.nsf 
> /mnt/media/weekly/b/archive/l_DE1448.nsf 
> 88ef0d516c5696bf120d749f4b1fc9ba  /mnt/C/notes/data/archive/l_DE1448.nsf
> 88ef0d516c5696bf120d749f4b1fc9ba  /mnt/media/daily/f/archive/l_DE1448.nsf
> 88ef0d516c5696bf120d749f4b1fc9ba  /mnt/media/weekly/b/archive/l_DE1448.nsf
> 
> OTOH does it make any difference whether the  drive at the external USB
> drive is mounted as ext2 instead of ext3 ?
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin Korb  Phone:(407) 252-6853
Systems Administrator   Internet:
FutureQuest, Inc.   ke...@futurequest.net  (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page:   http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5uLoQACgkQVKC1jlbQAQfHPQCg9IiwPEolmcJuRAEtjMJoRXcN
cOYAnRHQlXruI2M7qVnbSPGy6h7IOgni
=C60/
-END PGP SIGNATURE-
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 8450] New: --link-dest seems not to work mounted NTFS file systems

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8450

   Summary: --link-dest seems not to work mounted NTFS file
systems
   Product: rsync
   Version: 3.0.8
  Platform: x86
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
AssignedTo: way...@samba.org
ReportedBy: toralf.foers...@gmx.de
 QAContact: rsync...@samba.org


I use rsync to create daily/weekly/monthly backups to an external USB drive of
my Gentoo Linux using this command line options :

--archive --delete --delete-excluded --exclude-from=/exclude.list
--link-dest= --stats --verbose  

This works like a charm for all files located on my ext3 Linux partition.
However for the 2 files of a NTFS drive not.

Here rsync always transfers an unchanged file too, even if  contains
that file (from a backup made yesterday) and therefore it should better be
hard-linked only instead of being transferred twice.

What is really curious - at the beginning of using this backup solution (some
weeks ago) the  doesn't contain an older version of that file, and there
hard-linking worked IIRC.

The NTFS drive is mounted read-only:

tfoerste@n22 ~ $ mount | grep C
/dev/sda1 on /mnt/C type ntfs
(ro,noexec,nosuid,nodev,noatime,uid=1000,gid=100,umask=0027)

  is /mnt/C/notes/data/archive/
 is /mnt/media/daily/f/
 is /mnt/media/weekly/b (given as ../../weekly/b at the command line)

Here you can see that different hard links are used :

tfoerste@n22 ~ $ ls -il /mnt/C/notes/data/archive/l_DE1448.nsf
/mnt/media/daily/f/archive/l_DE1448.nsf 
/mnt/media/weekly/b/archive/l_DE1448.nsf 
  80719 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
/mnt/C/notes/data/archive/l_DE1448.nsf
2293796 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
/mnt/media/daily/f/archive/l_DE1448.nsf
2310874 -r-xr-x--- 1 tfoerste users 8126464 Sep  2 13:40
/mnt/media/weekly/b/archive/l_DE1448.nsf

md5sum proves identical content, the file attributes shouldn't changed too
(NTFS is always mounted read-only, never read-write) :

tfoerste@n22 ~ $ md5sum /mnt/C/notes/data/archive/l_DE1448.nsf
/mnt/media/daily/f/archive/l_DE1448.nsf 
/mnt/media/weekly/b/archive/l_DE1448.nsf 
88ef0d516c5696bf120d749f4b1fc9ba  /mnt/C/notes/data/archive/l_DE1448.nsf
88ef0d516c5696bf120d749f4b1fc9ba  /mnt/media/daily/f/archive/l_DE1448.nsf
88ef0d516c5696bf120d749f4b1fc9ba  /mnt/media/weekly/b/archive/l_DE1448.nsf

OTOH does it make any difference whether the  drive at the external USB
drive is mounted as ext2 instead of ext3 ?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 8445] Add a non-trusted filter-file option that would limit the rules and ignore syntax errors

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8445

--- Comment #2 from Ruediger Meier  2011-09-12 10:45:12 UTC 
---
Thx, for this detailed reply. After reading I think we have to 2 different
issues here.


(In reply to comment #1)
> (In reply to comment #0)
> > invalid modifier sequence at 't' in filter rule: -/tmp
> 
> You'll note that rule is missing a space, so it was a fitler-rule syntax 
> error.

1. I'am sure the there was never a syntax error in .rsync-filer. Instead the
error occurred because the user added effectively a single character while
rsync was reading it. (The same reason why bash scripts show syntax errors when
editing them during execution).
So think it would be worth to improve rsync's way of reading the filter files
all about because rsync is suppossed to run for hours to sync directories while
they are used and it's able to handle vanished files etc.
I'd even wondered why rsync has read that particuar .rsync-filer again after
being 10 hours inside that directory already. 
I haven't watched the source code but I guess it would help simply to avoid
file operations like fseek on the filter files.



>  Rsync treats a failure to parse filter rules as something that it should
> complain about in a fatal error so that you get a chance to fix it.

2. So I this would put this on the wishlist:
 new option --ignore-broken-filters
 Behaves like in case of vanished files. Just print a warning but don't exit an
ignore the broken filter. When sync is finished exit 2; 



> So, it seems to me that the issue here is that you're trusting user-generated
> filter rules in a backup situation, which may not be a good idea

Because all our users have to do with very large amount of data I want them to
help me with the filter rules.


> (e.g. consider
> the inclusion of a filter-rule import that references a secret file in order 
> to
> try to sniff its contents).

My users can only write the filter files into their own dirs. If they want to
backup their own secrets then this is not my problem.


> What you could do instead is to do a pre-copy
> restrictive parse of all the filter files in the backup hierarchy and turn 
> them
> into a single set of global rules, dropping any syntax error lines and 
> ignoring
> any rules that shouldn't be trusted

This would be possible and I even though about this to implement more
intelligent filters than simple in/exclude lists. But in practice
find /home -name ".rsync-filter"
takes about 1-2 hours here with high IO load on the file server and it would
slow down the whole backup process about 20-30%.




> Another option is to mark the rules in the filter files as only hide rules 

A good idea regarding the security points above but regarding point 1 it woud
be a fake. rsync would not exit with fatal error but would use a totally messed
up filter if user changed it during backup process.


cu,
Rudi

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Ignoring /boot

[Matthias Schniedermeyer]

On 12.09.2011 11:03, Ben Short wrote:
> Hi,
> 
> I have the following script that I'm writing to backup my gentoo linux
> system.
> 
> ...
>
> I would expect boot to have files in it? What is going wrong?

The culprit should be: --one-file-system



Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

RE: Ignoring /boot

[Voelker, Bernhard]

Ben Short wrote:

> I have the following script that I'm writing to backup my gentoo linux system.
> ...
> RSYNC_OPTS="--archive --one-file-system --perms --executability --progress 
> --stats --delete-after --hard-links --keep-dirlinks --verbose --inplace"
> RSYNC_USER="bs"
> RSYNC_SERVER="192.168.6.6"
> RSYNC_MODULE="ben-desktop"
> RSYNC_PATH="/"
> RSYNC_EXCLUDES="--exclude=/usr/portage/distfiles --exclude=/tmp 
> --exclude=/var --exclude=/home --exclude=/root --exclude=/usr --exclude=/bin 
> --exclude=/opt --excl$
> 
> MOUNT_FOR_RSYNC="/boot"
> 
> mount ${MOUNT_FOR_RSYNC}
> 
> #ls -la /boot
> 
> rsync --dry-run ${RSYNC_OPTS} --rsync-path="sudo rsync" ${RSYNC_EXCLUDES} -e 
> ssh ${RSYNC_PATH} ${RSYNC_USER}@${RSYNC_SERVER}::${RSYNC_MODULE}
> 
> umount ${MOUNT_FOR_RSYNC}
> ...
> I would expect boot to have files in it? What is going wrong?

 -x, --one-file-system   don't cross filesystem boundaries

Berny
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Ignoring /boot

[Ben Short]

Hi,

I have the following script that I'm writing to backup my gentoo linux
system.

- start of script -

#!/bin/sh
#
#

RSYNC_OPTS="--archive --one-file-system --perms --executability --progress
--stats --delete-after --hard-links --keep-dirlinks --verbose --inplace"
RSYNC_USER="bs"
RSYNC_SERVER="192.168.6.6"
RSYNC_MODULE="ben-desktop"
RSYNC_PATH="/"
RSYNC_EXCLUDES="--exclude=/usr/portage/distfiles --exclude=/tmp
--exclude=/var --exclude=/home --exclude=/root --exclude=/usr --exclude=/bin
--exclude=/opt --excl$

MOUNT_FOR_RSYNC="/boot"

mount ${MOUNT_FOR_RSYNC}

#ls -la /boot

rsync --dry-run ${RSYNC_OPTS} --rsync-path="sudo rsync" ${RSYNC_EXCLUDES} -e
ssh ${RSYNC_PATH} ${RSYNC_USER}@${RSYNC_SERVER}::${RSYNC_MODULE}

umount ${MOUNT_FOR_RSYNC}

- end of script -

When I run it I get the following output.

ben-desktop ben # ./backup
building file list ...
16 files to consider
./
boot/
dev/
lost+found/
media/
media/.keep_sys-fs_udisks-0
mnt/
mnt/.keep
mnt/exports/
mnt/exports/projects/
mnt/exports/scratch/
mnt/exports/users/
mnt/pbs/
mnt/usb/
proc/
sys/

Number of files: 16
Number of files transferred: 2
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 321
File list generation time: 0.002 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 518
Total bytes received: 56

sent 518 bytes  received 56 bytes  1148.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)


I would expect boot to have files in it? What is going wrong?

Regards

Ben
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Duration

[David Somers-Harris]

Is it normal for rsync to take 3 hours on this transfer?

Number of files: 27419348
Number of files transferred: 19501
Total file size: 185.39G bytes
Total transferred file size: 195.92M bytes
Literal data: 195.68M bytes
Matched data: 241.09K bytes
File list size: 402.01M
File list generation time: 0.561 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 600.61M
Total bytes received: 2.44M

sent 600.61M bytes  received 2.44M bytes  54.42K bytes/sec
total size is 185.39G  speedup is 307.42


A normal transfer of 600 MB for me between these two servers only takes a
few seconds Is there some way I can speed this up?
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html