Adding timestamp before printing file name
I am using rsync in the following manner rsync -av --inplace --update --log-file=/medhd/a3-swbase/05-nonfsw.log /medhc/a3-swbase/05-nonfsw /medhd/a3-swbase/ Normally the file name is outputted. Would it be possible to also add the time before the name?-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Passing result of shuf to rsync
‐‐‐ Original Message ‐‐‐
On Saturday, September 18, 2021 12:58 AM, Kevin Korb
wrote:
> Quoting from the man page
>
> SORTED TRANSFER ORDER
> Rsync always sorts the specified filenames into its internal
> transfer list. This handles the merging together of the contents of identi‐
> cally named directories, makes it easy to remove duplicate
> filenames, and may confuse someone when the files are transferred in a
> differ‐
> ent order than what was given on the command-line.
So what I am doing is simply rubbish. I need to make separate rsync calls. If
I try to put the results in an array, in there a limit to the number of files
that can be processed?
> If you need a particular file to be transferred prior to another,
> either separate the files into different rsync calls, or consider using
> --delay-updates (which doesn't affect the sorted transfer order,
> but does make the final file-updating phase happen much more rapidly).
>
> and...
>
> NOTE: sorting the list of files in the --files-from input
> helps rsync to be more efficient, as it will avoid re-visiting the path
> elements that are shared between adjacent entries. If
> the input is not sorted, some path elements (implied directories) may end
> up being scanned multiple times, and rsync will eventually
> unduplicate them after they get turned into file-list elements.
>
> On 9/17/21 8:41 PM, hancooper wrote:
>
> > ‐‐‐ Original Message ‐‐‐
> > On Saturday, September 18, 2021 12:29 AM, Kevin Korb k...@sanitarium.net
> > wrote:
> >
> > > Well, what you have should function. But rsync is going to sort the list.
> >
> > I want rsync to use the list as given to it by shuf.
> >
> > > On 9/17/21 8:26 PM, hancooper wrote:
> > >
> > > > ‐‐‐ Original Message ‐‐‐
> > > > On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync
> > > > rsync@lists.samba.org wrote:
> > > >
> > > > > Not really sure what you are trying to accomplish here. Seems like it
> > > > > should work the way you have it. Note that many wonky rsync kludges
> > > > > are
> > > > > due to people not realizing that rsync can have multiple source
> > > > > arguments. Instead of the source simply being . it can be a list of
> > > > > stuff. Also, note that -a includes --recursive except when
> > > > > --files-from
> > > > > is in play.
> > > >
> > > > I am trying send via rsync a shuffled list of files.
> > > >
> > > > > On 9/17/21 7:11 PM, hancooper via rsync wrote:
> > > > >
> > > > > > I am trying to pass filename results that have been shuffled, then
> > > > > > pass
> > > > > > the file to rsync.
> > > > > > Not quite sure whether to use --files-from=- or some other way.
> > > > > > shuf -n "$nf" -e ${dpath}/${incl} |
> > > > > > rsync -av --update --files-from=- . "$dst"
> > > > >
> > > > > --
> > > > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > > > > Kevin Korb Phone: (407) 252-6853
> > > > > Systems Administrator Internet:
> > > > > FutureQuest, Inc. ke...@futurequest.net (work)
> > > > > Orlando, Florida k...@sanitarium.net (personal)
> > > > > Web page: https://sanitarium.net/
> > > > > PGP public key available on web site.
> > > > >
> > > > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > > > >
> > > > > ---
> > > > >
> > > > > Please use reply-all for most replies to avoid omitting the mailing
> > > > > list.
> > > > > To unsubscribe or change options:
> > > > > https://lists.samba.org/mailman/listinfo/rsync
> > > > > Before posting, read:
> > > > > http://www.catb.org/~esr/faqs/smart-questions.html
> > >
> > > --
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > >
> > > Kevin Korb Phone: (407) 252-6853
> > >
> > > Systems Administrator Internet:
> > >
> > > FutureQuest, Inc. ke...@futurequest.net (work)
> > >
> > > Orlando, Florida k...@sanitarium.net (personal)
> > >
> > > Web page: https://sanitarium.net/
> > >
> > > PGP public key available on web site.
> > >
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
>
> --
>
> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. ke...@futurequest.net (work)
> Orlando, Florida k...@sanitarium.net (personal)
> Web page: https://sanitarium.net/
> PGP public key available on web site.
> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Passing result of shuf to rsync
Quoting from the man page
SORTED TRANSFER ORDER
Rsync always sorts the specified filenames into its internal
transfer list. This handles the merging together of the contents of identi‐
cally named directories, makes it easy to remove duplicate
filenames, and may confuse someone when the files are transferred in a
differ‐
ent order than what was given on the command-line.
If you need a particular file to be transferred prior to another,
either separate the files into different rsync calls, or consider using
--delay-updates (which doesn't affect the sorted transfer order,
but does make the final file-updating phase happen much more rapidly).
and...
NOTE: sorting the list of files in the --files-from input
helps rsync to be more efficient, as it will avoid re-visiting the path
elements that are shared between adjacent entries. If
the input is not sorted, some path elements (implied directories) may end
up being scanned multiple times, and rsync will eventually
unduplicate them after they get turned into file-list elements.
On 9/17/21 8:41 PM, hancooper wrote:
> ‐‐‐ Original Message ‐‐‐
> On Saturday, September 18, 2021 12:29 AM, Kevin Korb
> wrote:
>
>> Well, what you have should function. But rsync is going to sort the list.
>
> I want rsync to use the list as given to it by shuf.
>
>> On 9/17/21 8:26 PM, hancooper wrote:
>>
>>> ‐‐‐ Original Message ‐‐‐
>>> On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync
>>> rsync@lists.samba.org wrote:
>>>
Not really sure what you are trying to accomplish here. Seems like it
should work the way you have it. Note that many wonky rsync kludges are
due to people not realizing that rsync can have multiple source
arguments. Instead of the source simply being . it can be a list of
stuff. Also, note that -a includes --recursive except when --files-from
is in play.
>>>
>>> I am trying send via rsync a shuffled list of files.
>>>
On 9/17/21 7:11 PM, hancooper via rsync wrote:
> I am trying to pass filename results that have been shuffled, then pass
> the file to rsync.
> Not quite sure whether to use --files-from=- or some other way.
> shuf -n "$nf" -e ${dpath}/${incl} |
> rsync -av --update --files-from=- . "$dst"
--
~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Florida k...@sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
---
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>
>> --
>>
>> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
>> Kevin Korb Phone: (407) 252-6853
>> Systems Administrator Internet:
>> FutureQuest, Inc. ke...@futurequest.net (work)
>> Orlando, Florida k...@sanitarium.net (personal)
>> Web page: https://sanitarium.net/
>> PGP public key available on web site.
>> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone:(407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Passing result of shuf to rsync
‐‐‐ Original Message ‐‐‐
On Saturday, September 18, 2021 12:29 AM, Kevin Korb
wrote:
> Well, what you have should function. But rsync is going to sort the list.
I want rsync to use the list as given to it by shuf.
> On 9/17/21 8:26 PM, hancooper wrote:
>
> > ‐‐‐ Original Message ‐‐‐
> > On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync
> > rsync@lists.samba.org wrote:
> >
> > > Not really sure what you are trying to accomplish here. Seems like it
> > > should work the way you have it. Note that many wonky rsync kludges are
> > > due to people not realizing that rsync can have multiple source
> > > arguments. Instead of the source simply being . it can be a list of
> > > stuff. Also, note that -a includes --recursive except when --files-from
> > > is in play.
> >
> > I am trying send via rsync a shuffled list of files.
> >
> > > On 9/17/21 7:11 PM, hancooper via rsync wrote:
> > >
> > > > I am trying to pass filename results that have been shuffled, then pass
> > > > the file to rsync.
> > > > Not quite sure whether to use --files-from=- or some other way.
> > > > shuf -n "$nf" -e ${dpath}/${incl} |
> > > > rsync -av --update --files-from=- . "$dst"
> > >
> > > --
> > >
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > >
> > > Kevin Korb Phone: (407) 252-6853
> > >
> > > Systems Administrator Internet:
> > >
> > > FutureQuest, Inc. ke...@futurequest.net (work)
> > >
> > > Orlando, Florida k...@sanitarium.net (personal)
> > >
> > > Web page: https://sanitarium.net/
> > >
> > > PGP public key available on web site.
> > >
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > > ---
> > >
> > > Please use reply-all for most replies to avoid omitting the mailing list.
> > > To unsubscribe or change options:
> > > https://lists.samba.org/mailman/listinfo/rsync
> > > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
> --
>
> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. ke...@futurequest.net (work)
> Orlando, Florida k...@sanitarium.net (personal)
> Web page: https://sanitarium.net/
> PGP public key available on web site.
> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Passing result of shuf to rsync
Well, what you have should function. But rsync is going to sort the list.
On 9/17/21 8:26 PM, hancooper wrote:
> ‐‐‐ Original Message ‐‐‐
> On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync
> wrote:
>
>> Not really sure what you are trying to accomplish here. Seems like it
>> should work the way you have it. Note that many wonky rsync kludges are
>> due to people not realizing that rsync can have multiple source
>> arguments. Instead of the source simply being . it can be a list of
>> stuff. Also, note that -a includes --recursive except when --files-from
>> is in play.
>
> I am trying send via rsync a shuffled list of files.
>
>
>> On 9/17/21 7:11 PM, hancooper via rsync wrote:
>>
>>> I am trying to pass filename results that have been shuffled, then pass
>>> the file to rsync.
>>> Not quite sure whether to use --files-from=- or some other way.
>>> shuf -n "$nf" -e ${dpath}/${incl} |
>>> rsync -av --update --files-from=- . "$dst"
>>
>> --
>>
>> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
>> Kevin Korb Phone: (407) 252-6853
>> Systems Administrator Internet:
>> FutureQuest, Inc. ke...@futurequest.net (work)
>> Orlando, Florida k...@sanitarium.net (personal)
>> Web page: https://sanitarium.net/
>> PGP public key available on web site.
>> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,
>> --
>>
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone:(407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Passing result of shuf to rsync
‐‐‐ Original Message ‐‐‐
On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync
wrote:
> Not really sure what you are trying to accomplish here. Seems like it
> should work the way you have it. Note that many wonky rsync kludges are
> due to people not realizing that rsync can have multiple source
> arguments. Instead of the source simply being . it can be a list of
> stuff. Also, note that -a includes --recursive except when --files-from
> is in play.
I am trying send via rsync a shuffled list of files.
> On 9/17/21 7:11 PM, hancooper via rsync wrote:
>
> > I am trying to pass filename results that have been shuffled, then pass
> > the file to rsync.
> > Not quite sure whether to use --files-from=- or some other way.
> > shuf -n "$nf" -e ${dpath}/${incl} |
> > rsync -av --update --files-from=- . "$dst"
>
> --
>
> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. ke...@futurequest.net (work)
> Orlando, Florida k...@sanitarium.net (personal)
> Web page: https://sanitarium.net/
> PGP public key available on web site.
> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,
> --
>
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Passing result of shuf to rsync
Not really sure what you are trying to accomplish here. Seems like it
should work the way you have it. Note that many wonky rsync kludges are
due to people not realizing that rsync can have multiple source
arguments. Instead of the source simply being . it can be a list of
stuff. Also, note that -a includes --recursive except when --files-from
is in play.
On 9/17/21 7:11 PM, hancooper via rsync wrote:
> I am trying to pass filename results that have been shuffled, then pass
> the file to rsync.
> Not quite sure whether to use --files-from=- or some other way.
>
>
> shuf -n "$nf" -e ${dpath}/${incl} |
> rsync -av --update --files-from=- . "$dst"
>
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone:(407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Passing result of shuf to rsync
I am trying to pass filename results that have been shuffled, then pass the
file to rsync.
Not quite sure whether to use --files-from=- or some other way.
shuf -n "$nf" -e ${dpath}/${incl} |
rsync -av --update --files-from=- . "$dst"--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Skip file transfer on detecton of error code
Occasionally rsync issues errors, which result in slowing a system during a transfer. Would it be possible to skip the transfer of the file rather than having rsync continue to insist, trying the transfer many times? rsync: read errors mapping "/media/hagbard/hc1/d5-comms/c-periodc/a-exposn/newsci/2011/n2809.pdf": Input/output error (5)-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
