Re: Solution for rsync overall progress data display
with --no-i-r you at least get some idea of the # of files to check. /kc On Sun, Nov 26, 2017 at 09:34:25PM +, Simon Hobson via rsync said: >> I looking for a solution to display overall rsync progress on an LCD display as a bargraph. >> I have found 2 parameters: >> >> --progress >> This option tells rsync to print information showing the >> progress of the transfer. This gives a bored user something to >> watch. Implies --verbose if it wasn't already specified. >> >> While rsync is transferring a regular file, it updates a >> progress line that looks like this: >> >> 782448 63% 110.64kB/s0:00:04 >> >> But they are not showing the overall progress during the transfer what I need. > >Bear in mind that until the sync is almost finished, rsync does NOT know how much is left to do. AIUI, one thread is running a compare, working down the directory tree and building a list of files that aren't up to date on the target. Another thread is then taking files from this list and syncing them. >So at any point in time, there is a queue of files to be synced which is NOT complete, and a process that's syncing those files one at a time. Until the first thread is done, there isn't even a list of files, and until the sync is running, there isn't information on how much needs to be transferred for each of those files. > >It's well worth reading Andrew Tridgell's PHD thesis where the algorithm is detailed. It's quite readable and gives a good insight into how rsync works. >https://www.samba.org/~tridge/phd_thesis.pdf > > >-- >Please use reply-all for most replies to avoid omitting the mailing list. >To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync >Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ken Chase - Heavy Computing Inc. Guelph Canada -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Solution for rsync overall progress data display
> I looking for a solution to display overall rsync progress on an LCD display > as a bargraph. > I have found 2 parameters: > > --progress > This option tells rsync to print information showing the > progress of the transfer. This gives a bored user something to > watch. Implies --verbose if it wasn't already specified. > > While rsync is transferring a regular file, it updates a > progress line that looks like this: > > 782448 63% 110.64kB/s0:00:04 > > But they are not showing the overall progress during the transfer what I need. Bear in mind that until the sync is almost finished, rsync does NOT know how much is left to do. AIUI, one thread is running a compare, working down the directory tree and building a list of files that aren't up to date on the target. Another thread is then taking files from this list and syncing them. So at any point in time, there is a queue of files to be synced which is NOT complete, and a process that's syncing those files one at a time. Until the first thread is done, there isn't even a list of files, and until the sync is running, there isn't information on how much needs to be transferred for each of those files. It's well worth reading Andrew Tridgell's PHD thesis where the algorithm is detailed. It's quite readable and gives a good insight into how rsync works. https://www.samba.org/~tridge/phd_thesis.pdf -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Solution for rsync overall progress data display
--info=progress2 On 11/26/2017 09:27 AM, netbsd--- via rsync wrote: > Hello, > > I looking for a solution to display overall rsync progress on an LCD > display as a bargraph. > I have found 2 parameters: > > --progress > This option tells rsync to print information > showing the > progress of the transfer. This gives a bored user > something to > watch. Implies --verbose if it wasn't already specified. > > While rsync is transferring a regular file, it > updates a > progress line that looks like this: > > 782448 63% 110.64kB/s 0:00:04 > > But they are not showing the overall progress during the transfer what I > need. > > All I would need is to get data out of rsync either the completed data's > bytes size or completed percentage what I could pipe into another C > program or a text file what my app could periodically read to construct > this graph. > > One solution would be to run a dry rsync, get the number of overall > bytes going to be transferred and keep running du -s /datadir to get the > directories size where the rsync goes into but this is not feasible with > over 1TB data and lot of small files. > > Thank you. > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., signature.asc Description: OpenPGP digital signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Solution for rsync overall progress data display
Hello, I looking for a solution to display overall rsync progress on an LCD display as a bargraph. I have found 2 parameters: --progress This option tells rsync to print information showing the progress of the transfer. This gives a bored user something to watch. Implies --verbose if it wasn't already specified. While rsync is transferring a regular file, it updates a progress line that looks like this: 782448 63% 110.64kB/s0:00:04 But they are not showing the overall progress during the transfer what I need. All I would need is to get data out of rsync either the completed data's bytes size or completed percentage what I could pipe into another C program or a text file what my app could periodically read to construct this graph. One solution would be to run a dry rsync, get the number of overall bytes going to be transferred and keep running du -s /datadir to get the directories size where the rsync goes into but this is not feasible with over 1TB data and lot of small files. Thank you. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
