Re: [v2] jdupes 1.11.1 port to BusyBox
On 12/19/2018 8:16 AM, Tito wrote: On 11/12/18 18:29, Jody Bruchon wrote: This updated version of the jdupes duplicate file finder port to BusyBox adds a whole host of feature exclusions that allow further reduction of the applet's size. Nearly all of the final actions performed by jdupes (deletion, linking, etc.) can be done by piping the output to a shell script anyway, though it's not as convenient or quick as having the program do it for you. All final actions other than printing match sets can now be disabled, and all of the progress bar code can now be tossed out as well. With the new exclusions available, a BusyBox defconfig with a fully stripped down jdupes weighs in just barely under 10K: $ size busybox_with* text data bss dec hex filename 1009790 6763 3648 1020201 f9129 busybox_with_jdupes 1000754 6747 3512 1011013 f6d45 busybox_with_jdupes_minimal 991308 6731 3216 1001255 f4727 busybox_without_jdupes Modified time support being disabled will reduce the size of the heavily used per-file struct by a time_t and throws out time checks in a some "hot" code paths, speeding things up a bit more. The clear downside is that this opens the gates wide for TOCTTOU errors if chosen and I've noted that in the help text. Hi, by looking at the code it seems not to be very busyboxified, no xmalloc, bb_error_msg, getopt32 nor any other libbb function, do you plan to use them? Ciao, Tito I've now converted it to use several libbb functions including x[z]malloc and bb_error_msg_and_die, eliminating lots of fprintf/exit combos along the way. I haven't yet converted getopt_long to getopt32long. This is what the size output looks like after those changes: $ size busybox*miscutils/jdupes.o text data bss dec hex filename 1005784 6755 3664 1016203 f818b busybox 1009790 6763 3648 1020201 f9129 busybox_with_jdupes 991308 6731 3216 1001255 f4727 busybox_without_jdupes 16945 8 392 17345 43c1 miscutils/jdupes.o I've dropped 4K from the final binary. I'll get the getopt32long conversion done and re-submit. Thanks! Jody ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
On 11/12/18 18:29, Jody Bruchon wrote: This updated version of the jdupes duplicate file finder port to BusyBox adds a whole host of feature exclusions that allow further reduction of the applet's size. Nearly all of the final actions performed by jdupes (deletion, linking, etc.) can be done by piping the output to a shell script anyway, though it's not as convenient or quick as having the program do it for you. All final actions other than printing match sets can now be disabled, and all of the progress bar code can now be tossed out as well. With the new exclusions available, a BusyBox defconfig with a fully stripped down jdupes weighs in just barely under 10K: $ size busybox_with* text data bss dec hex filename 1009790 6763 3648 1020201 f9129 busybox_with_jdupes 1000754 6747 3512 1011013 f6d45 busybox_with_jdupes_minimal 991308 6731 3216 1001255 f4727 busybox_without_jdupes Modified time support being disabled will reduce the size of the heavily used per-file struct by a time_t and throws out time checks in a some "hot" code paths, speeding things up a bit more. The clear downside is that this opens the gates wide for TOCTTOU errors if chosen and I've noted that in the help text. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox Hi, by looking at the code it seems not to be very busyboxified, no xmalloc, bb_error_msg, getopt32 nor any other libbb function, do you plan to use them? Ciao, Tito ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
On 12/19/2018 7:10 AM, Xabier Oneca -- xOneca wrote: Hi, The redundant unknown option text is there so the big 'case' has a default block. It should never be reached, but it's there for safety just in case. I suppose the fprintf() could be changed out to the new unsupported_option() function for a tiny space savings, though it's probably not worth a whole new submission just to swap that one line out and the only gain would be reduction of the string size by a few bytes. I meant the other way around: leave the default case, and remove each disabled case. So you have only one fprintf for all disabled options. Ah, I see. I could probably do that. Cheers, Xabier Oneca_,,_ ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
Hi, > The redundant unknown option text is there so the big 'case' has a default > block. It should never be reached, but it's there for safety just in case. I > suppose the fprintf() could be changed out to the new unsupported_option() > function for a tiny space savings, though it's probably not worth a whole new > submission just to swap that one line out and the only gain would be > reduction of the string size by a few bytes. I meant the other way around: leave the default case, and remove each disabled case. So you have only one fprintf for all disabled options. Cheers, Xabier Oneca_,,_ ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
On 12/19/2018 5:02 AM, Xabier Oneca -- xOneca wrote:
Hello Jody,
Why are you treating disabled options with the message 'Option %s is
disabled in this build', when you already provide similar message for
unknown options ('Sorry, using '-%c' is not supported.')? Won't save
much, but I see them a bit redundant...
The redundant unknown option text is there so the big 'case' has a
default block. It should never be reached, but it's there for safety
just in case. I suppose the fprintf() could be changed out to the new
unsupported_option() function for a tiny space savings, though it's
probably not worth a whole new submission just to swap that one line out
and the only gain would be reduction of the string size by a few bytes.
Just my two cents,
Xabier Oneca_,,_
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
Hello Jody,
Why are you treating disabled options with the message 'Option %s is
disabled in this build', when you already provide similar message for
unknown options ('Sorry, using '-%c' is not supported.')? Won't save
much, but I see them a bit redundant...
Just my two cents,
Xabier Oneca_,,_
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [v2] jdupes 1.11.1 port to BusyBox
It's been a week since I sent this in. Does anyone have any questions, comments, thoughts or opinions? On 12/11/2018 12:29 PM, Jody Bruchon wrote: This updated version of the jdupes duplicate file finder port to BusyBox adds a whole host of feature exclusions that allow further reduction of the applet's size. Nearly all of the final actions performed by jdupes (deletion, linking, etc.) can be done by piping the output to a shell script anyway, though it's not as convenient or quick as having the program do it for you. All final actions other than printing match sets can now be disabled, and all of the progress bar code can now be tossed out as well. With the new exclusions available, a BusyBox defconfig with a fully stripped down jdupes weighs in just barely under 10K: $ size busybox_with* text data bss dec hex filename 1009790 6763 3648 1020201 f9129 busybox_with_jdupes 1000754 6747 3512 1011013 f6d45 busybox_with_jdupes_minimal 991308 6731 3216 1001255 f4727 busybox_without_jdupes Modified time support being disabled will reduce the size of the heavily used per-file struct by a time_t and throws out time checks in a some "hot" code paths, speeding things up a bit more. The clear downside is that this opens the gates wide for TOCTTOU errors if chosen and I've noted that in the help text. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
[v2] jdupes 1.11.1 port to BusyBox
This updated version of the jdupes duplicate file finder port to BusyBox adds a whole host of feature exclusions that allow further reduction of the applet's size. Nearly all of the final actions performed by jdupes (deletion, linking, etc.) can be done by piping the output to a shell script anyway, though it's not as convenient or quick as having the program do it for you. All final actions other than printing match sets can now be disabled, and all of the progress bar code can now be tossed out as well. With the new exclusions available, a BusyBox defconfig with a fully stripped down jdupes weighs in just barely under 10K: $ size busybox_with* text data bss dec hex filename 1009790 6763 3648 1020201 f9129 busybox_with_jdupes 1000754 6747 3512 1011013 f6d45 busybox_with_jdupes_minimal 991308 6731 3216 1001255 f4727 busybox_without_jdupes Modified time support being disabled will reduce the size of the heavily used per-file struct by a time_t and throws out time checks in a some "hot" code paths, speeding things up a bit more. The clear downside is that this opens the gates wide for TOCTTOU errors if chosen and I've noted that in the help text. /* jdupes (C) 2015-2018 Jody Bruchon Derived from fdupes (C) 1999-2018 Adrian Lopez Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ //config:config JDUPES //config: bool "jdupes (18.9 kb)" //config: default y //config: help //config: A powerful duplicate file finder and manager //config: //config:config FEATURE_JDUPES_BTRFS //config: bool "btrfs dedupe support" //config: select PLATFORM_LINUX //config: default y //config: depends on JDUPES //config: help //config: Support btrfs/XFS block-level deduplication in jdupes //config: //config:config FEATURE_JDUPES_USER_ORDER //config: bool "Parameter order operations" //config: default y //config: depends on JDUPES //config: help //config: Enable jdupes options that use parameter order (-I, -O) //config: //config:config FEATURE_JDUPES_HARDLINKS //config: bool "Hard link support" //config: default y //config: depends on JDUPES //config: help //config: Support hard link checking and creation in jdupes //config: //config:config FEATURE_JDUPES_SYMLINKS //config: bool "Symbolic link support" //config: default y //config: depends on JDUPES //config: help //config: Support symbolic link checking and creation in jdupes //config: //config:config FEATURE_JDUPES_PERMS //config: bool "Permission no-match support (-p)" //config: default y //config: depends on JDUPES //config: help //config: Enable no-match if file user/group different //config: //config:config FEATURE_JDUPES_SUMMARIZE //config: bool "Summarized output support (-m/-M)" //config: default y //config: depends on JDUPES //config: help //config: jdupes options to output a summary of matched file information //config: //config:config FEATURE_JDUPES_DELETE //config: bool "File deletion support (-d)" //config: default y //config: depends on JDUPES //config: help //config: jdupes option for deleting files after processing //config: //config:config FEATURE_JDUPES_EXCLUDE //config: bool "File exclusion support (-X)" //config: default y //config: depends on JDUPES //config: help //config: jdupes option for excluding files from match checking //config: //config:config FEATURE_JDUPES_SOFTABORT //config: bool "Process partials on abort support (-Z)" //config: default y //config: depends on JDUPES //config: help //config: Support jdupes receiving a signal such as CTRL+C and //config: processing files matched up to that point instead of //config: immediately terminating and losing all progress. //config: //con
