[Wireshark-dev] Optimizer options with autotools and CMake?
I've noticed, on occasion, that sometimes the CMake builds on the UN*X buildbots get warnings that the autotools builds don't. Recently, I tried to figure out what was different about the CMake builds; after some fixes that removed some incorrect differences between them, I eventually stumbled across the way to get CMake to produce makefiles that did verbose builds, printing the compile command as part of the build output, and found that the difference was very simple - autotools builds weren't building with a -O option, while CMake builds were building with -O2. The optimizer is what does some, if not all, of the dataflow analysis that produces use before set warnings, so those warnings were showing up on the CMake builds but not the autotools builds. So: Is there any reason not to do whatever is necessary to use a -O option on the autotools builds? In addition to getting extra warnings from optimized builds, we also presumably want optimized code in the binaries we ship, and want to at least require third-party packagers to explicitly decide *not* to ship optimized code. Should we use -O2, -Os, or some other option (for both autotools *and* CMake)? The GCC 4.9.1 manual says of -O2 Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. As compared to -O, this option increases both compilation time and the performance of the generated code. and of -Os: Optimize for size. -Os enables all -O2 optimizations that do not typically increase code size. It also performs further optimizations designed to reduce code size. -Os disables the following optimization flags: -falign-functions -falign-jumps -falign-loops -falign-labels -freorder-blocks -freorder-blocks-and-partition -fprefetch-loop-arrays and the Clang manual at http://clang.llvm.org/docs/UsersManual.html says, err, umm, nothing useful, because it really sucks at documenting command-line options. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Optimizer options with autotools and CMake?
On Sun, Aug 3, 2014 at 1:40 PM, Guy Harris g...@alum.mit.edu wrote: I've noticed, on occasion, that sometimes the CMake builds on the UN*X buildbots get warnings that the autotools builds don't. Recently, I tried to figure out what was different about the CMake builds; after some fixes that removed some incorrect differences between them, I eventually stumbled across the way to get CMake to produce makefiles that did verbose builds, printing the compile command as part of the build output, and found that the difference was very simple - autotools builds weren't building with a -O option, while CMake builds were building with -O2. I could have sworn my autotools builds used -O2... *goes to check* Yup, my master-1.12 vanilla autotools build adds -O2 to CFLAGS and CXXFLAGS. I don't have a master autotools build handy, but I can't recall any changes that would have affected that... The optimizer is what does some, if not all, of the dataflow analysis that produces use before set warnings, so those warnings were showing up on the CMake builds but not the autotools builds. So: Is there any reason not to do whatever is necessary to use a -O option on the autotools builds? In addition to getting extra warnings from optimized builds, we also presumably want optimized code in the binaries we ship, and want to at least require third-party packagers to explicitly decide *not* to ship optimized code. Should we use -O2, -Os, or some other option (for both autotools *and* CMake)? The GCC 4.9.1 manual says of -O2 Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. As compared to -O, this option increases both compilation time and the performance of the generated code. and of -Os: Optimize for size. -Os enables all -O2 optimizations that do not typically increase code size. It also performs further optimizations designed to reduce code size. -Os disables the following optimization flags: -falign-functions -falign-jumps -falign-loops -falign-labels -freorder-blocks -freorder-blocks-and-partition -fprefetch-loop-arrays and the Clang manual at http://clang.llvm.org/docs/UsersManual.html says, err, umm, nothing useful, because it really sucks at documenting command-line options. I vote that: 1. In development (i.e. the default when getting the git repository) we default to -Og for compilers which support it. The GCC 4.9.1 manual says: Optimize debugging experience. -Og enables optimizations that do not interfere with debugging. It should be the optimization level of choice for the standard edit-compile-debug cycle, offering a reasonable level of optimization while maintaining fast compilation and a good debugging experience. And fall back to -O2 when the compiler does not support -Og (older versions of GCC... I dunno about clang). 2. In release builds (including source tarballs and buildbots) we default to -O2. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Optimizer options with autotools and CMake?
On Aug 3, 2014, at 11:01 AM, Evan Huus eapa...@gmail.com wrote: On Sun, Aug 3, 2014 at 1:40 PM, Guy Harris g...@alum.mit.edu wrote: I've noticed, on occasion, that sometimes the CMake builds on the UN*X buildbots get warnings that the autotools builds don't. Recently, I tried to figure out what was different about the CMake builds; after some fixes that removed some incorrect differences between them, I eventually stumbled across the way to get CMake to produce makefiles that did verbose builds, printing the compile command as part of the build output, and found that the difference was very simple - autotools builds weren't building with a -O option, while CMake builds were building with -O2. I could have sworn my autotools builds used -O2... *goes to check* Yup, my master-1.12 vanilla autotools build adds -O2 to CFLAGS and CXXFLAGS. I don't have a master autotools build handy, but I can't recall any changes that would have affected that... Autotools changing to default to -O2 in a version later than the version installed on the 32-bit OS X buildbot (which I think is running Snow Leopard, so the autotools version might be the one that shipped with Snow Leopard - that was before Apple stopped shipping autotools)? ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Optimizer options with autotools and CMake?
On 08/03/2014 01:40 PM, Guy Harris wrote: I've noticed, on occasion, that sometimes the CMake builds on the UN*X buildbots get warnings that the autotools builds don't. Recently, I tried to figure out what was different about the CMake builds; after some fixes that removed some incorrect differences between them, I eventually stumbled across the way to get CMake to produce makefiles that did verbose builds, printing the compile command as part of the build output, and found that the difference was very simple - autotools builds weren't building with a -O option, while CMake builds were building with -O2. The optimizer is what does some, if not all, of the dataflow analysis that produces use before set warnings, so those warnings were showing up on the CMake builds but not the autotools builds. So: Is there any reason not to do whatever is necessary to use a -O option on the autotools builds? Hmmm, that's funny: my (autofoo-generated) Makefile's have -O2 in CFLAGS and CXXFLAGS (I had noticed because sometimes I have to disable it to get optimization out of the way when debugging). Reading this thread: https://lists.gnu.org/archive/html/autoconf/2006-04/msg00032.html It seems that this is because autoconf (or my distribution's build of it anyway) chose this default for me. Were chasing down this difference on the Mac buildbots or the Ubuntu one(s)? Maybe the answer is a simple autoconf upgrade away. Then again, if there are enough autoconfs out there that don't have this (pretty sane, IMO) default then doing it ourselves is probably not unreasonable. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Optimizer options with autotools and CMake?
On Sun, Aug 3, 2014 at 2:10 PM, Guy Harris g...@alum.mit.edu wrote: On Aug 3, 2014, at 11:01 AM, Evan Huus eapa...@gmail.com wrote: On Sun, Aug 3, 2014 at 1:40 PM, Guy Harris g...@alum.mit.edu wrote: I've noticed, on occasion, that sometimes the CMake builds on the UN*X buildbots get warnings that the autotools builds don't. Recently, I tried to figure out what was different about the CMake builds; after some fixes that removed some incorrect differences between them, I eventually stumbled across the way to get CMake to produce makefiles that did verbose builds, printing the compile command as part of the build output, and found that the difference was very simple - autotools builds weren't building with a -O option, while CMake builds were building with -O2. I could have sworn my autotools builds used -O2... *goes to check* Yup, my master-1.12 vanilla autotools build adds -O2 to CFLAGS and CXXFLAGS. I don't have a master autotools build handy, but I can't recall any changes that would have affected that... Autotools changing to default to -O2 in a version later than the version installed on the 32-bit OS X buildbot (which I think is running Snow Leopard, so the autotools version might be the one that shipped with Snow Leopard - that was before Apple stopped shipping autotools)? Huh, like Jeff I didn't realize that was an autotools default - I thought we were already setting it ourselves. +1 to explicitly specifying optimization flags on all build systems. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
[Wireshark-dev] Can we now assume IPv6 support on all target OSes?
Currently, there are a few bits of code in Wireshark that check, at compile time, whether INET6 is defined. For Windows nmake builds, we appear to hard-wire INET6 to be defined; the user would have to edit config.nmake to turn it off. For autotools builds, we go through a long series of tests for various flavors of IPv6 support. For CMake builds, we appear to hard-wire INET6 to be defined. Is there any reason to continue to try to handle OSes with no IPv6 support, or should we just get rid of the INET6 #define and unconditionally include the code that it currently controls? ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
[Wireshark-dev] [PATCH] fix a Buffer overrun possible for long command line arguments.
cpotted by cppcheck
Signed-off-by: Toralf Förster toralf.foers...@gmx.de
---
epan/dissectors/dcerpc/idl2wrs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/epan/dissectors/dcerpc/idl2wrs.c b/epan/dissectors/dcerpc/idl2wrs.c
index 8c4f73b..89c1eb0 100644
--- a/epan/dissectors/dcerpc/idl2wrs.c
+++ b/epan/dissectors/dcerpc/idl2wrs.c
@@ -3184,7 +3184,7 @@ int main(int argc, char *argv[])
eth_ft=fopen(ETH_FT, w);
eth_handoff=fopen(ETH_HANDOFF, w);
- sprintf(idlfile, %s.cnf, argv[1]);
+ snprintf(idlfile, %s.cnf, sizeof(idlfile)-4, argv[1]);
fh=fopen(idlfile,r);
if(fh){
readcnffile(fh);
--
2.0.4
___
Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org
Archives:http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Wireshark Git Mirror Maintenance
On 8/3/14, 11:34 AM, Evan Huus wrote: On Mon, May 13, 2013 at 7:54 PM, Gerald Combs ger...@wireshark.org mailto:ger...@wireshark.org wrote: On 5/10/13 1:47 PM, Evan Huus wrote: Hi Gerald I just cloned the Wireshark git mirror onto a new machine and was surprised at how large it was to download. Running an aggressive git gc on the finished clone reduced the disk usage on my machine from ~500MB to ~150MB. I'm a bit surprised - git is supposed to automatically garbage collect repositories when they get too cluttered, but perhaps its threshold for automatic gc is just very high. I pinged Balint (CCed) about this and he suggested running gc on a weekly basis and gc --aggressive on a monthly basis on the server. It would probably save a non-trivial amount of bandwidth in the long term as more people clone the repository. It might be due to our particular circumstances (a bare repository only updated via the mirror script) but git's automatic garbage collection doesn't seem to happen very often. The mirror script runs git gc --auto each time it synchronizes which keeps it from filling up the disk (which happened early on) but as you point out there is room for improvement. I added a cron job that runs git gc --aggressive each week. Here is the output from a manual run, which includes git count-objects -v before and after: 2013-05-13 14:38:12: Started. 2013-05-13 14:38:12: Synchronizing repository wireshark 2013-05-13 14:38:12: Object count start count: 0 size: 0 in-pack: 316591 packs: 45 size-pack: 567146 prune-packable: 0 garbage: 0 2013-05-13 14:38:12: Collecting garbage 2013-05-13 15:09:56: Object count start count: 0 size: 0 in-pack: 316596 packs: 2 size-pack: 127499 prune-packable: 0 garbage: 0 2013-05-13 15:09:56: Done So it's been over a year since this conversation and we have actually migrated to Git/Gerrit so I have no idea what Gerrit is doing in this regard (is there even a real git repository backing it, or is it all internal magic?), but I recently came across [1] which suggests that repeated use of --aggressive maybe wasn't such a good idea after all. It suggests just sticking to regular `git gc` except in cases of large one-time imports (like we did on migration) at which point you should run the apparently-very-slow `git repack -a -d --depth=250 --window=250`. FWIW, a fresh clone from Gerrit right now is 213MB - my local repo is only 161MB, and my current desktop is actually not beefy enough to run the recommended repack command so I have no idea what improvement that would give. It's a real git repository but any operations performed by Gerrit are done using JGit. The weekly automatic number update script runs `gerrit gc --all`, which uses JGit's garbage collector. Many sites including Google appear to run it one or more times a day. We may want to to the same. I tried running git `repack -a -d --depth=250 --window=250` on the server. It ran successfully and shrunk the repository from 248 MB to 208 MB but now the OS X builders are timing out during `git fetch`... ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] Wireshark Git Mirror Maintenance
On Sun, Aug 3, 2014 at 6:20 PM, Gerald Combs ger...@wireshark.org wrote: On 8/3/14, 11:34 AM, Evan Huus wrote: On Mon, May 13, 2013 at 7:54 PM, Gerald Combs ger...@wireshark.org mailto:ger...@wireshark.org wrote: On 5/10/13 1:47 PM, Evan Huus wrote: Hi Gerald I just cloned the Wireshark git mirror onto a new machine and was surprised at how large it was to download. Running an aggressive git gc on the finished clone reduced the disk usage on my machine from ~500MB to ~150MB. I'm a bit surprised - git is supposed to automatically garbage collect repositories when they get too cluttered, but perhaps its threshold for automatic gc is just very high. I pinged Balint (CCed) about this and he suggested running gc on a weekly basis and gc --aggressive on a monthly basis on the server. It would probably save a non-trivial amount of bandwidth in the long term as more people clone the repository. It might be due to our particular circumstances (a bare repository only updated via the mirror script) but git's automatic garbage collection doesn't seem to happen very often. The mirror script runs git gc --auto each time it synchronizes which keeps it from filling up the disk (which happened early on) but as you point out there is room for improvement. I added a cron job that runs git gc --aggressive each week. Here is the output from a manual run, which includes git count-objects -v before and after: 2013-05-13 14:38:12: Started. 2013-05-13 14:38:12: Synchronizing repository wireshark 2013-05-13 14:38:12: Object count start count: 0 size: 0 in-pack: 316591 packs: 45 size-pack: 567146 prune-packable: 0 garbage: 0 2013-05-13 14:38:12: Collecting garbage 2013-05-13 15:09:56: Object count start count: 0 size: 0 in-pack: 316596 packs: 2 size-pack: 127499 prune-packable: 0 garbage: 0 2013-05-13 15:09:56: Done So it's been over a year since this conversation and we have actually migrated to Git/Gerrit so I have no idea what Gerrit is doing in this regard (is there even a real git repository backing it, or is it all internal magic?), but I recently came across [1] which suggests that repeated use of --aggressive maybe wasn't such a good idea after all. It suggests just sticking to regular `git gc` except in cases of large one-time imports (like we did on migration) at which point you should run the apparently-very-slow `git repack -a -d --depth=250 --window=250`. FWIW, a fresh clone from Gerrit right now is 213MB - my local repo is only 161MB, and my current desktop is actually not beefy enough to run the recommended repack command so I have no idea what improvement that would give. It's a real git repository but any operations performed by Gerrit are done using JGit. The weekly automatic number update script runs `gerrit gc --all`, which uses JGit's garbage collector. Many sites including Google appear to run it one or more times a day. We may want to to the same. I tried running git `repack -a -d --depth=250 --window=250` on the server. It ran successfully and shrunk the repository from 248 MB to 208 MB but now the OS X builders are timing out during `git fetch`... Hmm, that's interesting, I would have expected a bigger improvement (given my local copy is still smaller than the one on the server). Perhaps it is worth trying an --aggressive gc just once (or passing the -f and -F flags to the existing repack command, which is probably even *more* aggressive). No idea why the buildbots would be timing out... the gc shouldn't have materially affected their ability to pull down deltas I don't think. ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
[Wireshark-dev] PD Column in Gerrit
Gerrit currently displays the status of the Petri-Dish flag in the change-summary list as a PD column. I don't think it's particularly useful, and it makes it harder to tell at a glance whether a change has been verified or not (since the two columns are right next to each other so you can't as easily scan for the green checkmark). Does anybody else prefer to keep it shown? Is there even a way to remove it (I looked in preferences but didn't see anything). Thoughts? Evan ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
[Wireshark-dev] The recent changes to proto.c appear to have broken things badly ...
Specifically: For some/many/all? dissectors, the protocol never appears in the 'protocol' column', isn't in the list of protos, filters for the protocol don't work. etc etc I guess something fails with respect to the proto_tree_add_item(..., proto_..., ...) call. Oddly enough, the actual dissection for the protocol does appear in the details pane. I believe the changes (5460d7f 3da89d6) should be reverted until they are properly tested/fixed. (When i reverted these two commits to proto.c, things were OK again. Bill ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
Re: [Wireshark-dev] The recent changes to proto.c appear to have broken things badly ...
On 8/3/2014 11:32 PM, Evan Huus wrote: On Sun, Aug 3, 2014 at 11:20 PM, Bill Meier wme...@newsguy.com mailto:wme...@newsguy.com wrote: Specifically: For some/many/all? dissectors, the protocol never appears in the 'protocol' column', isn't in the list of protos, filters for the protocol don't work. etc etc I guess something fails with respect to the proto_tree_add_item(..., proto_..., ...) call. Oddly enough, the actual dissection for the protocol does appear in the details pane. I believe the changes (5460d7f 3da89d6) should be reverted until they are properly tested/fixed. (When i reverted these two commits to proto.c, things were OK again. Bill OK, yes, this is very strange. The result of that change should be only that we *don't* fake the tree item in certain uncommon cases - it certainly shouldn't be causing wider problems like this. My understanding is that we should be able to, e.g. randomly not fake the tree 10% of the time without causing problems as it is an optimization only, so I'm not sure why adding *any* extra condition at all would break things like this. Is TRY_TO_FAKE_THIS_ITEM ever more than an optimization? Does anyone else know why *not* faking the tree could cause problems? See Bug #10345. If someone can confirm this, I do believe that the change should be reverted until the issue is straightened out. Bill ___ Sent via:Wireshark-dev mailing list wireshark-dev@wireshark.org Archives:http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
