Review Request 54950: Remove the use of FORTIFY_SOURCE from libprocess.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54950/ --- Review request for mesos and Michael Park. Bugs: MESOS-6829 https://issues.apache.org/jira/browse/MESOS-6829 Repository: mesos Description --- FORTIFY_SOURCE is no longer used when hardening. This is to prevent the warning that some versions of gcc/libc throw when FORTIFY_SOURCE is used without optimizations. The warning turns into an error via -Werror which is applied to MESOS_CPPFLAGS thus failing the whole build. Diffs - 3rdparty/libprocess/Makefile.am c33ae4306 Diff: https://reviews.apache.org/r/54950/diff/ Testing --- Build all of Mesos from source. Thanks, Aaron Wood
Review Request 54949: Remove the use of FORTIFY_SOURCE from Mesos.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54949/ --- Review request for mesos and Michael Park. Bugs: MESOS-6829 https://issues.apache.org/jira/browse/MESOS-6829 Repository: mesos Description --- FORTIFY_SOURCE is no longer used when hardening. This is to prevent the warning that some versions of gcc/libc throw when FORTIFY_SOURCE is used without optimizations. The warning turns into an error via `-Werror` which is applied to `MESOS_CPPFLAGS` thus failing the whole build. Diffs - src/Makefile.am abcf7eed7 Diff: https://reviews.apache.org/r/54949/diff/ Testing --- Build all of Mesos from source. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 30, 2016, 8:52 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Don't warn when stack protection isn't used. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for libprocess. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/libprocess/Makefile.am 9d496b8 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 30, 2016, 8:51 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Don't warn when stack protection isn't used. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for Mesos. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 7750ed7 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 30, 2016, 8:50 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Don't warn when stack protection isn't used. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for stout. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 30, 2016, 5:19 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Fixed newline issue in the macro. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for Mesos. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 7750ed7 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 30, 2016, 5:12 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Fixed newline issue in the macro. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for libprocess. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/libprocess/Makefile.am 9d496b8 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 30, 2016, 5:10 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Fixed newline issue in the macro. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for stout. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 29, 2016, 4:26 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Changed version of macro to work with CentOS 6. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for libprocess. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 29, 2016, 4:24 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Changed the version of the macro to work with CentOS 6. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for stout. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 29, 2016, 4:21 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Add hardened flags for Mesos. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 85eda53 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 29, 2016, 4:15 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for Mesos. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. The macro at 1a869696e4129279f7b99c3f9052717354b79a86 requires autoconf 2.64 which breaks on CentOS 6. Diffs - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 85eda53 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 29, 2016, 4:13 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Changed the version of the macro (taken from 391cb680171d3889965b1ead43d3a326c913bc25) Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for Mesos. Take compile flag macro at 391cb680171d3889965b1ead43d3a326c913bc25. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 85eda53 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
> On Nov. 19, 2016, 12:52 a.m., Michael Park wrote: > > m4/ax_check_compile_flag.m4, line 61 > > <https://reviews.apache.org/r/52645/diff/10/?file=1559579#file1559579line61> > > > > This seems to introduce a new requirement of `autoconf` version 2.64 or > > higher. Ran into this no CentOS 6. > > Benjamin Bannier wrote: > This is really unfortunate, but looking at the upstream history of this > file it appears that the latest upstream version from before this new > requirement was introduced (`391cb680171d3889965b1ead43d3a326c913bc25`) does > the job just as well (I checked this with . The patch introducing this new on > requirement on autoconf-2.64 (`1a869696e4129279f7b99c3f9052717354b79a86`) was > just to remove some antiquated code patterns, > > commit 1a869696e4129279f7b99c3f9052717354b79a86 > Author: Bastien ROUCARIÈS <roucaries.bast...@gmail.com> > Date: Tue Jan 6 18:53:54 2015 +0100 > > Modernize ax_append_flag, ax_check_*_flag > > Use AS_VAR* macro and AS_CASE > > I suggest we go with the version from > `391cb680171d3889965b1ead43d3a326c913bc25` which requires autoconf-2.59 for > now, but call this out in the commit message (SHA we took this from, > requirement to support autoconf-2.63 on centos-6.8). > > Some for the follow-up commits introducing this macro to libprocess and > stout. > > Benjamin Bannier wrote: > !fixup I check this with centos-6.8. Thanks for checking into this! I was just going to do the same, I'll get the remaining patches fixed up with a previous version of that macro. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review156367 --- On Nov. 9, 2016, 7:37 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Nov. 9, 2016, 7:37 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Add hardened flags for Mesos. > Take compile flag macro at 1a869696e4129279f7b99c3f9052717354b79a86. > > > Diffs > - > > configure.ac 5380cbc > m4/ax_check_compile_flag.m4 PRE-CREATION > src/Makefile.am 5a47c93 > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Nov. 17, 2016, 9:01 p.m., Michael Park wrote: > > 3rdparty/libprocess/src/decoder.hpp, lines 253-254 > > <https://reviews.apache.org/r/52647/diff/5/?file=140#file140line253> > > > > Do you happen to know what the request body length has to do with > > `CHAR_MAX` in the first place...? > > Aaron Wood wrote: > I'm not 100% clear on this but my guess is that it's from a negotiated > max body size between the server and clients within Mesos...? > > James Peach wrote: > AFAICT this is assigning the `Content-Length` using `basic_string& > operator=( CharT ch );`, which is entirely broken. Ignore my comment. This looks like a bug since it'll cause requests coming in from clients using gzip to be greatly truncated. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review156238 --- On Nov. 18, 2016, 4:24 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Nov. 18, 2016, 4:24 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp 76dca0b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp ab2b5a9 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 533104c > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/ --- (Updated Nov. 18, 2016, 4:27 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comment about not converting to an unsigned value. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in stout that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp da4deb9 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp b5a206f 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4cb3b5f 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 0b7ee07 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52886/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Nov. 18, 2016, 4:24 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments and made a few small fixes. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in libprocess that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/src/decoder.hpp 76dca0b 3rdparty/libprocess/src/encoder.hpp 005d1cc 3rdparty/libprocess/src/process.cpp ab2b5a9 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 533104c 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Nov. 17, 2016, 9:01 p.m., Michael Park wrote: > > 3rdparty/libprocess/src/decoder.hpp, line 21 > > <https://reviews.apache.org/r/52647/diff/5/?file=140#file140line21> > > > > `#include ` My fault, I have no idea why I decided to add a .h at the time :) - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review156238 --- On Nov. 7, 2016, 4:45 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Nov. 7, 2016, 4:45 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp 76dca0b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp ab2b5a9 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 533104c > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Nov. 17, 2016, 9:01 p.m., Michael Park wrote: > > 3rdparty/libprocess/src/decoder.hpp, lines 253-254 > > <https://reviews.apache.org/r/52647/diff/5/?file=140#file140line253> > > > > Do you happen to know what the request body length has to do with > > `CHAR_MAX` in the first place...? I'm not 100% clear on this but my guess is that it's from a negotiated max body size between the server and clients within Mesos...? - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review156238 ------- On Nov. 7, 2016, 4:45 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Nov. 7, 2016, 4:45 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp 76dca0b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp ab2b5a9 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 533104c > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 9, 2016, 7:37 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for Mesos. Take compile flag macro at 1a869696e4129279f7b99c3f9052717354b79a86. Diffs - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 5a47c93 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 9, 2016, 7:37 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Add `-Werror` back into `MESOS_CPPFLAGS`. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 5a47c93 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 9, 2016, 7:07 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for libprocess. Take compile flag macro at 1a869696e4129279f7b99c3f9052717354b79a86. Diffs - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > 3rdparty/stout/m4/ax_check_compile_flag.m4, line 1 > > <https://reviews.apache.org/r/52696/diff/4/?file=1551005#file1551005line1> > > > > For future updates it would be great if we'd write down the > > autoconf-archive release this file came from (it looks like the latest > > release containing it is `v2016.09.16`). > > Benjamin Bannier wrote: > You marked this as resolved, but I couldn't find the change. Could you > please update e.g., the commit message to include something like > > This commit adds ax_check_compiler_flag.m4 from > git://git.sv.gnu.org/autoconf-archive.git tag v2016.09.16. I've updated the description. That hash points to a diff that was made in 2015 so I figured it was best no to say v2016.09.16. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/#review154526 --- On Nov. 9, 2016, 7:05 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52696/ > --- > > (Updated Nov. 9, 2016, 7:05 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Add hardened flags for stout. > Take compile flag macro at 1a869696e4129279f7b99c3f9052717354b79a86. > > > Diffs > - > > 3rdparty/stout/Makefile.am 4e10ae2 > 3rdparty/stout/configure.ac f071f61 > 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52696/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 9, 2016, 7:05 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description (updated) --- Add hardened flags for stout. Take compile flag macro at 1a869696e4129279f7b99c3f9052717354b79a86. Diffs - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > src/Makefile.am, line 120 > > <https://reviews.apache.org/r/52645/diff/7/?file=1550864#file1550864line120> > > > > Not sure we want to remove the existing `-Werror`. > > Aaron Wood wrote: > From a discussion with a few people on Slack it sounded like this was in > the wrong place to begin with (along with `-Wall` and `-Wsign-compare`). I > found that when I had moved this to `AM_CXXFLAGS` it would actually apply > hard errors for warnings where it never did before. The issue now with > putting this back into `AM_CXXFLAGS` is that all of the `clang: error: > argument unused during compilation: '-pthread'` warnings you get when > building Mesos which were always there now fail the whole thing. I think we'd > have to unravel the overall build process more to fix that issue. > > Aaron Wood wrote: > Dropping this for now. If you feel it's really important we can open it > again :) > > Benjamin Bannier wrote: > I feel it is really important. With this we'd suddenly not fail a build > introducing new warnings anymore. This might lead to people (or some CI) to > conclude such a build would be fine. > > Let's just leave it in `MESOS_CPPFLAGS`. I can add it back to `MESOS_CPPFLAGS` but it never actually caused a failure when there were warnings. For example, the warnings you get on OS X about -pthread not being used would never fail the build when `-Werror` was in `MESOS_CPPFLAGS`. It would when it was in `AM_CXXFLAGS`. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review154524 --- On Nov. 8, 2016, 5:40 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Nov. 8, 2016, 5:40 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 5380cbc > m4/ax_check_compile_flag.m4 PRE-CREATION > src/Makefile.am 5a47c93 > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 8, 2016, 5:41 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 8, 2016, 5:41 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 8, 2016, 5:40 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 5a47c93 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- (Updated Nov. 8, 2016, 5:40 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp a4af54a 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/ --- (Updated Nov. 8, 2016, 5:39 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in stout that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp da4deb9 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp b5a206f 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4cb3b5f 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 0b7ee07 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52886/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- (Updated Nov. 8, 2016, 5:39 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp a4af54a 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- (Updated Nov. 8, 2016, 5:38 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp a4af54a 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 7, 2016, 9:53 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Specify the hash in the commit message for the macro we took (1a869696e4129279f7b99c3f9052717354b79a86). Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 5a47c93 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 7, 2016, 9:52 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Specify the hash in the commit message for the macro we took (1a869696e4129279f7b99c3f9052717354b79a86). Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 7, 2016, 9:51 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Specify the hash in the commit message for the macro we took (1a869696e4129279f7b99c3f9052717354b79a86). Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 7, 2016, 9:30 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments, added a new flag to enable/disable hardening, apply hardening by default. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac e65e5ca 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 7, 2016, 9:27 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments, added a new flag to enable/disable hardening, apply hardening by default. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac f071f61 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 7, 2016, 9:25 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments, added a new flag to enable/disable hardening, apply hardening by default. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 5380cbc m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am 5a47c93 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > src/Makefile.am, line 120 > > <https://reviews.apache.org/r/52645/diff/7/?file=1550864#file1550864line120> > > > > Not sure we want to remove the existing `-Werror`. > > Aaron Wood wrote: > From a discussion with a few people on Slack it sounded like this was in > the wrong place to begin with (along with `-Wall` and `-Wsign-compare`). I > found that when I had moved this to `AM_CXXFLAGS` it would actually apply > hard errors for warnings where it never did before. The issue now with > putting this back into `AM_CXXFLAGS` is that all of the `clang: error: > argument unused during compilation: '-pthread'` warnings you get when > building Mesos which were always there now fail the whole thing. I think we'd > have to unravel the overall build process more to fix that issue. Dropping this for now. If you feel it's really important we can open it again :) - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review154524 ------- On Nov. 1, 2016, 7:37 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Nov. 1, 2016, 7:37 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac c8d48be > m4/ax_check_compile_flag.m4 PRE-CREATION > src/Makefile.am c2f9e44 > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > 3rdparty/stout/Makefile.am, line 27 > > <https://reviews.apache.org/r/52696/diff/4/?file=1551003#file1551003line27> > > > > I am not a big fan of unconditionally omitting frame pointers as this > > gives the optimizer one less register to work with. Unfortunately one > > cannot easily tell the actual impact of this from the info here. Is this > > strictly needed here or just nice to have? Going to drop this since we've all agreed on Slack to have the frame pointer modification done in a separate patch. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/#review154526 ------- On Nov. 2, 2016, 3:35 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52696/ > --- > > (Updated Nov. 2, 2016, 3:35 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > stout. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/stout/Makefile.am 4e10ae2 > 3rdparty/stout/configure.ac cbb0fdb > 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52696/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52695: Harden libprocess
> On Nov. 2, 2016, 9:32 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/Makefile.am, line 30 > > <https://reviews.apache.org/r/52695/diff/4/?file=1551006#file1551006line30> > > > > I am not a big fan of unconditionally omitting frame pointers as this > > gives the optimizer one less register to work with. Unfortunately one > > cannot easily tell the actual impact of this from the info here. Is this > > strictly needed here or just nice to have? > > James Peach wrote: > The performance benefit of omitting frame pointers is likely to be > marginal on x64_64, if it is a win at all. The rationale for adding this is > that it makes stack walking reliable in all cases, so debugability is > improved and you can get reasonable results when uting `perf`. Since most > users will build with default options I suggested to Aaron that we should > make it the default. > > Benjamin Bannier wrote: > Thanks James, that makes sense. > > Since this seems all related to debugability what about enabling it _only > for builds with `--enable-debug`_ (e.g., perf results already now also don't > necessarily give full info w/o debug symbols)? Tangentially related, tcmalloc > can fail in debug builds with omitted frame pointers, so disabling > `omit-frame-pointer` in debug builds might safe us from some future > headaches, https://bugs.chromium.org/p/chromium/issues/detail?id=636489. > > `stack-protector-strong` can significantly increase the binary size, and > we should either only enable it for e.g., debug builds, or give users a > `configure` knob to disable it. > > For using `FORTIFY_SOURCE` I think we also need be a little more careful. > Support for it is somewhat broken in clang > (https://llvm.org/bugs/show_bug.cgi?id=16821), it only has useful effects in > builds with some level of optimization, and can e.g., mess up reports from > sanitizers injected by users. I can see good uses for a `configure` flag to > disable this compiler flag, but I am not sure what the default should be. Going to drop this since we've all agreed on Slack to have the frame pointer modification done in a separate patch. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/#review154527 --- On Nov. 2, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52695/ > --- > > (Updated Nov. 2, 2016, 3:14 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > libprocess. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 7131989 > 3rdparty/libprocess/configure.ac 1644035 > 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52695/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > src/Makefile.am, line 114 > > <https://reviews.apache.org/r/52645/diff/7/?file=1550864#file1550864line114> > > > > I am not a big fan of unconditionally omitting frame pointers as this > > gives the optimizer one less register to work with. Unfortunately one > > cannot easily tell the actual impact of this from the info here. Is this > > strictly needed here or just nice to have? Going to drop this since we've all agreed on Slack to have the frame pointer modification done in a separate patch. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review154524 ------- On Nov. 1, 2016, 7:37 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Nov. 1, 2016, 7:37 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac c8d48be > m4/ax_check_compile_flag.m4 PRE-CREATION > src/Makefile.am c2f9e44 > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Nov. 7, 2016, 4:45 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in libprocess that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/src/decoder.hpp 76dca0b 3rdparty/libprocess/src/encoder.hpp 005d1cc 3rdparty/libprocess/src/process.cpp ab2b5a9 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 533104c 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/ --- (Updated Nov. 7, 2016, 4:11 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in stout that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp da4deb9 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp b5a206f 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4cb3b5f 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 0b7ee07 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52886/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
> On Nov. 2, 2016, 9:54 a.m., Benjamin Bannier wrote: > > 3rdparty/stout/tests/json_tests.cpp, line 175 > > <https://reviews.apache.org/r/52886/diff/2/?file=1547700#file1547700line175> > > > > Not yours, but the ordering of parameters (should be `expected, > > actual`) is wrong here and in many other places in this file. > > > > It would be great if you could submit a separate cleanup patch for that. > > Neil Conway wrote: > FWIW, the next version of gtest changes this to treat the lhs and rhs of > an expectation equivalently > (https://github.com/google/googletest/commit/f364e188372e489230ef4e44e1aec6bcb08f3acf). We okay with dropping this then? - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/#review154529 --- On Oct. 27, 2016, 7:32 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52886/ > --- > > (Updated Oct. 27, 2016, 7:32 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in stout that > need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/stout/tests/cache_tests.cpp 0950c85 > 3rdparty/stout/tests/flags_tests.cpp da4deb9 > 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 > 3rdparty/stout/tests/hashset_tests.cpp 66e59db > 3rdparty/stout/tests/ip_tests.cpp b5a206f > 3rdparty/stout/tests/json_tests.cpp 2bc4c88 > 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 > 3rdparty/stout/tests/multimap_tests.cpp 488991b > 3rdparty/stout/tests/os/process_tests.cpp 4cb3b5f > 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 > 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b > 3rdparty/stout/tests/os_tests.cpp 0b7ee07 > 3rdparty/stout/tests/strings_tests.cpp 7dd3301 > > Diff: https://reviews.apache.org/r/52886/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran make && make check && make bench. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- (Updated Nov. 7, 2016, 3:51 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed recent comments. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp a4af54a 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
> On Nov. 2, 2016, 9:32 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/m4/ax_check_compile_flag.m4, line 1 > > <https://reviews.apache.org/r/52695/diff/4/?file=1551008#file1551008line1> > > > > For future updates it would be great if we'd write down the > > autoconf-archive release this file came from (it looks like the latest > > release containing it is `v2016.09.16`). I don't see any of the other macros having this information. Would you just prefer a comment at the very top indicating the release? I took this from HEAD a few weeks back from this location http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=blob_plain;f=m4/ax_check_compile_flag.m4 How can you tell it's from `v2016.09.16`? - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/#review154527 --- On Nov. 2, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52695/ > --- > > (Updated Nov. 2, 2016, 3:14 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > libprocess. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 7131989 > 3rdparty/libprocess/configure.ac 1644035 > 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52695/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > src/Makefile.am, line 120 > > <https://reviews.apache.org/r/52645/diff/7/?file=1550864#file1550864line120> > > > > Not sure we want to remove the existing `-Werror`. >From a discussion with a few people on Slack it sounded like this was in the >wrong place to begin with (along with `-Wall` and `-Wsign-compare`). I found >that when I had moved this to `AM_CXXFLAGS` it would actually apply hard >errors for warnings where it never did before. The issue now with putting this >back into `AM_CXXFLAGS` is that all of the `clang: error: argument unused >during compilation: '-pthread'` warnings you get when building Mesos which >were always there now fail the whole thing. I think we'd have to unravel the >overall build process more to fix that issue. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review154524 --- On Nov. 1, 2016, 7:37 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Nov. 1, 2016, 7:37 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac c8d48be > m4/ax_check_compile_flag.m4 PRE-CREATION > src/Makefile.am c2f9e44 > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 2, 2016, 3:35 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Added benchmarks that were done on OS X (Core i7 4770HQ @ 2.20 GHz with 16 GB RAM) Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac cbb0fdb 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments (updated) --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
> On Nov. 2, 2016, 9:33 a.m., Benjamin Bannier wrote: > > I would really like to see actual timings of e.g., an optimized build > > before and after introducing these new flags, e.g., the runtime of > > `stout-tests`. Just attached a bunch of benchmarking info that I had saved from when I posted it in the cxx Slack channel. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/#review154526 --- On Nov. 2, 2016, 3:35 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52696/ > --- > > (Updated Nov. 2, 2016, 3:35 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > stout. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/stout/Makefile.am 4e10ae2 > 3rdparty/stout/configure.ac cbb0fdb > 3rdparty/stout/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52696/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/18a2f590-75ad-49c5-a697-56b746f28cae__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/a6e07766-80cc-4bd7-856d-8952cac12562__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/046b37a9-5aff-4543-b3bb-5ac60daaf498__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/3baa96cf-be05-4ac0-ad4c-ef571386e8f4__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52695: Harden libprocess
> On Nov. 2, 2016, 9:32 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/Makefile.am, line 29 > > <https://reviews.apache.org/r/52695/diff/4/?file=1551006#file1551006line29> > > > > Let's not suppress this valid and potentially useful diagnostic for the > > whole codebase. It does not trigger a hard failure anyway. You're right, I had initially set this due to issues with the version of gmock that's used and clang. Now that there's no `-Werror` I'll take it out. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/#review154527 ------- On Nov. 2, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52695/ > --- > > (Updated Nov. 2, 2016, 3:14 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > libprocess. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 7131989 > 3rdparty/libprocess/configure.ac 1644035 > 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52695/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 2, 2016, 3:14 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Added benchmarks that were done on OS X (Core i7 4770HQ @ 2.20 GHz with 16 GB RAM) Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac 1644035 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. File Attachments (updated) --enable-optimized with hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt Hardening applied but no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt --enable-optimized with no hardening applied https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt No hardening applied and no --enable-optimized https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
> On Nov. 2, 2016, 9:32 a.m., Benjamin Bannier wrote: > > I would really like to see actual timings of e.g., an optimized build > > before and after introducing these new flags, e.g., the runtime of > > `libprocess-tests` and `benchmarks`. Just attached a bunch of benchmarking info that I had saved from when I posted it in the cxx Slack channel. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/#review154527 --- On Nov. 2, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52695/ > --- > > (Updated Nov. 2, 2016, 3:14 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > libprocess. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 7131989 > 3rdparty/libprocess/configure.ac 1644035 > 3rdparty/libprocess/m4/ax_check_compile_flag.m4 PRE-CREATION > > Diff: https://reviews.apache.org/r/52695/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > File Attachments > > > --enable-optimized with hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/875c9e6e-c73b-4e3c-8265-0f7c6dc00351__hardened-optimized.txt > Hardening applied but no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/932d28a7-2d31-471a-b438-647841a6853c__hardened-unoptimized.txt > --enable-optimized with no hardening applied > > https://reviews.apache.org/media/uploaded/files/2016/11/02/896944ea-9b31-4d62-b1b9-97fb4700a882__optimized.txt > No hardening applied and no --enable-optimized > > https://reviews.apache.org/media/uploaded/files/2016/11/02/b32667ce-3e3b-4d2b-b4f8-4c2404a0fc1c__unoptimized.txt > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Nov. 2, 2016, 10:52 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/src/encoder.hpp, line 290 > > <https://reviews.apache.org/r/52647/diff/4/?file=1547640#file1547640line290> > > > > I think using an `off_t` for a size is semantically incorrect; I'd stay > > with `size_t`. This requires adjusting the usage above. James Peach and I had a discussion about this and thought that `off_t` is more correct for representing file sizes. Why would you prefer to stick with `size_t`? > On Nov. 2, 2016, 10:52 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/src/process.cpp, line 3735 > > <https://reviews.apache.org/r/52647/diff/4/?file=1547641#file1547641line3735> > > > > While `vector::size_type` is the correct type here, we > > typically just use `size_t`. Wouldn't `container::size_type` be more portable? `size_t` could vary on the platform where `size_type` is container dependent. > On Nov. 2, 2016, 10:52 a.m., Benjamin Bannier wrote: > > 3rdparty/libprocess/src/tests/io_tests.cpp, line 284 > > <https://reviews.apache.org/r/52647/diff/4/?file=1547645#file1547645line284> > > > > `string::size_type` is the correct type, but we typically just use > > `size_t`. > > > > Not directly an issue, but to me casting the signed LHS to an unsigned > > type feels more dangerous than casting the unsigned RHS to signed since I > > feel we seem much less likely deal with very large unsigned values (RHS) > > than with negative numbers close to zero like `-1` on the RHS. I would > > personally would cast the RHS instead. What do you think? Same comment as above for your first comment. I agree with what you're saying about the casting here. I'll swap it around. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review154530 --- On Oct. 27, 2016, 4:51 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Oct. 27, 2016, 4:51 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp c79296b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp ab2b5a9 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 533104c > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 1, 2016, 7:37 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Missed the `ax_check_compile_flag.m4` macro that should have been included with the previous update. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac c8d48be m4/ax_check_compile_flag.m4 PRE-CREATION src/Makefile.am c2f9e44 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Nov. 1, 2016, 7:06 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac c8d48be src/Makefile.am c2f9e44 Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Nov. 1, 2016, 7:12 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Only use `-fstack-protector-strong` if it's available to us. Remove `-Werror` and tackle this in another discussion/JIRA. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/libprocess/Makefile.am 7131989 3rdparty/libprocess/configure.ac 1644035 Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Nov. 1, 2016, 7:10 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Only use `-fstack-protector-strong` when it's available. Remove `-Werror` and tackle this in another discussion/JIRA. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/stout/Makefile.am 4e10ae2 3rdparty/stout/configure.ac cbb0fdb Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/ --- (Updated Oct. 27, 2016, 7:32 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments about casting/types. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in stout that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp da4deb9 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp b5a206f 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4cb3b5f 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 0b7ee07 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52886/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
> On Oct. 17, 2016, 3:39 p.m., James Peach wrote: > > 3rdparty/stout/tests/ip_tests.cpp, line 50 > > <https://reviews.apache.org/r/52886/diff/1/?file=1538071#file1538071line50> > > > > Probably better to just use the same type here. Since > > ``network.get().prefix()`` returns ``int``, consider > > ``numify(prefix).get()``. Good idea! - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/#review152886 ------- On Oct. 21, 2016, 6:29 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52886/ > --- > > (Updated Oct. 21, 2016, 6:29 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in stout that > need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/stout/tests/cache_tests.cpp 0950c85 > 3rdparty/stout/tests/flags_tests.cpp 94ba915 > 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 > 3rdparty/stout/tests/hashset_tests.cpp 66e59db > 3rdparty/stout/tests/ip_tests.cpp 59e69a5 > 3rdparty/stout/tests/json_tests.cpp 2bc4c88 > 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 > 3rdparty/stout/tests/multimap_tests.cpp 488991b > 3rdparty/stout/tests/os/process_tests.cpp 4977d02 > 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 > 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b > 3rdparty/stout/tests/os_tests.cpp 6a7b836 > 3rdparty/stout/tests/strings_tests.cpp 7dd3301 > > Diff: https://reviews.apache.org/r/52886/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran make && make check && make bench. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Oct. 27, 2016, 4:51 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Addressed comments about the encoder changes. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in libprocess that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/src/decoder.hpp c79296b 3rdparty/libprocess/src/encoder.hpp 005d1cc 3rdparty/libprocess/src/process.cpp ab2b5a9 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 533104c 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/subprocess_tests.cpp 0dc1c62 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- (Updated Oct. 25, 2016, 8:32 p.m.) Review request for mesos, James Peach, Michael Park, and Neil Conway. Changes --- Declare/assign PROFILE_FILE only when `--enable-perftools` is used. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp 3936f47 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52754: Remove unused code which now throws errors with the new hardening flags
> On Oct. 24, 2016, 6 p.m., James Peach wrote: > > 3rdparty/libprocess/src/profiler.cpp, line 35 > > <https://reviews.apache.org/r/52754/diff/1/?file=1531401#file1531401line35> > > > > Why remove this? It looks like it is actually used? You're right, it does look like it's used. Without removing it I get: ``` ../../../3rdparty/libprocess/src/profiler.cpp:35:12: error: unused variable 'PROFILE_FILE' [-Werror,-Wunused-const-variable] const char PROFILE_FILE[] = "perftools.out"; ``` I get the same error if I take it out of the unnamed namespace. I don't see it defined anywhere else in the codebase. Is it coming from some third party source...? - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/#review153715 --- On Oct. 21, 2016, 6:31 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52754/ > --- > > (Updated Oct. 21, 2016, 6:31 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new errors about unused functions, > variables, etc. that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/include/process/profiler.hpp f6fccfb > 3rdparty/libprocess/src/profiler.cpp 0c4949e > 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c > 3rdparty/libprocess/src/tests/process_tests.cpp 3936f47 > 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf > > Diff: https://reviews.apache.org/r/52754/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran make && make check && make bench. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
> On Oct. 18, 2016, 3:15 a.m., James Peach wrote: > > 3rdparty/stout/Makefile.am, line 27 > > <https://reviews.apache.org/r/52696/diff/2/?file=1529825#file1529825line27> > > > > Where does ``VARIANTS`` come from? I need to fix this. I wrongly thought VARIANTS was something that was set elsewhere. I'll apply the appropriate flags in another way. Ideally this was supposed to apply -fPIC and -fPIE only to shared libs. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/#review153025 --- On Oct. 21, 2016, 6:29 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52696/ > --- > > (Updated Oct. 21, 2016, 6:29 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > stout. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/stout/Makefile.am fda069d > > Diff: https://reviews.apache.org/r/52696/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
> On Oct. 24, 2016, 5:50 p.m., James Peach wrote: > > src/Makefile.am, line 114 > > <https://reviews.apache.org/r/52645/diff/5/?file=1529823#file1529823line114> > > > > I wasn't able to figure this line out, so it probably needs a comment > > to explain where ``VARIANTS`` comes from and what this filtering is doing. I need to fix this. I wrongly thought `VARIANTS` was something that was set elsewhere. I'll apply the appropriate flags in another way. Ideally this was supposed to apply `-fPIC` and `-fPIE` only to shared libs. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review153713 --- On Oct. 21, 2016, 6:31 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 21, 2016, 6:31 p.m.) > > > Review request for mesos, James Peach, Michael Park, and Neil Conway. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 034bb91 > src/Makefile.am fd01e1d > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Oct. 18, 2016, 3:14 a.m., James Peach wrote: > > 3rdparty/libprocess/src/tests/io_tests.cpp, line 284 > > <https://reviews.apache.org/r/52647/diff/3/?file=1538065#file1538065line284> > > > > Can you just make ``length`` type ``ssize_t``? `length` is `ssize_t` (set on line 235) - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review153024 --- On Oct. 14, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Oct. 14, 2016, 3:14 p.m.) > > > Review request for mesos, Michael Park and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp c79296b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp 18a8e20 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 8d6c8c4 > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
> On Oct. 18, 2016, 3:14 a.m., James Peach wrote: > > 3rdparty/libprocess/src/encoder.hpp, line 291 > > <https://reviews.apache.org/r/52647/diff/3/?file=1538060#file1538060line291> > > > > It's not obvious to me that this is the right change since the > > surrounding code tries to deal with ``off_t``. Can you explain this some > > more? In the `backup` method on line 276 it's being compared with a `size_t` as well as in the `remaining` method on line 283. But in `next` on line 267 `size` which is a `size_t` is being casted to an `off_t` and then assigned to `index`. The other encoders above in that file seem to be working with their own private `index` as a `size_t` so even though it looks like it's being used in different ways in `FileEncoder` I thought it should be a `size_t`. Since this specific class is for files maybe we should keep it as `off_t` and just cast the things used for comparisons to `off_t`. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review153024 ------- On Oct. 14, 2016, 3:14 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Oct. 14, 2016, 3:14 p.m.) > > > Review request for mesos, Michael Park and Neil Conway. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new sign comparison errors in libprocess > that need to be fixed for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/src/decoder.hpp c79296b > 3rdparty/libprocess/src/encoder.hpp 005d1cc > 3rdparty/libprocess/src/process.cpp 18a8e20 > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 8d6c8c4 > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Oct. 14, 2016, 3:20 p.m.) Review request for mesos, Michael Park and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 020b0e1 Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Oct. 14, 2016, 3:20 p.m.) Review request for mesos, Michael Park and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/stout/Makefile.am fda069d Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 14, 2016, 3:20 p.m.) Review request for mesos, Michael Park and Neil Conway. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Review Request 52886: Fix new sign comparison errors in stout produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52886/ --- Review request for mesos, Michael Park and Neil Conway. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new sign comparison errors in stout that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp 94ba915 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp 59e69a5 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4977d02 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 6a7b836 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52886/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52647: Fix new sign comparison errors in libprocess produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Oct. 14, 2016, 3:14 p.m.) Review request for mesos, Michael Park and Neil Conway. Changes --- Targeted this RR at libprocess. There will be a separate one opened for stout. Summary (updated) - Fix new sign comparison errors in libprocess produced by hardened flags Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description (updated) --- The hardening flags produced many new sign comparison errors in libprocess that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/src/decoder.hpp c79296b 3rdparty/libprocess/src/encoder.hpp 005d1cc 3rdparty/libprocess/src/process.cpp 18a8e20 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 8d6c8c4 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review152381 --- src/Makefile.am (line 112) <https://reviews.apache.org/r/52645/#comment221381> Move these into `configure.ac`. - Aaron Wood On Oct. 11, 2016, 10:47 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 11, 2016, 10:47 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 034bb91 > src/Makefile.am fd01e1d > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/#review152315 --- 3rdparty/libprocess/Makefile.am (line 29) <https://reviews.apache.org/r/52695/#comment221282> Only use `-fstack-protector-strong` if we have GCC >= 4.9. - Aaron Wood On Oct. 11, 2016, 10:47 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52695/ > --- > > (Updated Oct. 11, 2016, 10:47 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > libprocess. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 020b0e1 > > Diff: https://reviews.apache.org/r/52695/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/#review152316 --- 3rdparty/stout/Makefile.am (line 26) <https://reviews.apache.org/r/52696/#comment221284> Only use `-fstack-protector-strong` if we have GCC >= 4.9. - Aaron Wood On Oct. 11, 2016, 10:47 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52696/ > --- > > (Updated Oct. 11, 2016, 10:47 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > stout. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/stout/Makefile.am fda069d > > Diff: https://reviews.apache.org/r/52696/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Oct. 11, 2016, 10:47 p.m.) Review request for mesos and Michael Park. Changes --- Depend on the other RR for fixing new errors. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/stout/Makefile.am fda069d Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 11, 2016, 10:47 p.m.) Review request for mesos and Michael Park. Changes --- Depend on the other RR for fixing new errors. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Oct. 11, 2016, 10:47 p.m.) Review request for mesos and Michael Park. Changes --- Depend on the other RR for fixing new errors. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 020b0e1 Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Review Request 52754: Remove unused code which now throws errors with the new hardening flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52754/ --- Review request for mesos and Michael Park. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new errors about unused functions, variables, etc. that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/process_tests.cpp 3936f47 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf Diff: https://reviews.apache.org/r/52754/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran make && make check && make bench. Thanks, Aaron Wood
Re: Review Request 52647: Fix new sign comparison errors produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Oct. 11, 2016, 10:43 p.m.) Review request for mesos and Michael Park. Changes --- Target this RR for sign comparison fixes only. Summary (updated) - Fix new sign comparison errors produced by hardened flags Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description (updated) --- The hardening flags produced many new sign comparison errors that need to be fixed for Mesos to compile/run. Diffs (updated) - 3rdparty/libprocess/src/decoder.hpp c79296b 3rdparty/libprocess/src/encoder.hpp 005d1cc 3rdparty/libprocess/src/process.cpp f1d746c 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 8d6c8c4 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/subprocess_tests.cpp c8350cf 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp 94ba915 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp 59e69a5 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 4977d02 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp 6a7b836 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52647: Fix new errors/warnings produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/#review152192 --- Need to break this up into one RR for the sign compare fixes and another RR for the unused code removals. - Aaron Wood On Oct. 10, 2016, 3:51 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52647/ > --- > > (Updated Oct. 10, 2016, 3:51 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6239 > https://issues.apache.org/jira/browse/MESOS-6239 > > > Repository: mesos > > > Description > --- > > The hardening flags produced many new warnings/errors that need to be fixed > for Mesos to compile/run. > > > Diffs > - > > 3rdparty/libprocess/include/process/profiler.hpp f6fccfb > 3rdparty/libprocess/src/encoder.hpp af083d1 > 3rdparty/libprocess/src/process.cpp 02a1925 > 3rdparty/libprocess/src/profiler.cpp 0c4949e > 3rdparty/libprocess/src/socket.cpp 1e49518 > 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c > 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 > 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 > 3rdparty/libprocess/src/tests/http_tests.cpp 2538f56 > 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f > 3rdparty/libprocess/src/tests/process_tests.cpp b9feec7 > 3rdparty/libprocess/src/tests/subprocess_tests.cpp 66ccff9 > 3rdparty/stout/tests/cache_tests.cpp 0950c85 > 3rdparty/stout/tests/flags_tests.cpp 94ba915 > 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 > 3rdparty/stout/tests/hashset_tests.cpp 66e59db > 3rdparty/stout/tests/ip_tests.cpp 59e69a5 > 3rdparty/stout/tests/json_tests.cpp 2bc4c88 > 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 > 3rdparty/stout/tests/multimap_tests.cpp 488991b > 3rdparty/stout/tests/os/process_tests.cpp 1e26877 > 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 > 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b > 3rdparty/stout/tests/os_tests.cpp c2900b8 > 3rdparty/stout/tests/strings_tests.cpp 7dd3301 > > Diff: https://reviews.apache.org/r/52647/diff/ > > > Testing > --- > > Made sure compilation, tests, and benchmarks worked with both gcc and clang. > Ran `make && make check && make bench`. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review152137 --- src/Makefile.am (line 100) <https://reviews.apache.org/r/52645/#comment220913> http://savannah.gnu.org/patch/?8186 http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=commitdiff;h=39683064bbccb4008f239262cb681a970bf53603 - Aaron Wood On Oct. 10, 2016, 7:50 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 10, 2016, 7:50 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 034bb91 > src/Makefile.am fd01e1d > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review152136 --- configure.ac (line 455) <https://reviews.apache.org/r/52645/#comment220912> Support `-fstack-protector-strong` when possible, otherwise use `-fstack-protector` and don't make 4.9 a requirement. src/Makefile.am <https://reviews.apache.org/r/52645/#comment220911> Removing this and putting it in AM_CXXFLAGS causes builds to fail, at least on OS X due to these warnings being treated as hard errors: ``` clang: warning: argument unused during compilation: '-pthread' clang: warning: argument unused during compilation: '-pie' ``` - Aaron Wood On Oct. 10, 2016, 7:50 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 10, 2016, 7:50 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 034bb91 > src/Makefile.am fd01e1d > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- (Updated Oct. 10, 2016, 7:53 p.m.) Review request for mesos and Michael Park. Changes --- Fix spelling and clarify comment about `-Wall`. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/stout/Makefile.am fda069d Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- (Updated Oct. 10, 2016, 7:52 p.m.) Review request for mesos and Michael Park. Changes --- Fix spelling and clarify comment about `-Wall`. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs (updated) - 3rdparty/libprocess/Makefile.am 020b0e1 Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 10, 2016, 7:50 p.m.) Review request for mesos and Michael Park. Changes --- Fix spelling and clarify comment about `-Wall`. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 10, 2016, 6:16 p.m.) Review request for mesos and Michael Park. Changes --- Extra docs about some of the flags we moved out of MESOS_CPPFLAGS. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 10, 2016, 6:13 p.m.) Review request for mesos and Michael Park. Changes --- Move some of the warnings that were set in MESOS_CPPFLAGS to AM_CXXFLAGS. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
> On Oct. 10, 2016, 5:32 p.m., James Peach wrote: > > configure.ac, line 460 > > <https://reviews.apache.org/r/52645/diff/2/?file=1529726#file1529726line460> > > > > GCC 4.9 is in devtoolset-3 for RHEL 6, so I think this is fine. IMHO it > > is still worth giving the mailing list a heads-up though. Sure, will send out a note about this. > On Oct. 10, 2016, 5:32 p.m., James Peach wrote: > > src/Makefile.am, line 117 > > <https://reviews.apache.org/r/52645/diff/2/?file=1529727#file1529727line117> > > > > Since you are now putting the compiler flags in the correct variables, > > you can remove then from ``MESOS_CPPFLAGS`` (which should only contain > > preprocessor options). These were here before my changes so I had left them as is. I'll move them over to the right spot now. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review152008 ------- On Oct. 10, 2016, 3:42 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 10, 2016, 3:42 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > configure.ac 034bb91 > src/Makefile.am fd01e1d > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new errors/warnings produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Oct. 10, 2016, 3:51 p.m.) Review request for mesos and Michael Park. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new warnings/errors that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/encoder.hpp af083d1 3rdparty/libprocess/src/process.cpp 02a1925 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/socket.cpp 1e49518 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 2538f56 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/process_tests.cpp b9feec7 3rdparty/libprocess/src/tests/subprocess_tests.cpp 66ccff9 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp 94ba915 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp 59e69a5 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 1e26877 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp c2900b8 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Review Request 52696: Harden stout
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52696/ --- Review request for mesos and Michael Park. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to stout. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/stout/Makefile.am fda069d Diff: https://reviews.apache.org/r/52696/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Review Request 52695: Harden libprocess
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52695/ --- Review request for mesos and Michael Park. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to libprocess. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 020b0e1 Diff: https://reviews.apache.org/r/52695/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 10, 2016, 3:42 p.m.) Review request for mesos and Michael Park. Changes --- Addressed comments. Only target Mesos in this patch. Other RR's will contain changes for libprocess and stout. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs (updated) - configure.ac 034bb91 src/Makefile.am fd01e1d Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
On Oct. 7, 2016, 10:15 p.m., Aaron Wood wrote: > > (1) Do we need to make the `CXXFLAGS` conditional on being supported by the > > current compiler? Seems like these flags are quite specific to (certain > > versions of?) gcc/clang. > > > > (2) You should split this review into three separate reviews: a single > > review should make changes to at most one of Mesos, libprocess, and stout. > > > > (3) What _specific_ attack vectors are these changes intended to prevent? 1. I believe the only flag that we need to watch out for with compatability is the `-fstack-protector-strong`. Since Mesos currently requires GCC >= 4.8.1 I think we should be good with the rest. Since `-fstack-protector-strong` is supported in GCC >= 4.9 I propose that we require at least this version. 2. Will do that right now :) 3. Overall the changes here should help prevent buffer overflows, stack overflows, and general memory corruption attacks. Having position independent code/binaries will also better take advantage of address space layout randomization which makes it much harder to successfully perform exploits. This should ideally give us better protection from zero days as well. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/#review151886 --- On Oct. 7, 2016, 7:22 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52645/ > --- > > (Updated Oct. 7, 2016, 7:22 p.m.) > > > Review request for mesos and Michael Park. > > > Bugs: MESOS-6229 > https://issues.apache.org/jira/browse/MESOS-6229 > > > Repository: mesos > > > Description > --- > > Use a default set of flags to provide additional security and hardening to > Mesos. Additionally, check and catch more warnings/errors. > > > Diffs > - > > 3rdparty/libprocess/Makefile.am 020b0e1 > 3rdparty/stout/Makefile.am fda069d > src/Makefile.am bfdb66a > > Diff: https://reviews.apache.org/r/52645/diff/ > > > Testing > --- > > Compared the benchmarks with and without the flags being used. Also did a > comparsion with the flags being used with and without optimizations and > without the flags being used with and without optimizations. Overall the > performance hit was very small with a 3-8% overhead (optimizations brings > this down slightly). Most benchmarks were about 5% (or less) slower. > > > Thanks, > > Aaron Wood > >
Re: Review Request 52647: Fix new errors/warnings produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- (Updated Oct. 7, 2016, 7:22 p.m.) Review request for mesos and Michael Park. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new warnings/errors that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/encoder.hpp af083d1 3rdparty/libprocess/src/process.cpp 02a1925 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/socket.cpp 1e49518 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 2538f56 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/process_tests.cpp b9feec7 3rdparty/libprocess/src/tests/subprocess_tests.cpp 66ccff9 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp 94ba915 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp 59e69a5 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 1e26877 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp c2900b8 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- (Updated Oct. 7, 2016, 7:22 p.m.) Review request for mesos and Michael Park. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 020b0e1 3rdparty/stout/Makefile.am fda069d src/Makefile.am bfdb66a Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Review Request 52645: Harden Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52645/ --- Review request for mesos and Michael Park. Bugs: MESOS-6229 https://issues.apache.org/jira/browse/MESOS-6229 Repository: mesos Description --- Use a default set of flags to provide additional security and hardening to Mesos. Additionally, check and catch more warnings/errors. Diffs - 3rdparty/libprocess/Makefile.am 020b0e1 3rdparty/stout/Makefile.am fda069d src/Makefile.am bfdb66a Diff: https://reviews.apache.org/r/52645/diff/ Testing --- Compared the benchmarks with and without the flags being used. Also did a comparsion with the flags being used with and without optimizations and without the flags being used with and without optimizations. Overall the performance hit was very small with a 3-8% overhead (optimizations brings this down slightly). Most benchmarks were about 5% (or less) slower. Thanks, Aaron Wood
Review Request 52647: Fix new errors/warnings produced by hardened flags
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52647/ --- Review request for mesos and Michael Park. Bugs: MESOS-6239 https://issues.apache.org/jira/browse/MESOS-6239 Repository: mesos Description --- The hardening flags produced many new warnings/errors that need to be fixed for Mesos to compile/run. Diffs - 3rdparty/libprocess/include/process/profiler.hpp f6fccfb 3rdparty/libprocess/src/encoder.hpp af083d1 3rdparty/libprocess/src/process.cpp 02a1925 3rdparty/libprocess/src/profiler.cpp 0c4949e 3rdparty/libprocess/src/socket.cpp 1e49518 3rdparty/libprocess/src/tests/benchmarks.cpp 945007c 3rdparty/libprocess/src/tests/decoder_tests.cpp 4535614 3rdparty/libprocess/src/tests/encoder_tests.cpp 9e57375 3rdparty/libprocess/src/tests/http_tests.cpp 2538f56 3rdparty/libprocess/src/tests/io_tests.cpp b85c79f 3rdparty/libprocess/src/tests/process_tests.cpp b9feec7 3rdparty/libprocess/src/tests/subprocess_tests.cpp 66ccff9 3rdparty/stout/tests/cache_tests.cpp 0950c85 3rdparty/stout/tests/flags_tests.cpp 94ba915 3rdparty/stout/tests/hashmap_tests.cpp 2626d67 3rdparty/stout/tests/hashset_tests.cpp 66e59db 3rdparty/stout/tests/ip_tests.cpp 59e69a5 3rdparty/stout/tests/json_tests.cpp 2bc4c88 3rdparty/stout/tests/linkedhashmap_tests.cpp 7a80769 3rdparty/stout/tests/multimap_tests.cpp 488991b 3rdparty/stout/tests/os/process_tests.cpp 1e26877 3rdparty/stout/tests/os/sendfile_tests.cpp e221689 3rdparty/stout/tests/os/systems_tests.cpp 110ba5b 3rdparty/stout/tests/os_tests.cpp c2900b8 3rdparty/stout/tests/strings_tests.cpp 7dd3301 Diff: https://reviews.apache.org/r/52647/diff/ Testing --- Made sure compilation, tests, and benchmarks worked with both gcc and clang. Ran `make && make check && make bench`. Thanks, Aaron Wood
Re: Review Request 51068: Prevent memory leaks
> On Aug. 16, 2016, 11:35 p.m., Benjamin Mahler wrote: > > Hey Aaron, I was not able to identify the leaks you were addressing. Also > > there seems to be some object lifetime issues introduced with this patch. It looks like I was wrong about this, I had thought the copy constructor of tuple was getting called when it wasn't the copy constructor at all. I did some more testing with this and found that the original version behaves properly. I think we can close this out. - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51068/#review145930 --- On Aug. 17, 2016, 6:21 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51068/ > --- > > (Updated Aug. 17, 2016, 6:21 p.m.) > > > Review request for mesos. > > > Repository: mesos > > > Description > --- > > This should prevent any of the promises that are created in the various > ZookeeperProcess class methods from leaking memory. > > > Diffs > - > > docs/contributors.yaml 3f06000 > src/zookeeper/zookeeper.cpp e105377 > > Diff: https://reviews.apache.org/r/51068/diff/ > > > Testing > --- > > make && make check > > > Thanks, > > Aaron Wood > >
Re: Review Request 51068: Prevent memory leaks
> On Aug. 16, 2016, 11:35 p.m., Benjamin Mahler wrote: > > src/zookeeper/zookeeper.cpp, lines 196-220 > > <https://reviews.apache.org/r/51068/diff/1/?file=1472488#file1472488line196> > > > > Promise is now on the stack here, but the asynchronous callbacks > > (voidCompletion, stringCompletion, statCompletion, dataCompletion) need to > > access the promise to satisfy the future. There doesn't appear to have been > > a leak here in that the callbacks (voidCompletion, stringCompletion, > > statCompletion, dataCompletion) delete the promise after satisfying it. Can > > you add more detail as to why you're making this change? > > Aaron Wood wrote: > Hi Ben, thanks for reviewing this patch! Can I ask you the same thing > that I asked in my most recent comment here? > https://github.com/apache/mesos/pull/157 > > I see what you're saying about the callbacks taking care of deleting the > promise when necessary, I missed that the first time around. The callbacks > should be taking the pointer to the promise from the args object that's > allocated on the heap here, right? If that's the case, I'm thinking that > what's happening in the original version is this: > > 1. The promise object is allocated on the heap > 2. The pointer to this object is passed into the copy constructor of the > tuple so that a copy is taken internally in args > 3. The future is returned without deleting the memory for the promise > 4. Later on when one of the callbacks is called and the promise gets > deleted, the promise that actually gets deleted is the copy that was taken in > the tuple which was passed to one of the zookeeper C functions > 5. The original promise that was copied into the tuple via the copy > constructor still lives Sorry, meant to link directly to the post https://github.com/apache/mesos/pull/157#issuecomment-240482442 - Aaron --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51068/#review145930 --- On Aug. 17, 2016, 6:21 p.m., Aaron Wood wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51068/ > --- > > (Updated Aug. 17, 2016, 6:21 p.m.) > > > Review request for mesos. > > > Repository: mesos > > > Description > --- > > This should prevent any of the promises that are created in the various > ZookeeperProcess class methods from leaking memory. > > > Diffs > - > > docs/contributors.yaml 3f06000 > src/zookeeper/zookeeper.cpp e105377 > > Diff: https://reviews.apache.org/r/51068/diff/ > > > Testing > --- > > make && make check > > > Thanks, > > Aaron Wood > >
