Both are 3.0.0-9.1ubuntu1: eugene@eugene-Virtual-Machine:~/autofdo1/build$ apt list protobuf-compiler Listing... Done protobuf-compiler/bionic,now 3.0.0-9.1ubuntu1 amd64 [installed] eugene@eugene-Virtual-Machine:~/autofdo1/build$ apt list libprotobuf-dev Listing... Done libprotobuf-dev/bionic,now 3.0.0-9.1ubuntu1 amd64 [installed]
-----Original Message----- From: Wei Mi <w...@google.com> Sent: Tuesday, May 25, 2021 9:17 AM To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; Xinliang David Li <davi...@google.com>; Jan Hubicka <hubi...@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC It looks like some version problem about protobuf-compiler and libprotobuf-dev. Could you check what is the installed version on your end for those two packages and see if they are consistent? On my platform, they are both 3.12.4. On Tue, May 25, 2021 at 12:01 AM Eugene Rozenfeld <eugene.rozenf...@microsoft.com> wrote: > > That eliminates the previous error but there is a new one: > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [3/199] Building > CXX object > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quippe > r/perf_reader.cc.o > FAILED: > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > /usr/bin/c++ -I../third_party/perf_data_converter/src > -I../third_party/perf_data_converter/src/quipper -I../ > -I../third_party/glog/src -I../third_party/abseil -I../util -I. > -Ithird_party/glog -std=gnu++1z -MD -MT > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > -MF > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o.d > -o > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > -c ../third_party/perf_data_converter/src/quipper/perf_reader.cc > ../third_party/perf_data_converter/src/quipper/perf_reader.cc: In member > function 'bool > quipper::PerfReader::ReadCPUTopologyMetadata(quipper::DataReader*, size_t)': > ../third_party/perf_data_converter/src/quipper/perf_reader.cc:1518:46: error: > no match for 'operator[]' (operand types are 'const > google::protobuf::RepeatedField<unsigned int>' and 'int') > nrcpus = proto_uint32_metadata.data()[0]; > > -----Original Message----- > From: Wei Mi <w...@google.com> > Sent: Monday, May 24, 2021 8:54 PM > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; Xinliang > David Li <davi...@google.com>; Jan Hubicka <hubi...@ucw.cz>; > gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > It isn't exposed on my platform either. Looks like a bug in > perf_data_converter (i.e., quipper). Could you try adding #include > <unordered_map> in > third_party/perf_data_converter/src/quipper/huge_page_deducer.cc and see if > it fixes the problem? If it works, I will need to file a bug against > perf_data_converter. > > Thanks, > Wei. > > On Mon, May 24, 2021 at 8:33 PM Eugene Rozenfeld > <eugene.rozenf...@microsoft.com> wrote: > > > > That fixed the error I saw before but the build still fails. The > > errors start with > > > > > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja > > > > [2/217] Building CXX object > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quip > > pe > > r/huge_page_deducer.cc.o > > > > FAILED: > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quip > > pe > > r/huge_page_deducer.cc.o > > > > /usr/bin/c++ -I../third_party/perf_data_converter/src > > -I../third_party/perf_data_converter/src/quipper -I../ > > -I../third_party/glog/src -I../third_party/abseil -I../util -I. > > -Ithird_party/glog -std=gnu++1z -MD -MT > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o > > -MF > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o.d > > -o > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o > > -c ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc > > > > ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc: > > 14 > > 4:26: error: 'unordered_map' in namespace 'std' does not name a > > template type > > > > using container = std::unordered_map<key_t, value_t>; > > > > ^~~~~~~~~~~~~ > > > > > > > > > > > > > > > > From: Wei Mi <w...@google.com> > > Sent: Monday, May 24, 2021 8:12 PM > > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; > > Xinliang David Li <davi...@google.com>; Jan Hubicka > > <hubi...@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > > > > > Sorry, I added dependency for create_gcov but missed it for dump_gcov. > > Fixed it at > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F6ca36cdc30986f13583a3aef3e27746ca4fc5bf6&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315397315%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9G9uGhG%2BmK35bdV4QL9BOTVg6bFFQrVP46YdLOi3Ak4%3D&reserved=0. > > > > > > > > Thanks, > > > > Wei. > > > > > > > > On Mon, May 24, 2021 at 6:39 PM Eugene Rozenfeld > > <eugene.rozenf...@microsoft.com> wrote: > > > > Thank you Wei. Looks like something is still missing. This time > > perf_data.pb.h is not found. I'm getting the error below (on Ubuntu 18.04 > > with cmake 3.12.1): > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/241] > > Building CXX object CMakeFiles/dump_gcov_lib.dir/profile.cc.o > > FAILED: CMakeFiles/dump_gcov_lib.dir/profile.cc.o > > /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil > > -I../third_party/perf_data_converter/src > > -I../third_party/perf_data_converter/src/quipper -I../util -I. > > -Ithird_party/glog -std=gnu++1z -MD -MT > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o -MF > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o.d -o > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o -c ../profile.cc > > In file included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: > > warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define > > DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > > > In file included from ../profile.h:14:0, > > from ../profile.cc:5: > > ../base/macros.h:114:0: note: this is the location of the previous > > definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > > > In file included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: > > warning: "arraysize" redefined #define arraysize(x) (sizeof(x) / > > sizeof(*x)) > > > > In file included from ../profile.h:14:0, > > from ../profile.cc:5: > > ../base/macros.h:162:0: note: this is the location of the previous > > definition #define arraysize(array) > > (sizeof(ArraySizeHelper(array))) > > > > In file included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/compat/proto.h:14:10: > > fatal error: perf_data.pb.h: No such file or directory #include > > "perf_data.pb.h" > > ^~~~~~~~~~~~~~~~ > > compilation terminated. > > [6/241] Building CXX object > > CMakeFiles/dump_gcov_lib.dir/symbol_map.cc.o > > ninja: build stopped: subcommand failed. > > > > Thanks, > > > > Eugene > > > > From: Wei Mi <w...@google.com> > > Sent: Saturday, May 22, 2021 9:37 AM > > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; > > Xinliang David Li <davi...@google.com>; Jan Hubicka > > <hubi...@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > It is a proto library build dependency issue which didn't expose on my > > platform. I fix it at > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F98269aee9674cc885cc5eb1bd917eb2d12731710&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SL5mLhelrkeB26Mg7tNaml0qjZ2OklYPwQR9btVhsqk%3D&reserved=0. > > Please try again. > > > > Thanks, > > Wei. > > > > On Fri, May 21, 2021 at 6:28 PM Eugene Rozenfeld > > <mailto:eugene.rozenf...@microsoft.com> wrote: > > I tried following the instructions in "2.2 Build autofdo tool for gcc" in > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PYVm%2BHsRk%2Bfw7%2BaNgYNWn8zMnBsqcqqVSpyXQktd7sA%3D&reserved=0 > > got build failures: > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/228] > > Building CXX object CMakeFiles/create_gcov_lib.dir/profile.cc.o > > FAILED: CMakeFiles/create_gcov_lib.dir/profile.cc.o > > /usr/bin/c++ -I../ -I../third_party/glog/src > > -I../third_party/abseil -I../third_party/perf_data_converter/src > > -I../third_party/perf_data_converter/src/quipper -I../util -I. > > -Ithird_party/glog -std=gnu++1z -MD -MT > > CMakeFiles/create_gcov_lib.dir/profile.cc.o -MF > > CMakeFiles/create_gcov_lib.dir/profile.cc.o.d -o > > CMakeFiles/create_gcov_lib.dir/profile.cc.o -c ../profile.cc In file > > included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: > > warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define > > DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file included from > > ../profile.h:14:0, > > from ../profile.cc:5: > > ../base/macros.h:114:0: note: this is the location of the previous > > definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file > > included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: > > warning: "arraysize" redefined #define arraysize(x) (sizeof(x) / > > sizeof(*x)) In file included from ../profile.h:14:0, > > from ../profile.cc:5: > > ../base/macros.h:162:0: note: this is the location of the previous > > definition #define arraysize(array) (sizeof(ArraySizeHelper(array))) > > In file included from > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0, > > from ../sample_reader.h:18, > > from ../profile.h:15, > > from ../profile.cc:5: > > ../third_party/perf_data_converter/src/quipper/compat/proto.h:16:10: > > fatal error: perf_stat.pb.h: No such file or directory #include > > "perf_stat.pb.h" > > ^~~~~~~~~~~~~~~~ > > compilation terminated. > > > > What is supposed to generate perf_stat.pb.h? > > > > Thanks, > > > > Eugene > > > > From: Wei Mi <mailto:w...@google.com> > > Sent: Monday, May 10, 2021 4:47 PM > > To: Andi Kleen <mailto:a...@linux.intel.com> > > Cc: Hongtao Yu <mailto:h...@fb.com>; Xinliang David Li > > <mailto:davi...@google.com>; Jan Hubicka <mailto:hubi...@ucw.cz>; > > mailto:gcc@gcc.gnu.org; Eugene Rozenfeld > > <mailto:eugene.rozenf...@microsoft.com>; Wenlei He > > <mailto:wen...@fb.com> > > Subject: [EXTERNAL] Re: State of AutoFDO in GCC > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwQwWdjE%2FL%2F2r0P%2Bdn7r5ulE8kRmPPw8A2XOQuZnUhM%3D&reserved=0 > > has been updated. Now create_gcov/dump_gcov are added back and can be > > built separately. > > > > Please look at "2.2 Build autofdo tool for gcc" in > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi > > th > > ub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfe > > ld > > %40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141 > > af > > 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3 > > d8 > > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7 > > C1 > > 000&sdata=EwHtN1OqRmBwL6qMgTc1lAU6cz5%2BJzQZxpd5nos11FM%3D&r > > es > > erved=0 On Wed, Apr 28, 2021 at 10:40 PM Andi Kleen > > <mailto:a...@linux.intel.com> wrote: > > > > > > On Mon, Apr 26, 2021 at 06:40:56PM +0000, Hongtao Yu wrote: > > > > Andi, thanks for pointing out the perf script issues. Can you please > > > > elaborate a bit on the exact issue you have seen? We've been using > > > > specific output of perf script such as mmap, LBR and callstack events > > > > filtered by process id. It works fine so far but may certainly hit > > > > issues > > > > in the future with extended uses. > > > > > > Okay I took a look at the latest autofdo now. It seems to be > > > basically a LLVM project now that depends on LLVM to even build > > > with all kinds of dependency hell on some old LLVM version and other > > > packages. > > > > > > I guess gcc will really need a replacement that doesn't pull in > > > all of LLVM if it wants to continue supporting autofdo. > > > > > > I'm myself unable to build now. > > > > > > I'm using the old version I had a git fork of and that was before > > > all of this. I added a patch to make it work with the latest perf > > > by ignoring increased perf_attr and unknown perf events. > > > > > > Honza please use > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2F > > > gi > > > thub.com%2Fandikleen%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld > > > %4 > > > 0microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141 > > > af > > > 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZs > > > b3 > > > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > > > 3D > > > %7C1000&sdata=Dilu5PUjpAbA6e6d6GBd5idoKLAMMr7H5eCve%2FGdC8g%3D > > > &a > > > mp;reserved=0 -b perf-future > > > > > > for testing. > > > > > > -Andi > > >
