I checked the source of protobuf 3.0.0 and it didn't contain the operator[] in RepeatedField. Need to install a newer version of protobuf.
Thanks, Wei. On Tue, May 25, 2021 at 1:49 PM Eugene Rozenfeld <eugene.rozenf...@microsoft.com> wrote: > > Both are 3.0.0-9.1ubuntu1: > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ apt list protobuf-compiler > Listing... Done > protobuf-compiler/bionic,now 3.0.0-9.1ubuntu1 amd64 [installed] > eugene@eugene-Virtual-Machine:~/autofdo1/build$ apt list libprotobuf-dev > Listing... Done > libprotobuf-dev/bionic,now 3.0.0-9.1ubuntu1 amd64 [installed] > > -----Original Message----- > From: Wei Mi <w...@google.com> > Sent: Tuesday, May 25, 2021 9:17 AM > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; Xinliang > David Li <davi...@google.com>; Jan Hubicka <hubi...@ucw.cz>; gcc@gcc.gnu.org; > Wenlei He <wen...@fb.com> > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > It looks like some version problem about protobuf-compiler and > libprotobuf-dev. Could you check what is the installed version on your end > for those two packages and see if they are consistent? > > On my platform, they are both 3.12.4. > > On Tue, May 25, 2021 at 12:01 AM Eugene Rozenfeld > <eugene.rozenf...@microsoft.com> wrote: > > > > That eliminates the previous error but there is a new one: > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [3/199] Building > > CXX object > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quippe > > r/perf_reader.cc.o > > FAILED: > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > > /usr/bin/c++ -I../third_party/perf_data_converter/src > > -I../third_party/perf_data_converter/src/quipper -I../ > > -I../third_party/glog/src -I../third_party/abseil -I../util -I. > > -Ithird_party/glog -std=gnu++1z -MD -MT > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > > -MF > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o.d > > -o > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o > > -c ../third_party/perf_data_converter/src/quipper/perf_reader.cc > > ../third_party/perf_data_converter/src/quipper/perf_reader.cc: In member > > function 'bool > > quipper::PerfReader::ReadCPUTopologyMetadata(quipper::DataReader*, size_t)': > > ../third_party/perf_data_converter/src/quipper/perf_reader.cc:1518:46: > > error: no match for 'operator[]' (operand types are 'const > > google::protobuf::RepeatedField<unsigned int>' and 'int') > > nrcpus = proto_uint32_metadata.data()[0]; > > > > -----Original Message----- > > From: Wei Mi <w...@google.com> > > Sent: Monday, May 24, 2021 8:54 PM > > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; Xinliang > > David Li <davi...@google.com>; Jan Hubicka <hubi...@ucw.cz>; > > gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > It isn't exposed on my platform either. Looks like a bug in > > perf_data_converter (i.e., quipper). Could you try adding #include > > <unordered_map> in > > third_party/perf_data_converter/src/quipper/huge_page_deducer.cc and see if > > it fixes the problem? If it works, I will need to file a bug against > > perf_data_converter. > > > > Thanks, > > Wei. > > > > On Mon, May 24, 2021 at 8:33 PM Eugene Rozenfeld > > <eugene.rozenf...@microsoft.com> wrote: > > > > > > That fixed the error I saw before but the build still fails. The > > > errors start with > > > > > > > > > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja > > > > > > [2/217] Building CXX object > > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quip > > > pe > > > r/huge_page_deducer.cc.o > > > > > > FAILED: > > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quip > > > pe > > > r/huge_page_deducer.cc.o > > > > > > /usr/bin/c++ -I../third_party/perf_data_converter/src > > > -I../third_party/perf_data_converter/src/quipper -I../ > > > -I../third_party/glog/src -I../third_party/abseil -I../util -I. > > > -Ithird_party/glog -std=gnu++1z -MD -MT > > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o > > > -MF > > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o.d > > > -o > > > CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o > > > -c ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc > > > > > > ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc: > > > 14 > > > 4:26: error: 'unordered_map' in namespace 'std' does not name a > > > template type > > > > > > using container = std::unordered_map<key_t, value_t>; > > > > > > ^~~~~~~~~~~~~ > > > > > > > > > > > > > > > > > > > > > > > > From: Wei Mi <w...@google.com> > > > Sent: Monday, May 24, 2021 8:12 PM > > > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > > > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; > > > Xinliang David Li <davi...@google.com>; Jan Hubicka > > > <hubi...@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > > > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > > > > > > > > > Sorry, I added dependency for create_gcov but missed it for dump_gcov. > > > Fixed it at > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F6ca36cdc30986f13583a3aef3e27746ca4fc5bf6&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315397315%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9G9uGhG%2BmK35bdV4QL9BOTVg6bFFQrVP46YdLOi3Ak4%3D&reserved=0. > > > > > > > > > > > > Thanks, > > > > > > Wei. > > > > > > > > > > > > On Mon, May 24, 2021 at 6:39 PM Eugene Rozenfeld > > > <eugene.rozenf...@microsoft.com> wrote: > > > > > > Thank you Wei. Looks like something is still missing. This time > > > perf_data.pb.h is not found. I'm getting the error below (on Ubuntu 18.04 > > > with cmake 3.12.1): > > > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/241] > > > Building CXX object CMakeFiles/dump_gcov_lib.dir/profile.cc.o > > > FAILED: CMakeFiles/dump_gcov_lib.dir/profile.cc.o > > > /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil > > > -I../third_party/perf_data_converter/src > > > -I../third_party/perf_data_converter/src/quipper -I../util -I. > > > -Ithird_party/glog -std=gnu++1z -MD -MT > > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o -MF > > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o.d -o > > > CMakeFiles/dump_gcov_lib.dir/profile.cc.o -c ../profile.cc > > > In file included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: > > > warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define > > > DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > > > > > In file included from ../profile.h:14:0, > > > from ../profile.cc:5: > > > ../base/macros.h:114:0: note: this is the location of the previous > > > definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > > > > > In file included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: > > > warning: "arraysize" redefined #define arraysize(x) (sizeof(x) / > > > sizeof(*x)) > > > > > > In file included from ../profile.h:14:0, > > > from ../profile.cc:5: > > > ../base/macros.h:162:0: note: this is the location of the previous > > > definition #define arraysize(array) > > > (sizeof(ArraySizeHelper(array))) > > > > > > In file included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/compat/proto.h:14:10: > > > fatal error: perf_data.pb.h: No such file or directory #include > > > "perf_data.pb.h" > > > ^~~~~~~~~~~~~~~~ > > > compilation terminated. > > > [6/241] Building CXX object > > > CMakeFiles/dump_gcov_lib.dir/symbol_map.cc.o > > > ninja: build stopped: subcommand failed. > > > > > > Thanks, > > > > > > Eugene > > > > > > From: Wei Mi <w...@google.com> > > > Sent: Saturday, May 22, 2021 9:37 AM > > > To: Eugene Rozenfeld <eugene.rozenf...@microsoft.com> > > > Cc: Andi Kleen <a...@linux.intel.com>; Hongtao Yu <h...@fb.com>; > > > Xinliang David Li <davi...@google.com>; Jan Hubicka > > > <hubi...@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wen...@fb.com> > > > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > > > It is a proto library build dependency issue which didn't expose on my > > > platform. I fix it at > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F98269aee9674cc885cc5eb1bd917eb2d12731710&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SL5mLhelrkeB26Mg7tNaml0qjZ2OklYPwQR9btVhsqk%3D&reserved=0. > > > Please try again. > > > > > > Thanks, > > > Wei. > > > > > > On Fri, May 21, 2021 at 6:28 PM Eugene Rozenfeld > > > <mailto:eugene.rozenf...@microsoft.com> wrote: > > > I tried following the instructions in "2.2 Build autofdo tool for gcc" in > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PYVm%2BHsRk%2Bfw7%2BaNgYNWn8zMnBsqcqqVSpyXQktd7sA%3D&reserved=0 > > > got build failures: > > > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/228] > > > Building CXX object CMakeFiles/create_gcov_lib.dir/profile.cc.o > > > FAILED: CMakeFiles/create_gcov_lib.dir/profile.cc.o > > > /usr/bin/c++ -I../ -I../third_party/glog/src > > > -I../third_party/abseil -I../third_party/perf_data_converter/src > > > -I../third_party/perf_data_converter/src/quipper -I../util -I. > > > -Ithird_party/glog -std=gnu++1z -MD -MT > > > CMakeFiles/create_gcov_lib.dir/profile.cc.o -MF > > > CMakeFiles/create_gcov_lib.dir/profile.cc.o.d -o > > > CMakeFiles/create_gcov_lib.dir/profile.cc.o -c ../profile.cc In file > > > included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: > > > warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define > > > DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file included from > > > ../profile.h:14:0, > > > from ../profile.cc:5: > > > ../base/macros.h:114:0: note: this is the location of the previous > > > definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file > > > included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: > > > warning: "arraysize" redefined #define arraysize(x) (sizeof(x) / > > > sizeof(*x)) In file included from ../profile.h:14:0, > > > from ../profile.cc:5: > > > ../base/macros.h:162:0: note: this is the location of the previous > > > definition #define arraysize(array) (sizeof(ArraySizeHelper(array))) > > > In file included from > > > ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0, > > > from ../sample_reader.h:18, > > > from ../profile.h:15, > > > from ../profile.cc:5: > > > ../third_party/perf_data_converter/src/quipper/compat/proto.h:16:10: > > > fatal error: perf_stat.pb.h: No such file or directory #include > > > "perf_stat.pb.h" > > > ^~~~~~~~~~~~~~~~ > > > compilation terminated. > > > > > > What is supposed to generate perf_stat.pb.h? > > > > > > Thanks, > > > > > > Eugene > > > > > > From: Wei Mi <mailto:w...@google.com> > > > Sent: Monday, May 10, 2021 4:47 PM > > > To: Andi Kleen <mailto:a...@linux.intel.com> > > > Cc: Hongtao Yu <mailto:h...@fb.com>; Xinliang David Li > > > <mailto:davi...@google.com>; Jan Hubicka <mailto:hubi...@ucw.cz>; > > > mailto:gcc@gcc.gnu.org; Eugene Rozenfeld > > > <mailto:eugene.rozenf...@microsoft.com>; Wenlei He > > > <mailto:wen...@fb.com> > > > Subject: [EXTERNAL] Re: State of AutoFDO in GCC > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C0cd91708868547a3f63608d91f988b90%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575562315407308%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uwQwWdjE%2FL%2F2r0P%2Bdn7r5ulE8kRmPPw8A2XOQuZnUhM%3D&reserved=0 > > > has been updated. Now create_gcov/dump_gcov are added back and can be > > > built separately. > > > > > > Please look at "2.2 Build autofdo tool for gcc" in > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi > > > th > > > ub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfe > > > ld > > > %40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141 > > > af > > > 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3 > > > d8 > > > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7 > > > C1 > > > 000&sdata=EwHtN1OqRmBwL6qMgTc1lAU6cz5%2BJzQZxpd5nos11FM%3D&r > > > es > > > erved=0 On Wed, Apr 28, 2021 at 10:40 PM Andi Kleen > > > <mailto:a...@linux.intel.com> wrote: > > > > > > > > On Mon, Apr 26, 2021 at 06:40:56PM +0000, Hongtao Yu wrote: > > > > > Andi, thanks for pointing out the perf script issues. Can you > > > > > please > > > > > elaborate a bit on the exact issue you have seen? We've been using > > > > > specific output of perf script such as mmap, LBR and callstack > > > > > events > > > > > filtered by process id. It works fine so far but may certainly hit > > > > > issues > > > > > in the future with extended uses. > > > > > > > > Okay I took a look at the latest autofdo now. It seems to be > > > > basically a LLVM project now that depends on LLVM to even build > > > > with all kinds of dependency hell on some old LLVM version and other > > > > packages. > > > > > > > > I guess gcc will really need a replacement that doesn't pull in > > > > all of LLVM if it wants to continue supporting autofdo. > > > > > > > > I'm myself unable to build now. > > > > > > > > I'm using the old version I had a git fork of and that was before > > > > all of this. I added a patch to make it work with the latest perf > > > > by ignoring increased perf_attr and unknown perf events. > > > > > > > > Honza please use > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2F > > > > gi > > > > thub.com%2Fandikleen%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld > > > > %4 > > > > 0microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141 > > > > af > > > > 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZs > > > > b3 > > > > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > > > > 3D > > > > %7C1000&sdata=Dilu5PUjpAbA6e6d6GBd5idoKLAMMr7H5eCve%2FGdC8g%3D > > > > &a > > > > mp;reserved=0 -b perf-future > > > > > > > > for testing. > > > > > > > > -Andi > > > >
