Ah I see. I think if you run bin/bootstrap_toolchain.py manually it will build the stub for you.
I wouldn't be surprised if this requires some tweaks to the scripts to make it work since they weren't written for this exact case in mind (the original use case for the kudu stubs was for OSes that Kudu didn't support but Impala did, like CentOS 5). On Wed, Aug 21, 2019 at 11:12 AM Koltsov Dmitriy <[email protected]> wrote: > Hello, Tim. > > Thanks for reply. > > When I'm trying to build Impala I am of course using already built clouder > native-toolchain (cloned from github repo). This native-toolchain has no > > kudu in final build directory. So, when I'm trying to build Impala > pointing this build through IMPALA_TOOLCHAIN variable with > SKIP_TOOLCHAIN_BOOSTRAP=true then I have an error mentioned by me above > (see the fragment below). Could you, please, inform if there is a > possibility to instruct Impala building process not to use Kudu in Impala > build on Ubuntu 16.04 ppc64le ? > > > > -- ----------> Adding thirdparty library libunwind. <----------> > > > > -- Header files: > /home/build/impala/toolchain/libunwind-1.3-rc1-p3/include> > > > > -- Added shared library dependency libunwind:> > > > > /home/build/native-toolchain/build/libunwind-1.3-rc1-p3/lib64/> > > > > CMake Error at CMakeLists.txt:409 (find_package):> > > > > Could not find a package configuration file provided by "kuduClient" > with> > > > > any of the following names:> > > > >> > > > > kuduClientConfig.cmake> > > > > kuduclient-config.cmake> > > > >> > > > > Add the installation prefix of "kuduClient" to CMAKE_PREFIX_PATH or > set> > > > > "kuduClient_DIR" to a directory containing one of the above files. If> > > > > "kuduClient" provides a separate development package or SDK, be sure > it> > > > > has> > > > > been installed.> > > > >> > > > >> > > > > -- Configuring incomplete, errors occurred!> > > > > See also "/home/build/impala/CMakeFiles/CMakeOutput.log".> > > > > See also "/home/build/impala/CMakeFiles/CMakeError.log".> > > > > /home/build/impala#> > > > > /home/build/impala#> > > On 2019/08/21 16:47:48, Tim Armstrong <[email protected]> > <[email protected]> wrote: > > > Hi Dmitry,> > > > > > > The toolchain bootstrap process downloads pre-built binaries, and we > don't> > > > have any for PPC.> > > > > > > You would need to build the toolchain locally for PPC, then point the> > > > Impala build at it. These instructions might be helpful for you -> > > > > https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala> > > > > .> > > > > > > > > > > > > On Tue, Aug 20, 2019 at 3:53 PM Koltsov Dmitriy <[email protected]> > <[email protected]> wrote:> > > > > > > > I've looked at the contents of impala-config.sh. As I understand - I> > > > > should manually set KUDU_IS_SUPPORTED=false in order to make general > impala> > > > > build process to download kudu and build stub. But I use > native_toolchain> > > > > and so var SKIP_TOOLCHAIN_BOOTSTRAP=true. So it seems that there is > mutual> > > > > exclusion: if I want to build impala with github cloudera > native_toolchain> > > > > then code branch for kudu stub building is disabled> > > > > (SKIP_TOOLCHAIN_BOOTSTRAP=true), and if I set> > > > > SKIP_TOOLCHAIN_BOOTSTRAP=false to enable general build process to > download> > > > > and build kudu stub then there will be no other components available > from> > > > > cloudera native_toolchain.> > > > >> > > > > For example, when I use first case with SKIP_TOOLCHAIN_BOOTSTRAP=true > and> > > > > manually KUDU_IS_SUPPORTED=false set in impala-config.sh, then I have > the> > > > > following error when trying ./buildall.sh for cloudera impala:> > > > >> > > > > ...> > > > > -- ----------> Adding thirdparty library krb5. <----------> > > > > -- Header files: /usr/include> > > > > -- Added shared library dependency krb5:> > > > > /usr/lib/powerpc64le-linux-gnu/libkrb5.so> > > > > -- Found Kerberos: /usr/sbin/kadmin.local> > > > > -- Added shared library dependency rt:> > > > > /usr/lib/powerpc64le-linux-gnu/librt.so> > > > > -- Added shared library dependency dl:> > > > > /usr/lib/powerpc64le-linux-gnu/libdl.so> > > > > -- Found LIBUNWIND:> > > > > /home/build/native-toolchain/build/libunwind-1.3-rc1-p3/lib64/> > > > > -- ----------> Adding thirdparty library libunwind. <----------> > > > > -- Header files: > /home/build/impala/toolchain/libunwind-1.3-rc1-p3/include> > > > > -- Added shared library dependency libunwind:> > > > > /home/build/native-toolchain/build/libunwind-1.3-rc1-p3/lib64/> > > > > CMake Error at CMakeLists.txt:409 (find_package):> > > > > Could not find a package configuration file provided by "kuduClient" > with> > > > > any of the following names:> > > > >> > > > > kuduClientConfig.cmake> > > > > kuduclient-config.cmake> > > > >> > > > > Add the installation prefix of "kuduClient" to CMAKE_PREFIX_PATH or > set> > > > > "kuduClient_DIR" to a directory containing one of the above files. If> > > > > "kuduClient" provides a separate development package or SDK, be sure > it> > > > > has> > > > > been installed.> > > > >> > > > >> > > > > -- Configuring incomplete, errors occurred!> > > > > See also "/home/build/impala/CMakeFiles/CMakeOutput.log".> > > > > See also "/home/build/impala/CMakeFiles/CMakeError.log".> > > > > /home/build/impala#> > > > > /home/build/impala#> > > > >> > > > >> > > > > Could you please describe how can one exactly build cloudera impala > with> > > > > its native_toolchain and kudu stub simultaneously on Ubuntu 16.04 > ppc64le ?> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > Regards,> > > > > Koltsov Dmitriy> > > > > System Engineer, YADRO> > > > >> > > > >> > > > >> > > > > On 08/16/19 18:15, Lars Volker wrote:> > > > >> > > > > For the Impala toolchain you can start with this list of commits> > > > > <https://gerrit.cloudera.org/#/q/ppc64le> > <https://gerrit.cloudera.org/#/q/ppc64le>, in particular this review> > > > > <https://gerrit.cloudera.org/#/c/9077/> > <https://gerrit.cloudera.org/#/c/9077/>.> > > > >> > > > > I don't know about kudu, you might want to ask on their mailing list > or> > > > > their slack channel. If you look in impala-config.sh for> > > > > the KUDU_IS_SUPPORTED flag, you should find a way to build Impala > without> > > > > Kudu.> > > > >> > > > > Cheers, Lars> > > > >> > > > >> > > > >> > > > > On Fri, Aug 16, 2019 at 5:32 AM Koltsov Dmitriy <[email protected]> > <[email protected]>> > > > > wrote:> > > > >> > > > >> Hello, Lars!> > > > >>> > > > >> Thanks for operative answer.> > > > >>> > > > >> And could please inform about actual commit number of the> > > > >> native-toolchain on github which can be build successfully at that > moment ?> > > > >>> > > > >> Btw: I ask about it because build process for current commit of> > > > >> native-toolchain finishes with error. And error is because of kudu > build> > > > >> fail (other components of toolchain have been built fine). For now > build.sh> > > > >> file in this commit of kudu src have explicit switch to disable > ppc64le.> > > > >> And if I manually enable ppc64le case then code will try to build > kudu from> > > > >> github; but github version kudu tries to build own toolchain while> > > > >> thirdparty building and it seems that for know current commit of > github> > > > >> kudu has an error on the way to build kudu on Ubuntu 16.04 ppc64le.> > > > >>> > > > >> So another question is - Can Kudu be build on Ubuntu 16.04 ppc64le ? > If> > > > >> yes - could you please inform me about appropriate commit of github > kudu or> > > > >> maybe about another way to build it ?> > > > >>> > > > >>> > > > >>> > > > >> Regards,> > > > >> Koltsov Dmitriy> > > > >> System Engineer, YADRO> > > > >>> > > > >>> > > > >> On 16.08.2019 00:39, Lars Volker wrote:> > > > >>> > > > >> In short the answer is "likely not". We don't have any tests for > ppc64le.> > > > >> About two years ago we had some commits to enable the toolchain on > power,> > > > >> but I can't recall or find any changes to Impala itself related to > that> > > > >> effort.> > > > >>> > > > >> Cheers, Lars> > > > >>> > > > >> On Thu, Aug 15, 2019 at 11:53 AM Koltsov Dmitriy <[email protected]> > <[email protected]>> > > > >> wrote:> > > > >>> > > > >>> Hello!> > > > >>>> > > > >>> Could you please inform if Impala can be build and run on Ubuntu > 16.04> > > > >>> ppc64le ?> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>> --> > > > >>> Regards,> > > > >>> Koltsov Dmitriy> > > > >>> System Engineer, YADRO.> > > > >>>> > > > >>>> > > > >>> > > > >> > > > >
