Well when I built the debug one and replaced it my cluster it seemed to
have more dependencies which it was not able to find;
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-debug$
ldd impalad
./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
`GLIBCXX_3.4.20' not found (required by ./impalad)
linux-vdso.so.1 => (0x00007ffd87dc6000)
libjsig.so => not found
libTestUtil.so => not found
libUtil.so => not found
libRuntime.so => not found
libExec.so => not found
libCodeGen.so => not found
libExprs.so => not found
libRpc.so => not found
libService.so => not found
libStatestore.so => not found
libCatalog.so => not found
libResourceBroker.so => not found
libImpalaThrift.so => not found
libGlobalFlags.so => not found
libCommon.so => not found
libUdf.so => not found
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
(0x00007fd4661be000)
libThriftSaslTransport.so => not found
libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007fd465f5f000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007fd465b83000)
libgutil.so => not found
libhdfs.so.0.0.0 => not found
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd46596a000)
libjvm.so => not found
libkudu_client.so.0 => not found
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fd465762000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd46555e000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007fd46525a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd464f54000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007fd464d3e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007fd464b20000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd46475b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd4663d9000)
I went ahead and build the retail one and that was OKAY.
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail$
ldd impalad
./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.8'
not found (required by ./impalad)
./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
`GLIBCXX_3.4.20' not found (required by ./impalad)
linux-vdso.so.1 => (0x00007fffc2679000)
libjsig.so => not found
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
(0x00007f153629d000)
libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f153603e000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f1535c62000)
libjvm.so => not found
libkudu_client.so.0 => not found
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1535a5a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1535856000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f1535552000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f153524c000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f1535036000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f1534e18000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1534a53000)
/lib64/ld-linux-x86-64.so.2 (0x00007f15364b8000)
However I see the sizes being different. The one which came with cloudera
parcels was of 37M and one which I built was 384MB!! Why is that ?
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail$
du -sh impalad
348M impalad
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail$
du -sh ~/impala.retail.orig
37M /home/ubuntu/impala.retail.orig
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail$
du -sh libfesupport.so
346M libfesupport.so
ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail$
du -sh ~/libfesupport.so.retail.orig
39M /home/ubuntu/libfesupport.so.retail.orig
To conclude I am able to verify the changes which I made in the retail
build but not with the debug build. We can probably document what needs to
be replaced to test changes in a real cluster with someone's custom impala
binaries.
1. impalad
in /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail
2. libfesupport.so
in /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-retail
3. impala-frontend-0.1-SNAPSHOT.jar in
/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/. The
impala-frontend-0.1-SNAPSHOT.jar file in
/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/lib/ symlinks
to the one in /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/.
This is all which is need to test your changes.
Regards,
Pradeep Nayak
http://pradeepnayak.in
http://twitter.com/_prdp
On Tue, Sep 13, 2016 at 4:13 PM, Pradeep Nayak <pradeep1...@gmail.com>
wrote:
> Okay thanks Tim. I tried that.
>
> I am seeing the below error, once I repalced the impalad and
> libfesupport.so. Same errors for statestore and catalogd.
>
> + true
> + export
> IMPALA_BIN=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-debug
> +
> IMPALA_BIN=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-debug
> + [[ true = '' ]]
> + '[' impalad = impalad ']'
> + exec
> /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/../../bin/impalad
> --flagfile=/run/cloudera-scm-agent/process/92-impala-IMPALAD/impala-conf/impalad_flags
> /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/impala/sbin-debug/impalad:
> error while loading shared libraries: libTestUtil.so: cannot open shared
> object file: No such file or directory
>
>
> Regards,
> Pradeep Nayak
>
> http://pradeepnayak.in
> http://twitter.com/_prdp
>
> On Tue, Sep 13, 2016 at 2:16 PM, Tim Armstrong <tarmstr...@cloudera.com>
> wrote:
>
>> If the statestored and catalogd symlinks point to the new impalad binary
>> then you're good.
>>
>> On Tue, Sep 13, 2016 at 2:02 PM, Pradeep Nayak <pradeep1...@gmail.com>
>> wrote:
>>
>>> My confusion should I update the bin location or the sbin location here
>>> : /opt/cloudera/parcels/CDH/lib/impala/sbin. Here the catalogd and
>>> statestored appear to be symlinks to impalad.
>>>
>>> Regards,
>>> Pradeep Nayak
>>>
>>> http://pradeepnayak.in
>>> http://twitter.com/_prdp
>>>
>>> On Tue, Sep 13, 2016 at 1:57 PM, Pradeep Nayak <pradeep1...@gmail.com>
>>> wrote:
>>>
>>>> Okay, I can override impalad at /opt/cloudera/parcels/CDH-5
>>>> .8.0-1.cdh5.8.0.p0.42/bin/impalad with the one which I have built. How
>>>> about catalogd and statestored ?
>>>>
>>>> Regards,
>>>> Pradeep Nayak
>>>>
>>>> http://pradeepnayak.in
>>>> http://twitter.com/_prdp
>>>>
>>>> On Mon, Sep 12, 2016 at 3:41 PM, Matthew Jacobs <m...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Correct, impala-data-source-api*.jar rarely changes, but it will when
>>>>> we rename our java packages: https://gerrit.cloudera.org/#/c/3937/
>>>>>
>>>>> If you go through this process again with a build after that change
>>>>> goes in, then you'll need to update that jar.
>>>>>
>>>>> You can build it by running mvn package in Impala/ext-data-source.
>>>>>
>>>>> On Mon, Sep 12, 2016 at 3:29 PM, Tim Armstrong <
>>>>> tarmstr...@cloudera.com> wrote:
>>>>> > It looks like impala-data-source* may not be built in all case. It
>>>>> doesn't
>>>>> > really change much if at all so I doubt you will need to update it
>>>>> anyway.
>>>>> >
>>>>> > P.S. I agree with Jim, this is exactly the kind of question this
>>>>> list is
>>>>> > for.
>>>>> >
>>>>> > On Mon, Sep 12, 2016 at 2:08 PM, Pradeep Nayak <
>>>>> pradeep1...@gmail.com>
>>>>> > wrote:
>>>>> >
>>>>> >> So here is what I understand:
>>>>> >>
>>>>> >> I need to pick the files built here: /home/ubuntu/Impala/be/
>>>>> >> build/debug/service
>>>>> >>
>>>>> >> ubuntu@ip-172-30-2-163:~/Impala/be/build/debug/service$ ls -l
>>>>> >> total 330928
>>>>> >> -rwxrwxr-x 1 ubuntu ubuntu 59342696 Sep 12 19:25 hs2-util-test
>>>>> >> *-rwxrwxr-x 1 ubuntu ubuntu 58803080 Sep 12 19:24 impalad*
>>>>> >> *-rwxrwxr-x 1 ubuntu ubuntu 63169736 Sep 12 19:25 libfesupport.so*
>>>>> >> *-rwxrwxr-x 1 ubuntu ubuntu 37056896 Sep 6 21:41 libService.so*
>>>>> >> -rwxrwxr-x 1 ubuntu ubuntu 59526680 Sep 12 19:25 query-options-test
>>>>> >> -rwxrwxr-x 1 ubuntu ubuntu 60960616 Sep 12 19:25 session-expiry-test
>>>>> >> ubuntu@ip-172-30-2-163:~/Impala/be/build/debug/service$
>>>>> >>
>>>>> >> and replace it with one on my test cluster for backend part.
>>>>> >>
>>>>> >>
>>>>> >> For the frontend, i need to pick the below file:
>>>>> >>
>>>>> >> */home/ubuntu/Impala/fe/target/impala-frontend-0.1-SNAPSHOT.jar *
>>>>> >>
>>>>> >>
>>>>> >> I couldn't find the impala-data-source-api*.jar file
>>>>> >>
>>>>> >>
>>>>> >> Regards,
>>>>> >> Pradeep Nayak
>>>>> >>
>>>>> >> http://pradeepnayak.in
>>>>> >> http://twitter.com/_prdp
>>>>> >>
>>>>> >> On Mon, Sep 12, 2016 at 1:26 PM, Tim Armstrong <
>>>>> tarmstr...@cloudera.com>
>>>>> >> wrote:
>>>>> >>
>>>>> >>> I don't think this is documented anywhere. You would need to
>>>>> replace the
>>>>> >>> impalad binary and libfesupport.so under sbin-retail or sbin-debug
>>>>> for the
>>>>> >>> release or debug builds respectively. You also need to replace
>>>>> >>> impala-frontend-*.jar and impala-data-source-api*.jar.
>>>>> >>>
>>>>> >>> You also need to make sure the statestored and catalogd symlinks
>>>>> point to
>>>>> >>> the impalad binary,
>>>>> >>>
>>>>> >>> On Mon, Sep 12, 2016 at 1:06 PM, Pradeep Nayak <
>>>>> pradeep1...@gmail.com>
>>>>> >>> wrote:
>>>>> >>>
>>>>> >>>> This question may sound noob, but apologies in advance.
>>>>> >>>>
>>>>> >>>> I have changes in the impala source code (both backend and
>>>>> frontend
>>>>> >>>> components). How do I go about installing this version of Impala
>>>>> on my
>>>>> >>>> cloudera test cluster ? I know the path where I can replace the
>>>>> front end
>>>>> >>>> built jar file, but not sure of the backend parts.
>>>>> >>>>
>>>>> >>>> Regards,
>>>>> >>>> Pradeep Nayak
>>>>> >>>>
>>>>> >>>> http://pradeepnayak.in
>>>>> >>>> http://twitter.com/_prdp
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>>
>>>>
>>>>
>>>
>>
>
