[
https://issues.apache.org/jira/browse/HDFS-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15064776#comment-15064776
]
Bob Hansen commented on HDFS-9325:
----------------------------------
All good questions, [~James Clampffer].
I'm supposing a couple of use cases:
* Consumer wants the mvn compile to produce a library (static or dynamic) that
can be used with their product
* Consumer wants a source tarball that they can drop into a third_party
collection (much like we have with our third-party libs) for their product.
Additionally, there's the observation that libhdfs has some dependencies on the
presence and structure of the hadoop source tree:
* Three different directories for .proto files
* The libhdfs source header
* The minidfs cluster and libhdfs test structure
For the second use case above (creating a source tarball), I wanted to reduce
or remove those external dependencies. To implement that, I added to our cmake
files an abstraction for "I'm running within the hadoop tree" vs "I'm running
from a libhdfspp stand-alone source distribution." These are controlled by the
HADOOP_BUILD variable, which is set to 1 whenever being run by mvn.
The cmake changes reduce the set of Hadoop tree dependencies by copying all of
them into a single space in the target/extern directory if HADOOP_BUILD is set.
Then all of the rest of the build process can count on those files being in
extern/ regardless of HADOOP_BUILD or not. We also only compile the tests that
depend on the rest of the hadoop framework if HADOOP_BUILD is set, removing the
last of the dependencies.
{quote}
Part 1: gathering up dependencies
1) antrun now passes in a HADOOP_BUILD variable (seems like this is
unconditional?)
2) when HADOOP_BUILD is present copy all of the protobuf files and hdfs.h from
libhdfs into <project's target dir>/extern
{quote}
That is correct.
{quote}
Part 2: dumping libhdfs++ and dependencies elsewhere to be consumed by another
application, this is where I'm less certain
1) run the build as usual
2) if you go into the build output directory you can do a make install with a
DESTDIR variable set it can drop a tarball there
-Am I missing something about running cmake first? Is there something analogous
to passing DESTDIR into maven or antrun rather than having cmake generate the
makefiles first?
{quote}
My intention is when we get to making source tarballs for output, the steps
will be:
1. mvn builds a tarball with the libhdfspp/* and target/extern/* files.
2. The consumer untars libhdfspp_src.tar.gz locally
3a. The consumer runs {{cmake; make; sudo make install}} to install libhdfspp
on the target machine (I have tested this)
3b. *or* the consumer runs {{cmake; make; make install
DEST_DIR=../third_party_libs}} (I have tested this)
3c. *or* the consumer just includes libhdfspp in their cmake tree (much like we
are for the libhdfspp third_party; I haven't tested this yet - we should do it
as an example)
For now, I'm doing step 1 by hand.
bq. Also I'm a little unclear what exactly changed that makes libdl.so
required. Any thoughts here?
I think it was always there; we just hadn't tested building the tests in an
environment where it was necessary (namely, building OpenSSL to use dynamic
bindings)
> Allow the location of hadoop source tree resources to be passed to CMake
> during a build.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-9325
> URL: https://issues.apache.org/jira/browse/HDFS-9325
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: James Clampffer
> Assignee: Bob Hansen
> Attachments: HDFS-9325.HDFS-8707.001.patch,
> HDFS-9325.HDFS-8707.002.patch, HDFS-9325.HDFS-8707.003.patch,
> HDFS-9325.HDFS-8707.004.patch, HDFS-9325.HDFS-8707.005.patch
>
>
> It would be nice if CMake could take an optional parameter with the location
> of hdfs.h that typically lives at libhdfs/includes/hdfs/hdfs.h, otherwise it
> would default to this location. This would be useful for projects using
> libhdfs++ that gather headers defining library APIs in a single location.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
