As a side note, Ivy resolvers don't like pom-only releases. So using Ivy or an ivy backed tool (SBT) requires you specify the full list of dependencies explicitly.
On Mon, Sep 22, 2014 at 11:36 AM, Andrew Purtell <[email protected]> wrote: > > HBase does some build magic so they can release against both hadoop1 and > hadoop2 by changing the version to include -hadoopX in the name so they can > different named artifacts for each when they release. So obviously, its > possible for us to do it too :) > > It's an ugly hack not magic. We have a script in dev-support, > generate-hadoopX-poms.sh, that will munge the versions found in a POM > and write a new set of POMs: > > # Now we do search and replace of explicit strings. The best > # way of seeing what the below does is by doing a diff between > # the original pom and the generated pom (pom.xml.hadoop1 or > # pom.xml.hadoop2). We replace the compat.module variable with > # either hbase-hadoop1-compat or hbase-hadoop2-compat, we > # replace the version string in all poms, we change modules > # to include reference to the non-standard pom name, we > # adjust relative paths so child modules can find the parent pom, > # and we enable/disable hadoop 1 and hadoop 2 profiles as > # appropriate removing a comment string too. We output the > # new pom beside the original. > > We then point Maven at the appropriate munged and version-suffixed > root POM, once for hadoop1, once for hadoop2. > > This practice will thankfully be a thing of the past after HBase 1.0. > I don't recommend anyone else adopt it. > > > On Mon, Sep 22, 2014 at 10:19 AM, Jesse Yates <[email protected]> > wrote: > > From @Flavio > > > >> I think the jars on maven central are built against hadoop2, so it would > >> be REALLY nice to publish both flavours in the next release > > > > > > At the very least, the jars in maven should be what we build by default > > (which, AFAIK, is hadoop1). HBase does some build magic so they can > release > > against both hadoop1 and hadoop2 by changing the version to include > -hadoopX > > in the name so they can different named artifacts for each when they > > release. So obviously, its possible for us to do it too :) > > > > The question to me is if we should even continue to support hadoop1 in > main? > > I don't think we should, though we could continue it in the 4.X line for > > compatibility's sake. > > > > ------------------- > > Jesse Yates > > @jesse_yates > > jyates.github.com > > > > On Mon, Sep 22, 2014 at 9:31 AM, James Taylor <[email protected]> > > wrote: > >> > >> +1 to doing the same for hbase-testing-util. Thanks for the analysis, > >> Andrew! > >> > >> James > >> > >> On Mon, Sep 22, 2014 at 9:18 AM, Andrew Purtell <[email protected]> > >> wrote: > >> > On Thu, Sep 18, 2014 at 3:01 PM, James Taylor <[email protected] > > > >> > wrote: > >> >> I see. That makes sense, but it's more of an HBase request than a > >> >> Phoenix request. If HBase had a "client-only" pom, then Phoenix could > >> >> have a "client-only" pom as well. > >> >> > >> >> Thanks, > >> >> James > >> > > >> > If you specify only the 'hbase-client' Maven module as the HBase > >> > dependency then this pulls in client-only deps (hbase-common, > >> > hbase-protocol, commons-codec, commons-collections, commons-io, > >> > commons-lang, commons-logging, guava, protobuf-java, netty, zookeeper, > >> > htrace-core, jackson-mapper-asl, log4j) > >> > > >> > I think Phoenix might be pulling in all of the other HBase modules > >> > because phoenix-core specifies the 'hbase-testing-util' HBase module > >> > as a dependency, and not at test scope: > >> > > >> > <dependencies> > >> > <dependency> > >> > <groupId>org.apache.hbase</groupId> > >> > <artifactId>hbase-testing-util</artifactId> > >> > <exclusions> > >> > <exclusion> > >> > <groupId>org.jruby</groupId> > >> > <artifactId>jruby-complete</artifactId> > >> > </exclusion> > >> > </exclusions> > >> > </dependency> > >> > ... > >> > > >> > hbase-testing-util doesn't contain any code. It is a module you can > >> > use that will pull in *everything* needed to start up mini cluster > >> > tests, all of the HBase modules including hbase-server and the compat > >> > modules, with compile scope. Maven doc says about compile scope: "This > >> > is the default scope, used if none is specified. Compile dependencies > >> > are available in all classpaths of a project. Furthermore, those > >> > dependencies are propagated to dependent projects." > >> > > >> > Other test dependencies in the phoenix-core POM are included at test > >> > scope and tagged as optional, e.g. > >> > > >> > <dependency> > >> > <groupId>org.apache.hadoop</groupId> > >> > <artifactId>hadoop-test</artifactId> > >> > <optional>true</optional> > >> > <scope>test</scope> > >> > </dependency> > >> > > >> > Perhaps the same should be done for hbase-testing-util ? > >> > > >> > > >> >> On Thu, Sep 18, 2014 at 1:52 PM, Flavio Pompermaier > >> >> <[email protected]> wrote: > >> >>> Because it is not clear which are the exact dependencies to add to a > >> >>> pom.xml > >> >>> that uses the Phoenix client. > >> >>> For example, why do I need to transitively hold all depency of > >> >>> hbase-server > >> >>> in the client side or both hadoop (1 and 2) compatibility jars?! > >> >>> Wouldn't be > >> >>> more easy to create a client jar that select only those needed > classes > >> >>> of > >> >>> such jar? > >> >>> > >> >>> On Sep 18, 2014 9:12 PM, "Mujtaba Chohan" <[email protected]> > wrote: > >> >>>> > >> >>>> Falvio - Client jar is composed on multiple dependency jars > extracted > >> >>>> in > >> >>>> one and is for convenience available in binary download only. This > >> >>>> type of > >> >>>> bundled jars are not supposed to go in maven repo. as maven > >> >>>> automatically > >> >>>> resolves required dependencies. To use maven repo., use the > >> >>>> phoenix-core as > >> >>>> artifact id and maven will pull in all the required dependencies > for > >> >>>> your > >> >>>> project. Then if you want a single jar for your project, you can > >> >>>> bundle your > >> >>>> project jars in one. > >> >>>> > >> >>>> <dependency> > >> >>>> <groupId>org.apache.phoenix</groupId> > >> >>>> <artifactId>phoenix-core</artifactId> > >> >>>> <version>4.1.0</version> > >> >>>> </dependency> > >> >>>> > >> >>>> //mujtaba > >> >>>> > >> >>>> On Thu, Sep 18, 2014 at 11:54 AM, Flavio Pompermaier > >> >>>> <[email protected]> wrote: > >> >>>>> > >> >>>>> Is the client jar available on maven central? > >> >>>>> > >> >>>>> On Sep 18, 2014 8:46 PM, "anil gupta" <[email protected]> > wrote: > >> >>>>>> > >> >>>>>> Hi Flavio, > >> >>>>>> > >> >>>>>> Phoenix-core is for sever side. If this app is for client side > then > >> >>>>>> you > >> >>>>>> can use client jars. Maybe you can try to use > >> >>>>>> phoenix-client-without-hbase.jar. In this case, i think you will > at > >> >>>>>> least > >> >>>>>> need hbase-client jar in your client app to be added in your pom. > >> >>>>>> > >> >>>>>> Thanks, > >> >>>>>> Anil Gupta > >> >>>>>> > >> >>>>>> On Thu, Sep 18, 2014 at 6:13 AM, Flavio Pompermaier > >> >>>>>> <[email protected]> wrote: > >> >>>>>>> > >> >>>>>>> Is it really necessary the dependency to hbase-server..? Then > if I > >> >>>>>>> try > >> >>>>>>> to run my application from a webapp using resteasy I get this > >> >>>>>>> error: > >> >>>>>>> > >> >>>>>>> Caused by: > >> >>>>>>> > org.jboss.as.server.deployment.DeploymentUnitProcessingException: > >> >>>>>>> JBAS011232: Only one JAX-RS Application Class allowed. > >> >>>>>>> com.sun.jersey.api.core.ClasspathResourceConfig > >> >>>>>>> com.sun.jersey.server.impl.application.DeferredResourceConfig > >> >>>>>>> com.sun.jersey.api.core.ScanningResourceConfig > >> >>>>>>> org.apache.hadoop.hbase.rest.ResourceConfig > >> >>>>>>> com.sun.jersey.api.core.ClassNamesResourceConfig > >> >>>>>>> com.sun.jersey.api.core.ApplicationAdapter > >> >>>>>>> com.sun.jersey.api.core.WebAppResourceConfig > >> >>>>>>> com.sun.jersey.api.core.PackagesResourceConfig > >> >>>>>>> com.sun.jersey.api.core.DefaultResourceConfig > >> >>>>>>> com.sun.jersey.api.core.ResourceConfig > >> >>>>>>> at > >> >>>>>>> > >> >>>>>>> > org.jboss.as.jaxrs.deployment.JaxrsScanningProcessor.scan(JaxrsScanningProcessor.java:209) > >> >>>>>>> > >> >>>>>>> On Thu, Sep 18, 2014 at 11:40 AM, Flavio Pompermaier > >> >>>>>>> <[email protected]> wrote: > >> >>>>>>>> > >> >>>>>>>> I was able to obtain a minimal set of dependencies *for > hadoop1) > >> >>>>>>>> using: > >> >>>>>>>> <dependency> > >> >>>>>>>> <groupId>org.apache.phoenix</groupId> > >> >>>>>>>> <artifactId>phoenix</artifactId> > >> >>>>>>>> <version>4.1.0</version> > >> >>>>>>>> </dependency> > >> >>>>>>>> <dependency> > >> >>>>>>>> <groupId>org.apache.hbase</groupId> > >> >>>>>>>> <artifactId>hbase-client</artifactId> > >> >>>>>>>> <version>0.98.4-hadoop1</version> > >> >>>>>>>> </dependency> > >> >>>>>>>> <!-- Need hbase server just for > >> >>>>>>>> ConstantSizeRegionSplitPolicy class.. --> > >> >>>>>>>> <dependency> > >> >>>>>>>> <groupId>org.apache.hbase</groupId> > >> >>>>>>>> <artifactId>hbase-server</artifactId> > >> >>>>>>>> <version>0.98.4-hadoop1</version> > >> >>>>>>>> <exclusions> > >> >>>>>>>> <exclusion> > >> >>>>>>>> <artifactId>*</artifactId> > >> >>>>>>>> <groupId>*</groupId> > >> >>>>>>>> </exclusion> > >> >>>>>>>> </exclusions> > >> >>>>>>>> </dependency> > >> >>>>>>>> <dependency> > >> >>>>>>>> <groupId>org.apache.phoenix</groupId> > >> >>>>>>>> <artifactId>phoenix-hadoop-compat</artifactId> > >> >>>>>>>> <version>4.1.0</version> > >> >>>>>>>> <exclusions> > >> >>>>>>>> <!-- conflict with > >> >>>>>>>> hbase-client-0.98.4-hadoop1 version --> > >> >>>>>>>> <exclusion> > >> >>>>>>>> <artifactId>hbase-common</artifactId> > >> >>>>>>>> <groupId>org.apache.hbase</groupId> > >> >>>>>>>> </exclusion> > >> >>>>>>>> </exclusions> > >> >>>>>>>> </dependency> > >> >>>>>>>> > >> >>>>>>>> but I still get this exception (that however seems not causing > >> >>>>>>>> any > >> >>>>>>>> issue): > >> >>>>>>>> java.lang.RuntimeException: Could not create interface > >> >>>>>>>> org.apache.phoenix.trace.PhoenixSpanReceiver Is the hadoop > >> >>>>>>>> compatibility jar > >> >>>>>>>> on the classpath? > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:60) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.trace.TracingCompat.newTraceMetricSource(TracingCompat.java:40) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.trace.util.Tracing.addTraceMetricsSource(Tracing.java:294) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.jdbc.PhoenixConnection.<clinit>(PhoenixConnection.java:125) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1516) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1489) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1489) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:129) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) > >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:233) > >> >>>>>>>> Caused by: java.util.NoSuchElementException > >> >>>>>>>> at > >> >>>>>>>> > java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:357) > >> >>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:445) > >> >>>>>>>> at > >> >>>>>>>> > >> >>>>>>>> > org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:46) > >> >>>>>>>> ... 14 more > >> >>>>>>>> > >> >>>>>>>> How can I fix those problems? > >> >>>>>>>> > >> >>>>>>>> On Thu, Sep 18, 2014 at 10:34 AM, Flavio Pompermaier > >> >>>>>>>> <[email protected]> wrote: > >> >>>>>>>>> > >> >>>>>>>>> Hi to all, > >> >>>>>>>>> > >> >>>>>>>>> I'm trying to write a J2EE application that uses the Phoenix > >> >>>>>>>>> JDBC > >> >>>>>>>>> driver for HBase 0.98 (not on yarn for tests but it will be in > >> >>>>>>>>> production) > >> >>>>>>>>> through maven. > >> >>>>>>>>> Do I really need all the jars of the hbase-server side > contained > >> >>>>>>>>> in > >> >>>>>>>>> the phoenix core jar..? > >> >>>>>>>>> I can't see a phoenix-client jar on the central maven > repository > >> >>>>>>>>> while in the documentation there's to use that jar! > >> >>>>>>>>> > >> >>>>>>>>> Thanks in advance, > >> >>>>>>>>> Flavio > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> -- > >> >>>>>> Thanks & Regards, > >> >>>>>> Anil Gupta > >> >>>> > >> >>>> > >> >>> > >> > > >> > > >> > > >> > -- > >> > Best regards, > >> > > >> > - Andy > >> > > >> > Problems worthy of attack prove their worth by hitting back. - Piet > >> > Hein (via Tom White) > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) > -- Dan Di Spaltro
