This seems to work for me. Does it work for you?
diff --git a/hbase-spark/pom.xml b/hbase-spark/pom.xml
index 594aa2a..6d191e3 100644
--- a/hbase-spark/pom.xml
+++ b/hbase-spark/pom.xml
@@ -568,6 +568,9 @@
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
<parallel>false</parallel>
+ <systemProperties>
+
<org.apache.hadoop.hbase.shaded.io.netty.packagePrefix>org.apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase.shaded.io.netty.packagePrefix>
+ </systemProperties>
</configuration>
<executions>
<execution>
St.Ack
On Tue, Oct 3, 2017 at 8:45 AM, Amit Kabra <amitkabrai...@gmail.com> wrote:
> Thanks Stack / Sean Busbey for replying.
>
> OS : Ubuntu 16.04.2 , 64 bit.
> Eclipse : Version: Neon.3 Release (4.6.3)
> HBase branch : branch-2
> Command line test to reproduce : mvn clean package
> -Dtest=TestIncrementalBackup
> Reproduce from eclipse , right click on TestIncBackupRestore and run as
> junit from test class TestIncrementalBackup.
> No I am not embedding hbase in my application. I have just checked out
> hbase , switched to branch-2 and run the unit test from command line or
> from eclipse. Failing with same error in both cases.
> Yes the trailing period is also present.
>
> Thanks,
> Amit Kabra.
>
>
>
>
>
> On Tue, Oct 3, 2017 at 8:53 PM, Stack <st...@duboce.net> wrote:
>
> > Thank you for the detail.
> >
> > Pardon the questions below asking for yet more detail. I am unable to
> > reproduce locally or on another os (though we see this issue up on our
> > build box).
> >
> > What is your OS when you see the below?
> >
> > On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra <amitkabrai...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I am using "branch-2" branch of hbase, when I run unit test I get
> > following
> > > error for netty "java.lang.UnsatisfiedLinkError: failed to load the
> > > required native library"
> > >
> > >
> > > This is running a unit test in your eclipse environment?
> >
> > You are trying to run an hbase-spark unit test when you see the above?
> >
> >
> >
> >
> > > *I already have following set in "maven-surefire-plugin" in pom.xml as
> > > per http://hbase.apache.org/book.html#thirdparty
> > > <http://hbase.apache.org/book.html#thirdparty>*
> > >
> > >
> > >
> >
> > Are you embedding hbase into your application?
> >
> >
> >
> > > <systemPropertyVariables>
> > > <!--
> > > <test.build.classes>${test.build.classes}</test.build.
> > classe
> > > s>
> > > -->
> > > <!--For shaded netty, to find the relocated .so.
> > > Trick from
> > > https://stackoverflow.com/questions/33825743/rename-
> > > files-inside-a-jar-using-some-maven-plugin
> > > <https://stackoverflow.com/questions/33825743/rename-
> > files-inside-a-jar-using-some-maven-plugin>
> > >
> > > The netty jar has a .so in it. Shading requires rename
> of
> > > the .so and then passing a system
> > > property so netty finds the renamed .so and associates
> it
> > > w/ the relocated netty files.
> > >
> > > The relocated netty is in hbase-thirdparty dependency.
> > Just
> > > set this propery globally rather
> > > than per module.
> > > -->
> > > <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix>
> > > org.apache.hadoop.hbase.shaded.</org.apache.hadoop.
> > hbase.shaded.io.netty.
> > > packagePrefix>
> > > </systemPropertyVariables>
> > >
> > >
> > >
> > > *And I see in the code as per HBASE-18271, all io.netty is already
> > replaced
> > > with org.apache.hadoop.hbase.shaded.io.netty*
> > >
> > >
> > The trailing period is also present?
> >
> >
> >
> > >
> > > If I run a test from eclipse , I see the error immediately and my test
> > > doesn't run, but when I run from command line , the test runs but I get
> > the
> > > error at the end when the mvn command finishes.
> > >
> > >
> > > Is it any eclipse test?
> >
> > Thank you. Let me try and fix this this morning.
> >
> > S
> >
> >
> >
> >
> >
> > > *Here is the complete error output.*
> > >
> > >
> > >
> > > [INFO]
> > > [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @
> > hbase-spark
> > > ---
> > > [INFO]
> > > [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark ---
> > > Discovery starting.
> > > Discovery completed in 1 second, 558 milliseconds.
> > > Run starting. Expected test count is: 79
> > > HBaseDStreamFunctionsSuite:
> > > Formatting using clusterid: testClusterID
> > > *** RUN ABORTED ***
> > > java.io.IOException: Shutting down
> > > at org.apache.hadoop.hbase.MiniHBaseCluster.init(
> > > MiniHBaseCluster.java:232)
> > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(
> > > MiniHBaseCluster.java:94)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.
> startMiniHBaseCluster(
> > > HBaseTestingUtility.java:1124)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:1078)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:949)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:943)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:872)
> > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.
> beforeAll(
> > > HBaseDStreamFunctionsSuite.scala:41)
> > > at org.scalatest.BeforeAndAfterAll$class.
> beforeAll(BeforeAndAfterAll.
> > > scala:187)
> > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.
> beforeAll(
> > > HBaseDStreamFunctionsSuite.scala:30)
> > > ...
> > > Cause: java.lang.RuntimeException: Failed construction of Master:
> class
> > > org.apache.hadoop.hbase.master.HMasterorg.apache.
> > > hadoop.hbase.shaded.io.netty.channel.epoll.
> > NativeStaticallyReferencedJniM
> > > ethods.epollin()I
> > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(
> > > JVMClusterUtil.java:145)
> > > at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(
> > > LocalHBaseCluster.java:217)
> > > at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(
> > > LocalHBaseCluster.java:152)
> > > at org.apache.hadoop.hbase.MiniHBaseCluster.init(
> > > MiniHBaseCluster.java:214)
> > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(
> > > MiniHBaseCluster.java:94)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.
> startMiniHBaseCluster(
> > > HBaseTestingUtility.java:1124)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:1078)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:949)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:943)
> > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
> > > HBaseTestingUtility.java:872)
> > > ...
> > > Cause: java.lang.UnsatisfiedLinkError: failed to load the required
> > > native
> > > library
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > Epoll.ensureAvailability(Epoll.java:78)
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38)
> > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>(
> > > NettyEventLoopGroupConfig.java:61)
> > > at org.apache.hadoop.hbase.regionserver.HRegionServer.<
> > > init>(HRegionServer.java:552)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475)
> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> > > NativeConstructorAccessorImpl.java:62)
> > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> > > DelegatingConstructorAccessorImpl.java:45)
> > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(
> > > JVMClusterUtil.java:140)
> > > ...
> > > Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase.
> > > shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM
> > > ethods.epollin()I
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > NativeStaticallyReferencedJniMethods.epollin(Native Method)
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > Native.<clinit>(Native.java:66)
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > Epoll.<clinit>(Epoll.java:33)
> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38)
> > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>(
> > > NettyEventLoopGroupConfig.java:61)
> > > at org.apache.hadoop.hbase.regionserver.HRegionServer.<
> > > init>(HRegionServer.java:552)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475)
> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> > > NativeConstructorAccessorImpl.java:62)
> > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> > > DelegatingConstructorAccessorImpl.java:45)
> > > ...
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] Reactor Summary:
> > > [INFO]
> > > [INFO] Apache HBase ....................................... SUCCESS [
> > > 1.575 s]
> > > [INFO] Apache HBase - Checkstyle .......................... SUCCESS [
> > > 0.317 s]
> > > [INFO] Apache HBase - Annotations ......................... SUCCESS [
> > > 0.537 s]
> > > [INFO] Apache HBase - Build Configuration ................. SUCCESS [
> > > 0.053 s]
> > > [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [
> > > 15.410 s]
> > > [INFO] Apache HBase - Common .............................. SUCCESS [
> > > 4.603 s]
> > > [INFO] Apache HBase - Metrics API ......................... SUCCESS [
> > > 1.213 s]
> > > [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [
> > > 0.985 s]
> > > [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [
> > > 0.863 s]
> > > [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [
> > > 1.750 s]
> > > [INFO] Apache HBase - Protocol ............................ SUCCESS [
> > > 4.880 s]
> > > [INFO] Apache HBase - Client .............................. SUCCESS [
> > > 5.233 s]
> > > [INFO] Apache HBase - Replication ......................... SUCCESS [
> > > 1.040 s]
> > > [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [
> > > 1.121 s]
> > > [INFO] Apache HBase - Procedure ........................... SUCCESS [
> > > 1.084 s]
> > > [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [
> > > 0.092 s]
> > > [INFO] Apache HBase - Server .............................. SUCCESS [
> > > 19.849 s]
> > > [INFO] Apache HBase - MapReduce ........................... SUCCESS [
> > > 4.221 s]
> > > [INFO] Apache HBase - Testing Util ........................ SUCCESS [
> > > 3.273 s]
> > > [INFO] Apache HBase - Thrift .............................. SUCCESS [
> > > 5.519 s]
> > > [INFO] Apache HBase - RSGroup ............................. SUCCESS [
> > > 3.408 s]
> > > [INFO] Apache HBase - Shell ............................... SUCCESS [
> > > 3.859 s]
> > > [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [
> > > 4.038 s]
> > > [INFO] Apache HBase - Backup .............................. SUCCESS
> > [01:13
> > > min]
> > > [INFO] Apache HBase - Integration Tests ................... SUCCESS [
> > > 4.229 s]
> > > [INFO] Apache HBase - Examples ............................ SUCCESS [
> > > 3.471 s]
> > > [INFO] Apache HBase - Rest ................................ SUCCESS [
> > > 4.448 s]
> > > [INFO] Apache HBase - External Block Cache ................ SUCCESS [
> > > 2.040 s]
> > > [INFO] Apache HBase - Spark ............................... FAILURE [
> > > 32.833 s]
> > > [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED
> > > [INFO] Apache HBase - Assembly ............................ SKIPPED
> > > [INFO] Apache HBase - Shaded .............................. SKIPPED
> > > [INFO] Apache HBase - Shaded - Client ..................... SKIPPED
> > > [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED
> > > [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED
> > > [INFO] Apache HBase - Archetypes .......................... SKIPPED
> > > [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
> > > [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype
> SKIPPED
> > > [INFO] Apache HBase - Archetype builder ................... SKIPPED
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] BUILD FAILURE
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] Total time: 03:26 min
> > > [INFO] Finished at: 2017-09-27T19:34:35+05:30
> > > [INFO] Final Memory: 345M/6055M
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [ERROR] Failed to execute goal org.scalatest:scalatest-maven-
> > > plugin:1.0:test
> > > (test) on project hbase-spark: There are test failures -> [Help 1]
> > > [ERROR]
> > > [ERROR] To see the full stack trace of the errors, re-run Maven with
> the
> > -e
> > > switch.
> > > [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> > > [ERROR]
> > > [ERROR] For more information about the errors and possible solutions,
> > > please read the following articles:
> > > [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
> > > MojoFailureException
> > > [ERROR]
> > > [ERROR] After correcting the problems, you can resume the build with
> the
> > > command
> > > [ERROR] mvn <goals> -rf :hbase-spark
> > >
> >
>
