I too got the same error when I tried running one of the test case in the latest trunk from the eclipse IDE. Thanks Stack for the update. Let me also try it out.
Regards Ram On Tue, Oct 3, 2017 at 9:43 PM, Stack <st...@duboce.net> wrote: > This seems to work for me. Does it work for you? > > > diff --git a/hbase-spark/pom.xml b/hbase-spark/pom.xml > index 594aa2a..6d191e3 100644 > --- a/hbase-spark/pom.xml > +++ b/hbase-spark/pom.xml > @@ -568,6 +568,9 @@ > <junitxml>.</junitxml> > <filereports>WDF TestSuite.txt</filereports> > <parallel>false</parallel> > + <systemProperties> > + > <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix> > org.apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase.shaded.io.netty. > packagePrefix> > + </systemProperties> > </configuration> > <executions> > <execution> > > St.Ack > > On Tue, Oct 3, 2017 at 8:45 AM, Amit Kabra <amitkabrai...@gmail.com> > wrote: > > > Thanks Stack / Sean Busbey for replying. > > > > OS : Ubuntu 16.04.2 , 64 bit. > > Eclipse : Version: Neon.3 Release (4.6.3) > > HBase branch : branch-2 > > Command line test to reproduce : mvn clean package > > -Dtest=TestIncrementalBackup > > Reproduce from eclipse , right click on TestIncBackupRestore and run as > > junit from test class TestIncrementalBackup. > > No I am not embedding hbase in my application. I have just checked out > > hbase , switched to branch-2 and run the unit test from command line or > > from eclipse. Failing with same error in both cases. > > Yes the trailing period is also present. > > > > Thanks, > > Amit Kabra. > > > > > > > > > > > > On Tue, Oct 3, 2017 at 8:53 PM, Stack <st...@duboce.net> wrote: > > > > > Thank you for the detail. > > > > > > Pardon the questions below asking for yet more detail. I am unable to > > > reproduce locally or on another os (though we see this issue up on our > > > build box). > > > > > > What is your OS when you see the below? > > > > > > On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra <amitkabrai...@gmail.com> > > > wrote: > > > > > > > Hello, > > > > > > > > I am using "branch-2" branch of hbase, when I run unit test I get > > > following > > > > error for netty "java.lang.UnsatisfiedLinkError: failed to load the > > > > required native library" > > > > > > > > > > > > This is running a unit test in your eclipse environment? > > > > > > You are trying to run an hbase-spark unit test when you see the above? > > > > > > > > > > > > > > > > *I already have following set in "maven-surefire-plugin" in pom.xml > as > > > > per http://hbase.apache.org/book.html#thirdparty > > > > <http://hbase.apache.org/book.html#thirdparty>* > > > > > > > > > > > > > > > > > > Are you embedding hbase into your application? > > > > > > > > > > > > > <systemPropertyVariables> > > > > <!-- > > > > <test.build.classes>${test.build.classes}</test.build. > > > classe > > > > s> > > > > --> > > > > <!--For shaded netty, to find the relocated .so. > > > > Trick from > > > > https://stackoverflow.com/questions/33825743/rename- > > > > files-inside-a-jar-using-some-maven-plugin > > > > <https://stackoverflow.com/questions/33825743/rename- > > > files-inside-a-jar-using-some-maven-plugin> > > > > > > > > The netty jar has a .so in it. Shading requires > rename > > of > > > > the .so and then passing a system > > > > property so netty finds the renamed .so and > associates > > it > > > > w/ the relocated netty files. > > > > > > > > The relocated netty is in hbase-thirdparty > dependency. > > > Just > > > > set this propery globally rather > > > > than per module. > > > > --> > > > > <org.apache.hadoop.hbase. > shaded.io.netty.packagePrefix> > > > > org.apache.hadoop.hbase.shaded.</org.apache.hadoop. > > > hbase.shaded.io.netty. > > > > packagePrefix> > > > > </systemPropertyVariables> > > > > > > > > > > > > > > > > *And I see in the code as per HBASE-18271, all io.netty is already > > > replaced > > > > with org.apache.hadoop.hbase.shaded.io.netty* > > > > > > > > > > > The trailing period is also present? > > > > > > > > > > > > > > > > > If I run a test from eclipse , I see the error immediately and my > test > > > > doesn't run, but when I run from command line , the test runs but I > get > > > the > > > > error at the end when the mvn command finishes. > > > > > > > > > > > > Is it any eclipse test? > > > > > > Thank you. Let me try and fix this this morning. > > > > > > S > > > > > > > > > > > > > > > > > > > *Here is the complete error output.* > > > > > > > > > > > > > > > > [INFO] > > > > [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ > > > hbase-spark > > > > --- > > > > [INFO] > > > > [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark --- > > > > Discovery starting. > > > > Discovery completed in 1 second, 558 milliseconds. > > > > Run starting. Expected test count is: 79 > > > > HBaseDStreamFunctionsSuite: > > > > Formatting using clusterid: testClusterID > > > > *** RUN ABORTED *** > > > > java.io.IOException: Shutting down > > > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( > > > > MiniHBaseCluster.java:232) > > > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( > > > > MiniHBaseCluster.java:94) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility. > > startMiniHBaseCluster( > > > > HBaseTestingUtility.java:1124) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:1078) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:949) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:943) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:872) > > > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite. > > beforeAll( > > > > HBaseDStreamFunctionsSuite.scala:41) > > > > at org.scalatest.BeforeAndAfterAll$class. > > beforeAll(BeforeAndAfterAll. > > > > scala:187) > > > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite. > > beforeAll( > > > > HBaseDStreamFunctionsSuite.scala:30) > > > > ... > > > > Cause: java.lang.RuntimeException: Failed construction of Master: > > class > > > > org.apache.hadoop.hbase.master.HMasterorg.apache. > > > > hadoop.hbase.shaded.io.netty.channel.epoll. > > > NativeStaticallyReferencedJniM > > > > ethods.epollin()I > > > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( > > > > JVMClusterUtil.java:145) > > > > at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster( > > > > LocalHBaseCluster.java:217) > > > > at org.apache.hadoop.hbase.LocalHBaseCluster.<init>( > > > > LocalHBaseCluster.java:152) > > > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( > > > > MiniHBaseCluster.java:214) > > > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( > > > > MiniHBaseCluster.java:94) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility. > > startMiniHBaseCluster( > > > > HBaseTestingUtility.java:1124) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:1078) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:949) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:943) > > > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > > > HBaseTestingUtility.java:872) > > > > ... > > > > Cause: java.lang.UnsatisfiedLinkError: failed to load the required > > > > native > > > > library > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > Epoll.ensureAvailability(Epoll.java:78) > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) > > > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( > > > > NettyEventLoopGroupConfig.java:61) > > > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< > > > > init>(HRegionServer.java:552) > > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > > Method) > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > > > > NativeConstructorAccessorImpl.java:62) > > > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > > > > DelegatingConstructorAccessorImpl.java:45) > > > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( > > > > JVMClusterUtil.java:140) > > > > ... > > > > Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase. > > > > shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM > > > > ethods.epollin()I > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > NativeStaticallyReferencedJniMethods.epollin(Native Method) > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > Native.<clinit>(Native.java:66) > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > Epoll.<clinit>(Epoll.java:33) > > > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) > > > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( > > > > NettyEventLoopGroupConfig.java:61) > > > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< > > > > init>(HRegionServer.java:552) > > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > > Method) > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > > > > NativeConstructorAccessorImpl.java:62) > > > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > > > > DelegatingConstructorAccessorImpl.java:45) > > > > ... > > > > [INFO] ------------------------------------------------------------ > > > > ------------ > > > > [INFO] Reactor Summary: > > > > [INFO] > > > > [INFO] Apache HBase ....................................... SUCCESS > [ > > > > 1.575 s] > > > > [INFO] Apache HBase - Checkstyle .......................... SUCCESS [ > > > > 0.317 s] > > > > [INFO] Apache HBase - Annotations ......................... SUCCESS [ > > > > 0.537 s] > > > > [INFO] Apache HBase - Build Configuration ................. SUCCESS [ > > > > 0.053 s] > > > > [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [ > > > > 15.410 s] > > > > [INFO] Apache HBase - Common .............................. SUCCESS [ > > > > 4.603 s] > > > > [INFO] Apache HBase - Metrics API ......................... SUCCESS [ > > > > 1.213 s] > > > > [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [ > > > > 0.985 s] > > > > [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [ > > > > 0.863 s] > > > > [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [ > > > > 1.750 s] > > > > [INFO] Apache HBase - Protocol ............................ SUCCESS [ > > > > 4.880 s] > > > > [INFO] Apache HBase - Client .............................. SUCCESS [ > > > > 5.233 s] > > > > [INFO] Apache HBase - Replication ......................... SUCCESS [ > > > > 1.040 s] > > > > [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [ > > > > 1.121 s] > > > > [INFO] Apache HBase - Procedure ........................... SUCCESS [ > > > > 1.084 s] > > > > [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [ > > > > 0.092 s] > > > > [INFO] Apache HBase - Server .............................. SUCCESS [ > > > > 19.849 s] > > > > [INFO] Apache HBase - MapReduce ........................... SUCCESS [ > > > > 4.221 s] > > > > [INFO] Apache HBase - Testing Util ........................ SUCCESS [ > > > > 3.273 s] > > > > [INFO] Apache HBase - Thrift .............................. SUCCESS [ > > > > 5.519 s] > > > > [INFO] Apache HBase - RSGroup ............................. SUCCESS [ > > > > 3.408 s] > > > > [INFO] Apache HBase - Shell ............................... SUCCESS > [ > > > > 3.859 s] > > > > [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [ > > > > 4.038 s] > > > > [INFO] Apache HBase - Backup .............................. SUCCESS > > > [01:13 > > > > min] > > > > [INFO] Apache HBase - Integration Tests ................... SUCCESS [ > > > > 4.229 s] > > > > [INFO] Apache HBase - Examples ............................ SUCCESS [ > > > > 3.471 s] > > > > [INFO] Apache HBase - Rest ................................ SUCCESS > [ > > > > 4.448 s] > > > > [INFO] Apache HBase - External Block Cache ................ SUCCESS [ > > > > 2.040 s] > > > > [INFO] Apache HBase - Spark ............................... FAILURE > [ > > > > 32.833 s] > > > > [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED > > > > [INFO] Apache HBase - Assembly ............................ SKIPPED > > > > [INFO] Apache HBase - Shaded .............................. SKIPPED > > > > [INFO] Apache HBase - Shaded - Client ..................... SKIPPED > > > > [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED > > > > [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED > > > > [INFO] Apache HBase - Archetypes .......................... SKIPPED > > > > [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED > > > > [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype > > SKIPPED > > > > [INFO] Apache HBase - Archetype builder ................... SKIPPED > > > > [INFO] ------------------------------------------------------------ > > > > ------------ > > > > [INFO] BUILD FAILURE > > > > [INFO] ------------------------------------------------------------ > > > > ------------ > > > > [INFO] Total time: 03:26 min > > > > [INFO] Finished at: 2017-09-27T19:34:35+05:30 > > > > [INFO] Final Memory: 345M/6055M > > > > [INFO] ------------------------------------------------------------ > > > > ------------ > > > > [ERROR] Failed to execute goal org.scalatest:scalatest-maven- > > > > plugin:1.0:test > > > > (test) on project hbase-spark: There are test failures -> [Help 1] > > > > [ERROR] > > > > [ERROR] To see the full stack trace of the errors, re-run Maven with > > the > > > -e > > > > switch. > > > > [ERROR] Re-run Maven using the -X switch to enable full debug > logging. > > > > [ERROR] > > > > [ERROR] For more information about the errors and possible solutions, > > > > please read the following articles: > > > > [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ > > > > MojoFailureException > > > > [ERROR] > > > > [ERROR] After correcting the problems, you can resume the build with > > the > > > > command > > > > [ERROR] mvn <goals> -rf :hbase-spark > > > > > > > > > >
