[
https://issues.apache.org/jira/browse/DRILL-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061413#comment-16061413
]
N Campbell edited comment on DRILL-5256 at 6/29/17 3:00 AM:
------------------------------------------------------------
The Apache JDBC drill will load if one explicitly sets
-Doadd.io.netty.allocator.numDirectArenas=<n> as a workaround.
This work around does not help the MAPR JDBC driver where you have to use
-Dio.netty.allocator.numDirectArenas
https://netty.io/4.1/api/io/netty/buffer/PooledByteBufAllocator.html
Enabling logging using the same JAR files but using an ORACLE JRE vs IBM JRE
shows different output for the arena size being reported. In both cases no -D
option was used to specify a value.
*ORACLE*
Processors 4
getPackage.getName(com.mapr.drill.jdbc41)
getPackage.getName(Driver)
main DEBUG scanner.ClassPathScanner - Scanning classpath for resources with
pathname "drill-module.conf".
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-auth-mechanism-maprsasl-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-hive-exec-shaded-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-java-exec-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-logical-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-memory-base-1.10.0.jar!/drill-module.conf.
main INFO config.DrillConfig - Configuration and plugin file(s) identified in
120ms.
Base Configuration:
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-memory-base-1.10.0.jar!/drill-module.conf
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-auth-mechanism-maprsasl-1.10.0.jar!/drill-module.conf
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-hive-exec-shaded-1.10.0.jar!/drill-module.conf
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-logical-1.10.0.jar!/drill-module.conf
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-java-exec-1.10.0.jar!/drill-module.conf
-
jar:file:/D:/Xfer/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-module.conf
main DEBUG config.DrillConfig - Setting up DrillConfig object.
main DEBUG config.DrillConfig - DrillConfig object initialized.
main DEBUG logging.InternalLoggerFactory - Using SLF4J as the default logging
framework
main DEBUG internal.PlatformDependent0 - java.nio.Buffer.address: available
main DEBUG internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
main DEBUG internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
main DEBUG internal.PlatformDependent0 - java.nio.Bits.unaligned: true
main DEBUG internal.PlatformDependent - Platform: Windows
main DEBUG internal.PlatformDependent - Java version: 8
main DEBUG internal.PlatformDependent - -Dio.netty.noUnsafe: false
main DEBUG internal.PlatformDependent - sun.misc.Unsafe: available
main DEBUG internal.PlatformDependent - -Dio.netty.noJavassist: false
main DEBUG internal.PlatformDependent - Javassist: available
main DEBUG internal.PlatformDependent - -Dio.netty.tmpdir:
C:\Users\campbeln\AppData\Local\Temp\2 (java.io.tmpdir)
main DEBUG internal.PlatformDependent - -Dio.netty.bitMode: 32
(sun.arch.data.model)
main DEBUG internal.PlatformDependent - -Dio.netty.noPreferDirect: false
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas:
2
* main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.numDirectArenas: 2*
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize:
16777216
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize:
512
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.smallCacheSize: 256
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.normalCacheSize: 64
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.maxCachedBufferCapacity: 32768
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.cacheTrimInterval: 8192
main DEBUG util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
IBM JRE
Processors 8
getPackage.getName(com.mapr.drill.jdbc41)
getPackage.getName(Driver)
main DEBUG scanner.ClassPathScanner - Scanning classpath for resources with
pathname "drill-module.conf".
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-auth-mechanism-maprsasl-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-hive-exec-shaded-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-java-exec-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-logical-1.10.0.jar!/drill-module.conf.
main DEBUG scanner.ClassPathScanner - - collected resource URL
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-memory-base-1.10.0.jar!/drill-module.conf.
main INFO config.DrillConfig - Configuration and plugin file(s) identified in
117ms.
Base Configuration:
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-hive-exec-shaded-1.10.0.jar!/drill-module.conf
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-java-exec-1.10.0.jar!/drill-module.conf
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-memory-base-1.10.0.jar!/drill-module.conf
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-logical-1.10.0.jar!/drill-module.conf
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-auth-mechanism-maprsasl-1.10.0.jar!/drill-module.conf
-
jar:file:/C:/tmp/Drill/DrillJDBC41/drill-common-1.10.0.jar!/drill-module.conf
main DEBUG config.DrillConfig - Setting up DrillConfig object.
main DEBUG config.DrillConfig - DrillConfig object initialized.
main DEBUG logging.InternalLoggerFactory - Using SLF4J as the default logging
framework
main DEBUG internal.PlatformDependent0 - java.nio.Buffer.address: available
main DEBUG internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
main DEBUG internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
main DEBUG internal.PlatformDependent0 - java.nio.Bits.unaligned: true
main DEBUG internal.PlatformDependent - Platform: Windows
main DEBUG internal.PlatformDependent - Java version: 8
main DEBUG internal.PlatformDependent - -Dio.netty.noUnsafe: false
main DEBUG internal.PlatformDependent - sun.misc.Unsafe: available
main DEBUG internal.PlatformDependent - -Dio.netty.noJavassist: false
main DEBUG internal.PlatformDependent - Javassist: available
main DEBUG internal.PlatformDependent - -Dio.netty.tmpdir:
C:\Users\campbeln\AppData\Local\Temp (java.io.tmpdir)
main DEBUG internal.PlatformDependent - -Dio.netty.bitMode: 64
(sun.arch.data.model)
main DEBUG internal.PlatformDependent - -Dio.netty.noPreferDirect: false
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas:
5
* main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.numDirectArenas: 0*
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize:
16777216
main DEBUG buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize:
512
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.smallCacheSize: 256
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.normalCacheSize: 64
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.maxCachedBufferCapacity: 32768
main DEBUG buffer.PooledByteBufAllocator -
-Dio.netty.allocator.cacheTrimInterval: 8192
was (Author: the6campbells):
The Apache JDBC drill will load if one explicitly sets
-Doadd.io.netty.allocator.numDirectArenas=<n> as a workaround.
This work around does not help the MAPR JDBC driver where you have to use
-Dio.netty.allocator.numDirectArenas
https://netty.io/4.1/api/io/netty/buffer/PooledByteBufAllocator.html
> Exception in thread "main" java.lang.ExceptionInInitializerError
> ----------------------------------------------------------------
>
> Key: DRILL-5256
> URL: https://issues.apache.org/jira/browse/DRILL-5256
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
> Affects Versions: 1.7.0, 1.8.0
> Environment: Windows 7, IBM SDK 1.6 and IBM SDK 1.7
> Reporter: Vasu
> Labels: jvm
>
> Below error while connecting to Drill Server
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.J9VMInternals.initialize(J9VMInternals.java:257)
> at
> oadd.org.apache.drill.exec.memory.BaseAllocator.<clinit>(BaseAllocator.java:44)
> at java.lang.J9VMInternals.initializeImpl(Native Method)
> at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
> at java.lang.J9VMInternals.initialize(J9VMInternals.java:202)
> at
> oadd.org.apache.drill.exec.memory.RootAllocatorFactory.newRoot(RootAllocatorFactory.java:38)
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:143)
> at
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:64)
> at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at
> oadd.net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at java.sql.DriverManager.getConnection(DriverManager.java:583)
> at java.sql.DriverManager.getConnection(DriverManager.java:245)
> at com.trianz.drill.ApacheDrillDemo.main(ApacheDrillDemo.java:13)
> Caused by: java.lang.NullPointerException
> at
> oadd.io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.<init>(PooledByteBufAllocatorL.java:93)
> at
> oadd.io.netty.buffer.PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:56)
> at
> oadd.org.apache.drill.exec.memory.AllocationManager.<clinit>(AllocationManager.java:60)
> at java.lang.J9VMInternals.initializeImpl(Native Method)
> at java.lang.J9VMInternals.initialize(J9VMInternals.java:235)
> ... 13 more
> When I tried to debug in to source code, following is the place where we are
> getting NULL POINTER EXCEPTION
>
> drill/exec/memory/base/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java
>
> Line: 93: this.chunkSize = directArenas[0].chunkSize;
> Below is the code snapshot.
> public InnerAllocator() {
> super(true);
> try {
> Field f =
> PooledByteBufAllocator.class.getDeclaredField("directArenas");
> f.setAccessible(true);
> this.directArenas = (PoolArena<ByteBuffer>[]) f.get(this);
> } catch (Exception e) {
> throw new RuntimeException("Failure while initializing allocator.
> Unable to retrieve direct arenas field.", e);
> }
> this.chunkSize = directArenas[0].chunkSize;
> if (memoryLogger.isTraceEnabled()) {
> Can anyone please help on this? Thanks in advance.
> Thanks,
> Vasu T
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
