[
https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875211#comment-15875211
]
ASF GitHub Bot commented on DRILL-5157:
---------------------------------------
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/707
@bitblender - The general problem with jar dependencies is that each Maven
project depends on a different version of library x. However, the JVM can load
only one version (if everything goes into the global class loader, as in
Drill.) So, we have to exclude all the random versions to ensure that a single
version is included. In general, we must assume that newer versions are
compatible with old versions. Else, we're in a world of hurt. (Guava is a good
example: Guava removes symbols that used to exist, so that a newer version is
not usable by code that needs an older one.)
All we can do is test to ensure that 1) we have a single version of the
library, and 2) everything works with that version. The nature of this PR is
that we violated item 1: we have multiple snappy versions. The one that gets
used depends on how the class path is formed: one goes first in production, a
different one in the IDE.
> Multiple Snappy versions on class path; causes unit test failures
> -----------------------------------------------------------------
>
> Key: DRILL-5157
> URL: https://issues.apache.org/jira/browse/DRILL-5157
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
>
> Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It
> fails with the following stack trace and the memory leak trace shown second.
> Strangely, this error appears only if the test is run as part of the overall
> suite. The error does not appear if the test is run individually in the
> debugger. This suggests that the problem described here is a side-effect of a
> problem created by an earlier test.
> Investigation (see notes below) reveal that this is a Maven dependency
> management problem.
> Stack trace that seems to show that the code was trying to find a Snappy
> native library:
> {code}
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402)
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169)
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96)
> at
> org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219)
> at
> org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479)
> at
> org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562)
> at
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178)
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> …
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
