[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946099#comment-15946099 ] Kunal Khatua commented on DRILL-5157: - No QA Verification required > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Labels: ready-to-commit > Fix For: 1.10.0 > > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945871#comment-15945871 ] Paul Rogers commented on DRILL-5157: Development-only issue, no QA verification needed. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Labels: ready-to-commit > Fix For: 1.10.0 > > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876182#comment-15876182 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/707 Merged into master by @jinfengni , but he omitted the magic message to close the PR. Closing manually. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Labels: ready-to-commit > Fix For: 1.10.0 > > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876183#comment-15876183 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers closed the pull request at: https://github.com/apache/drill/pull/707 > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Labels: ready-to-commit > Fix For: 1.10.0 > > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875621#comment-15875621 ] ASF GitHub Bot commented on DRILL-5157: --- Github user jinfengni commented on the issue: https://github.com/apache/drill/pull/707 +1 LGTM > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875331#comment-15875331 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/707 @jinfengni - please review the revisions. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875271#comment-15875271 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/707 Result: ``` [INFO] | +- org.apache.avro:avro:jar:1.7.4:provided [INFO] | | +- (org.xerial.snappy:snappy-java:jar:1.0.4.1:provided - omitted for conflict with 1.1.2.6) ... \- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile ``` > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875227#comment-15875227 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/707#discussion_r102118993 --- Diff: pom.xml --- @@ -1772,6 +1799,28 @@ sqlline ${sqlline.version} + + org.apache.avro + avro + 1.7.4 + + + org.xerial.snappy + snappy-java + + + + + org.apache.parquet + parquet-hadoop + 1.8.1-drill-r0 --- End diff -- Fixed. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875211#comment-15875211 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/707 @bitblender - The general problem with jar dependencies is that each Maven project depends on a different version of library x. However, the JVM can load only one version (if everything goes into the global class loader, as in Drill.) So, we have to exclude all the random versions to ensure that a single version is included. In general, we must assume that newer versions are compatible with old versions. Else, we're in a world of hurt. (Guava is a good example: Guava removes symbols that used to exist, so that a newer version is not usable by code that needs an older one.) All we can do is test to ensure that 1) we have a single version of the library, and 2) everything works with that version. The nature of this PR is that we violated item 1: we have multiple snappy versions. The one that gets used depends on how the class path is formed: one goes first in production, a different one in the IDE. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875209#comment-15875209 ] ASF GitHub Bot commented on DRILL-5157: --- Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/707#discussion_r102117453 --- Diff: pom.xml --- @@ -1331,6 +1336,28 @@ + org.apache.avro + avro + 1.7.4 --- End diff -- hive-exec-shade includes Avro 1.7.5. java-exec uses 1.7.7. Hive-exec uses ${avro.version} -- which is not set anywhere that I can find. The key here is to exclude snappy, so I moved the exclusion into java-exec. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781819#comment-15781819 ] ASF GitHub Bot commented on DRILL-5157: --- GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/707 DRILL-5157: Multiple Snappy versions on class path Multiple Snappy versions on class path; causes unit test failures. Drill's pom.xml files bring in multiple Snappy versions. Drill itself brings in a very old version that has a known problem loading the snappy native library. Other libraries bring in a newer version. The one that ends up first on the class path is non-deterministic, leading to random test failures. This fix updates the Snappy library to the latest and adds dependency management to exclude older versions brought in by Avro and Parquet. You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-rogers/drill DRILL-5157 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/707.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #707 commit 909efcec92c4a259d139dfc4008c431969d061f7 Author: Paul Rogers Date: 2016-12-28T01:21:09Z DRILL-5157: Multiple Snappy versions on class path Multiple Snappy versions on class path; causes unit test failures. This fix updates the Snappy library and adds dependency management to exclude older versions brought in by Avro and Parquet. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {c
[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15773994#comment-15773994 ] Paul Rogers commented on DRILL-5157: Turns out the Snappy errors are due to missing Maven dependency management for the Snappy jar. Drill includes three different versions, including our ancient version from 2012. (See above.) The result is two Snappy versions on the class path: a newer one and an older one. (Again, see above.) The newer versions work correctly, our older one fails on the Mac. (Known problem discussed on Slashdot.) For some reason, one is used when tests pass, the other when the error below occurs. The solution is to move to the latest version of Snappy and add the proper Maven dependency management. Once that is done, life is good. > Multiple Snappy versions on class path; causes unit test failures > - > > Key: DRILL-5157 > URL: https://issues.apache.org/jira/browse/DRILL-5157 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > > Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It > fails with the following stack trace and the memory leak trace shown second. > Strangely, this error appears only if the test is run as part of the overall > suite. The error does not appear if the test is run individually in the > debugger. This suggests that the problem described here is a side-effect of a > problem created by an earlier test. > Investigation (see notes below) reveal that this is a Maven dependency > management problem. > Stack trace that seems to show that the code was trying to find a Snappy > native library: > {code} > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317) > at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219) > at org.xerial.snappy.Snappy.(Snappy.java:44) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96) > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219) > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178) > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479) > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562) > at > org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178) > at > org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) > … > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)