[ https://issues.apache.org/jira/browse/DRILL-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886026#comment-15886026 ]
Zelaine Fong edited comment on DRILL-5300 at 2/27/17 4:06 PM: -------------------------------------------------------------- Based on these lines in your stack trace: {code} ... 5 common frames omitted 2017-02-27 04:32:57,867 [drill-executor-453] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception. java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) ~[drill-java-exec-1.9.0.jar:1.9.0] {code} The memory leak appears to be DRILL-5160. The missing snappy dependency is DRILL-5157. If you pick up the fix for DRILL-5157, that will avoid the dependency problem you're hitting. was (Author: zfong): Based on these lines in your stack trace: ... 5 common frames omitted 2017-02-27 04:32:57,867 [drill-executor-453] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception. java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402) ~[drill-java-exec-1.9.0.jar:1.9.0] The memory leak appears to be DRILL-5160. The missing snappy dependency is DRILL-5157. If you pick up the fix for DRILL-5157, that will avoid the dependency problem you're hitting. > SYSTEM ERROR: IllegalStateException: Memory was leaked by query while > querying parquet files > -------------------------------------------------------------------------------------------- > > Key: DRILL-5300 > URL: https://issues.apache.org/jira/browse/DRILL-5300 > Project: Apache Drill > Issue Type: Bug > Affects Versions: 1.9.0 > Environment: OS: Linux > Reporter: Muhammad Gelbana > Attachments: both_queries_logs.zip > > > Running the following query against parquet files (I modified some values for > privacy reasons) > {code:title=Query causing the long logs|borderStyle=solid} > SELECT AL4.NAME, AL5.SEGMENT2, SUM(AL1.AMOUNT), AL2.ATTRIBUTE4, > AL2.XXXXXXX_XXXXXXXX_CODE, AL8.D_BU, AL8.F_PL, AL18.COUNTRY, AL13.COUNTRY, > AL11.NAME FROM > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XX/RA_XXXX_TRX_LINE_GL_DIST_ALL` > AL1, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XX/RA_XXXXOMER_TRX_ALL` > AL2, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_XXXXXXX` > AL3, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_HR_COMMON/HR_ALL_ORGANIZATION_UNITS` > AL4, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_CODE_COMBINATIONS` > AL5, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXXXXXXX/XXAT_AR_MU_TAB` > AL8, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_FIN_COMMON/GL_XXXXXXX` > AL11, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXXX_XXXXS` > AL12, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_LOCATIONS` > AL13, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_XXXXS_ALL` > AL14, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_USES_ALL` > AL15, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_XXXXS_ALL` > AL16, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXX_XXXX_USES_ALL` > AL17, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_LOCATIONS` > AL18, > dfs.`/disk2/XXXXXXX/XXXXXXX/XXXXXXXX/data/../parquet/XXX_XXXXX_COMMON/XX_XXXXX_XXXXS` > AL19 WHERE (AL2.SHIP_TO_XXXX_USE_ID = AL15.XXXX_USE_ID AND > AL15.XXXX_XXXX_XXXX_ID = AL14.XXXX_XXXX_XXXX_ID AND AL14.XXXXX_XXXX_ID = > AL12.XXXXX_XXXX_ID AND AL12.LOCATION_ID = AL13.LOCATION_ID AND > AL17.XXXX_XXXX_XXXX_ID = AL16.XXXX_XXXX_XXXX_ID AND AL16.XXXXX_XXXX_ID = > AL19.XXXXX_XXXX_ID AND AL19.LOCATION_ID = AL18.LOCATION_ID AND > AL2.BILL_TO_XXXX_USE_ID = AL17.XXXX_USE_ID AND AL2.SET_OF_XXXXX_ID = > AL3.SET_OF_XXXXX_ID AND AL1.CODE_COMBINATION_ID = AL5.CODE_COMBINATION_ID AND > AL5.SEGMENT4 = AL8.MU AND AL1.SET_OF_XXXXX_ID = AL11.SET_OF_XXXXX_ID AND > AL2.ORG_ID = AL4.ORGANIZATION_ID AND AL2.XXXXOMER_TRX_ID = > AL1.XXXXOMER_TRX_ID) AND ((AL5.SEGMENT2 = '400001' AND AL1.AMOUNT <> 0 AND > AL4.NAME IN ('XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', > 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX', 'XXX-XX-XXXX') > AND AL3.NAME like '%-PR-%')) GROUP BY AL4.NAME, AL5.SEGMENT2, AL2.ATTRIBUTE4, > AL2.XXXXXXX_XXXXXXXX_CODE, AL8.D_BU, AL8.F_PL, AL18.COUNTRY, AL13.COUNTRY, > AL11.NAME > {code} > {code:title=Query causing the short logs|borderStyle=solid} > SELECT AL11.NAME > FROM > dfs.`/XXXXXXX/XXXXXXX/XXXXXXX/data/../parquet/XXX_XXX_COMMON/GL_XXXXXXX` XXXX > LIMIT 10 > {code} > This issue may be a duplicate for [this > one|https://issues.apache.org/jira/browse/DRILL-4398] but I created a new one > based on [this > suggestion|https://issues.apache.org/jira/browse/DRILL-4398?focusedCommentId=15884846&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15884846]. -- This message was sent by Atlassian JIRA (v6.3.15#6346)