[ https://issues.apache.org/jira/browse/DRILL-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arina Ielchiieva updated DRILL-3468: ------------------------------------ Fix Version/s: 1.9.0 > CTAS IOB > -------- > > Key: DRILL-3468 > URL: https://issues.apache.org/jira/browse/DRILL-3468 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow > Affects Versions: 1.2.0 > Reporter: Khurram Faraaz > Priority: Critical > Fix For: 1.9.0 > > > I am seeing a IOB when I use same table name in CTAS, after deleting the > previously create parquet file. > {code} > 0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT > CAST(columns[0] as INT ), CAST(columns[1] as BIGINT ), CAST(columns[2] as > CHAR(2) ), CAST(columns[3] as VARCHAR(52) ), CAST(columns[4] as TIMESTAMP ), > CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ), CAST(columns[7] as > DOUBLE), CAST( columns[8] as TIME) FROM `allData.csv`; > +-----------+----------------------------+ > | Fragment | Number of records written | > +-----------+----------------------------+ > | 0_0 | 11196 | > +-----------+----------------------------+ > 1 row selected (1.864 seconds) > {code} > Remove the parquet file that was created by the above CTAS. > {code} > [root@centos-01 aggregates]# hadoop fs -ls /tmp/tbl_allData > Found 1 items > -rwxr-xr-x 3 mapr mapr 397868 2015-07-07 21:08 > /tmp/tbl_allData/0_0_0.parquet > [root@centos-01 aggregates]# hadoop fs -rm /tmp/tbl_allData/0_0_0.parquet > 15/07/07 21:10:47 INFO Configuration.deprecation: io.bytes.per.checksum is > deprecated. Instead, use dfs.bytes-per-checksum > 15/07/07 21:10:47 INFO fs.TrashPolicyDefault: Namenode trash configuration: > Deletion interval = 0 minutes, Emptier interval = 0 minutes. > Deleted /tmp/tbl_allData/0_0_0.parquet > {code} > I see a IOB when I CTAS with same table name as the one that was removed in > the above step. > {code} > 0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_allData AS SELECT > CAST(columns[0] as INT ), CAST(columns[1] as BIGINT ), CAST(columns[2] as > CHAR(2) ), CAST(columns[3] as VARCHAR(52) ), CAST(columns[4] as TIMESTAMP ), > CAST(columns[5] as DATE ), CAST(columns[6] as BOOLEAN ), CAST(columns[7] as > DOUBLE), CAST( columns[8] as TIME) FROM `lessData.csv`; > Error: SYSTEM ERROR: IndexOutOfBoundsException: Index: 0, Size: 0 > [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] > (state=,code=0) > {code} > stack trace from drillbit.log > {code} > org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception > during fragment initialization: Failure while trying to check if a table or > view with given name [tbl_allData] already exists in schema [dfs.tmp]: Index: > 0, Size: 0 > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) > [drill-java-exec-1.1.0.jar:1.1.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure > while trying to check if a table or view with given name [tbl_allData] > already exists in schema [dfs.tmp]: Index: 0, Size: 0 > at > org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) > [drill-java-exec-1.1.0.jar:1.1.0] > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) > [drill-java-exec-1.1.0.jar:1.1.0] > ... 3 common frames omitted > Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 > at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45] > at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45] > at > org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219) > ~[drill-java-exec-1.1.0.jar:1.1.0] > ... 7 common frames omitted > 2015-07-07 21:12:14,643 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] INFO > o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > 2015-07-07 21:12:14,648 [2a63bf51-09ff-467b-1aab-df2b5030e9b9:foreman] ERROR > o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: > IndexOutOfBoundsException: Index: 0, Size: 0 > [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IndexOutOfBoundsException: Index: 0, Size: 0 > [Error Id: 6d6df8e9-699c-4475-8ad3-183c0a91dc99 on centos-02.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523) > ~[drill-common-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:737) > [drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:839) > [drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:781) > [drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73) > [drill-common-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:783) > [drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:892) > [drill-java-exec-1.1.0.jar:1.1.0] > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:253) > [drill-java-exec-1.1.0.jar:1.1.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected > exception during fragment initialization: Failure while trying to check if a > table or view with given name [tbl_allData] already exists in schema > [dfs.tmp]: Index: 0, Size: 0 > ... 4 common frames omitted > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Failure > while trying to check if a table or view with given name [tbl_allData] > already exists in schema [dfs.tmp]: Index: 0, Size: 0 > at > org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:222) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:88) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903) > [drill-java-exec-1.1.0.jar:1.1.0] > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242) > [drill-java-exec-1.1.0.jar:1.1.0] > ... 3 common frames omitted > Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 > at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45] > at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45] > at > org.apache.drill.exec.store.dfs.FileSelection.getFirstPath(FileSelection.java:100) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.BasicFormatMatcher.isReadable(BasicFormatMatcher.java:75) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:303) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.create(WorkspaceSchemaFactory.java:118) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:241) > ~[drill-java-exec-1.1.0.jar:1.1.0] > at > org.apache.drill.exec.planner.sql.handlers.SqlHandlerUtil.getTableFromSchema(SqlHandlerUtil.java:219) > ~[drill-java-exec-1.1.0.jar:1.1.0] > ... 7 common frames omitted > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)