[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-28 Thread Hanifi Gunes (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hanifi Gunes updated DRILL-2292:

Assignee: Deneche A. Hakim  (was: Hanifi Gunes)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt, 
 DRILL-2292.3.patch.txt, DRILL-2292.4.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-26 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Attachment: DRILL-2292.4.patch.txt

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt, 
 DRILL-2292.3.patch.txt, DRILL-2292.4.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-26 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Assignee: Hanifi Gunes  (was: Deneche A. Hakim)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt, 
 DRILL-2292.3.patch.txt, DRILL-2292.4.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-21 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Attachment: DRILL-2292.3.patch.txt

one final round of testing. I will update once the tests are done

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt, 
 DRILL-2292.3.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-20 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Assignee: Hanifi Gunes  (was: Steven Phillips)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-19 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Assignee: Steven Phillips  (was: Deneche A. Hakim)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Steven Phillips
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-17 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Fix Version/s: (was: 0.9.0)
   1.0.0

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 1.0.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-16 Thread Steven Phillips (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Phillips updated DRILL-2292:
---
Assignee: Deneche A. Hakim  (was: Steven Phillips)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 0.9.0

 Attachments: DRILL-2292.1.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-16 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Attachment: DRILL-2292.2.patch.txt

fixed RepeatedMapReaderImpl and RepeatedListReaderImpl to properly handle the 
following cases:
- repeated map nested in a repeated map
- repeated map nested in a repeated list
- repeated list nested in a repeated map
- repeated list nested in a repeated list

added unit test to make sure all cases are working

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 0.9.0

 Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-15 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Attachment: DRILL-2292.1.patch.txt

- fixed {{RepeatedMapReaderImpl.setPosition()}} to handle when the index is 
negative or *NO_VALUES*
- added unit test {{TestRepeatedReaders.testReadNestedRepeatedMap()}}

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 0.9.0

 Attachments: DRILL-2292.1.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-04-15 Thread Deneche A. Hakim (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-2292:

Assignee: Steven Phillips  (was: Deneche A. Hakim)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Steven Phillips
Priority: Blocker
 Fix For: 0.9.0

 Attachments: DRILL-2292.1.patch.txt


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps

2015-03-20 Thread Jacques Nadeau (JIRA) 

 [ 
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacques Nadeau updated DRILL-2292:
--
Assignee: Deneche A. Hakim  (was: Steven Phillips)

 CTAS broken when we have repeated maps
 --

 Key: DRILL-2292
 URL: https://issues.apache.org/jira/browse/DRILL-2292
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 0.9.0


 git.commit.id.abbrev=6676f2d
 Data Set :
 {code}
 {
   id : 1,
   map:{rm: [
 {mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a: 
 foo},{b:boo}]},
 {mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a: 
 bar},{c:1},{d:4.5}]}
   ]}
 }
 {code}
 The below CTAS statement fails though the underlying query succeeds
 {code}
 create table rep_map as select d.map from `temp.json` d;
 Query failed: Query stopped., index: -4, length: 4 (expected: range(0, 
 16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
 {code}
 Error from the logs :
 {code}
 Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4 
 (expected: range(0, 16384))
 at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
 at 
 org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309) 
 ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 ... 19 common frames omitted
 2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR 
 o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error 
 7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
 java.io.IOException: Failed to initialize FieldWriters.
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
  ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)