> On Aug. 21, 2014, 9:20 p.m., Hanifi Gunes wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader2.java, > > line 97 > > <https://reviews.apache.org/r/24758/diff/1/?file=661947#file661947line97> > > > > let's remove guard statement here.
We do actually need this guard here for the case where we are "reading" only coluumns that do not exist in the document. We cannot allocate the vectors in advance, because we do not know the types or potential future existence of a column ahead of time. Thus there cannot be any allocated vector to make a capacity check on each record read. > On Aug. 21, 2014, 9:20 p.m., Hanifi Gunes wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java, > > line 145 > > <https://reviews.apache.org/r/24758/diff/1/?file=661950#file661950line145> > > > > revise this. I tried looking around the MapWriter and related interfaces, and I'm not sure if it will be easy to clean this up. While I'm not sure it's the best way to leave it permenantly, it's not worth delaying this patch. I will open a JIRA to track it. - Jason ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24758/#review51209 ----------------------------------------------------------- On Aug. 22, 2014, 10:59 p.m., Jason Altekruse wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/24758/ > ----------------------------------------------------------- > > (Updated Aug. 22, 2014, 10:59 p.m.) > > > Review request for drill. > > > Repository: drill-git > > > Description > ------- > > Drill-1283 - JSON project pushdown > > > Diffs > ----- > > common/src/main/java/org/apache/drill/common/expression/PathSegment.java > 0ecfcd0 > common/src/main/java/org/apache/drill/common/expression/SchemaPath.java > 25ee8b4 > exec/java-exec/src/main/codegen/templates/BaseWriter.java e281bc7 > exec/java-exec/src/main/codegen/templates/ComplexWriters.java c390770 > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 406d883 > exec/java-exec/src/main/codegen/templates/ListWriters.java 278fddc > exec/java-exec/src/main/codegen/templates/MapWriters.java 7aa5a2e > > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java > 8b5577c > > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader2.java > 3e2c81c > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java > e310b81 > exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java > 01fcfe0 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java > d365c6f > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReaderWithState.java > cf885a4 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/ComplexWriterImpl.java > 2fa72f7 > > exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/VectorContainerWriter.java > 4f669c0 > exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java 5e52e82 > > exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java > 89beeb0 > > exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestJsonReader.java > 8b31958 > > exec/java-exec/src/test/resources/store/json/project_pushdown_json_physical_plan.json > PRE-CREATION > > exec/java-exec/src/test/resources/store/json/schema_change_int_to_string.json > PRE-CREATION > exec/java-exec/src/test/resources/store/json/single_column_long_file.json > PRE-CREATION > > exec/java-exec/src/test/resources/store/json/test_complex_read_with_star.json > PRE-CREATION > > Diff: https://reviews.apache.org/r/24758/diff/ > > > Testing > ------- > > Added new unit tests, ran full build on command line > > > Thanks, > > Jason Altekruse > >
