----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11587/ -----------------------------------------------------------
(Updated June 4, 2013, 5:33 a.m.) Review request for drill and Jacques Nadeau. Changes ------- Addressing review comments. Description ------- Added the JSONRecordReader based on the previous ScanJson work. Does not support nested fields, maps or lists yet. Currently it detects to move on to the next batch when any of the field batch cannot hold another item for the current item being written. This also assumes the default batch size can always hold at least one item from any field (which only is a problem for variable length vectors). Diffs (updated) ----- sandbox/prototype/common/src/main/java/org/apache/drill/common/logical/data/Scan.java c8d396b sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/DeadBuf.java dafb68c sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/AbstractFixedValueVector.java b32f067 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/BaseValueVector.java b001add sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/Fixed4.java 650029d sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableFixed4.java cc18538 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableValueVector.java 692ab87 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableVarLen4.java PRE-CREATION sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/TypeHelper.java 8e89c41 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen1.java d87029d sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen2.java ebd440a sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen4.java b3cd712 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VariableVector.java 4247f14 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/InboundRpcMessage.java be1ff6b sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/DiffSchema.java 016e097 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/Field.java e19c099 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/IdGenerator.java 728e8e1 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/ListSchema.java efdc8fd sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/NamedField.java aa0d6aa sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/OrderedField.java 67fd2fa sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/SchemaRecorder.java 54a4e0e sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/JacksonHelper.java 0643710 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperator.java e450ee9 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperatorIterator.java bf4053e sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/ScanJson.java a1c30e9 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/ProtobufSchemaTransformer.java a81a9d9 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/SchemaTransformer.java 54d851f sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java d5aaab2 sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/BatchExceededException.java PRE-CREATION sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/JSONRecordReader.java PRE-CREATION sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java 67c84ed sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/VectorHolder.java PRE-CREATION sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/JSONRecordReaderTest.java PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_1.json PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_2.json PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_3.json PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_4.json PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_5.json PRE-CREATION sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_6.json PRE-CREATION sandbox/prototype/exec/ref/src/main/java/org/apache/drill/exec/ref/rops/MultiLevelMap.java PRE-CREATION sandbox/prototype/exec/ref/src/main/java/org/apache/drill/exec/ref/rops/WindowingROP.java PRE-CREATION sandbox/prototype/exec/ref/src/test/resources/donuts.json 2d98b59 sandbox/prototype/exec/ref/src/test/resources/simple_plan.json 0f38476 sandbox/prototype/pom.xml 25f156d Diff: https://reviews.apache.org/r/11587/diff/ Testing ------- Thanks, Timothy Chen
