Re: Query a field with empty list

Currently in 1.2 I observe the following. 0: jdbc:drill:> select a from dfs.json.`/test.json`; +---+ | a | +---+ | null | +———+ But then flatten produces an error 0: jdbc:drill:> select flatten(a) from dfs.json.`/test.json`; Error: SYSTEM ERROR: ClassCastException: Cannot cast org.

[GitHub] drill pull request: Drill 3952

Github user adeneche commented on a diff in the pull request: https://github.com/apache/drill/pull/222#discussion_r43658717 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/DefaultFrameTemplate.java --- @@ -287,6 +318,36 @@ private int countPeers(f

[GitHub] drill pull request: Drill 3952

Github user adeneche commented on a diff in the pull request: https://github.com/apache/drill/pull/222#discussion_r43658789 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/DefaultFrameTemplate.java --- @@ -287,6 +318,36 @@ private int countPeers(f

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43660011 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java --- @@ -93,13 +93,15 @@ public void ensureAtLeastOneField(Complex

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43661248 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java --- @@ -189,8 +189,8 @@ protected void putVector(String name,

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43661592 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatch.java --- @@ -23,60 +23,214 @@ import org.apache.drill.exec.record.selection.

[jira] [Created] (DRILL-4005) Metadata Caching : Cache is getting created with null values in min, max, & original fields

Rahul Challapalli created DRILL-4005: Summary: Metadata Caching : Cache is getting created with null values in min, max, & original fields Key: DRILL-4005 URL: https://issues.apache.org/jira/browse/DRILL-4005

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43664058 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java --- @@ -335,7 +343,8 @@ public IterOutcome nextBatch()

[GitHub] drill pull request: Drill-3242: Offload RPC threads

GitHub user jacques-n opened a pull request: https://github.com/apache/drill/pull/229 Drill-3242: Offload RPC threads You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacques-n/drill DRILL-3242v2 Alternatively you can review and

[jira] [Created] (DRILL-4006) As json reader reads a field with empty lists, IOOB could happen

Sean Hsuan-Yi Chu created DRILL-4006: Summary: As json reader reads a field with empty lists, IOOB could happen Key: DRILL-4006 URL: https://issues.apache.org/jira/browse/DRILL-4006 Project: Apach

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43665769 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -122,6 +122,7 @@ protected void killIncoming(f

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43666158 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -325,10 +325,13 @@ public AggOutcome doWork() {

[GitHub] drill pull request: Drill 3952

Github user adeneche commented on a diff in the pull request: https://github.com/apache/drill/pull/222#discussion_r43668976 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/DefaultFrameTemplate.java --- @@ -287,6 +318,36 @@ private int countPeers(f

[GitHub] drill pull request: Drill 3952

Github user adeneche commented on the pull request: https://github.com/apache/drill/pull/222#issuecomment-153131868 Window operator has become increasingly complex. I already opened [DRILL-3662](https://issues.apache.org/jira/browse/DRILL-3662) to refactor the code. --- If your proj

[jira] [Created] (DRILL-4007) json reader treats empty list inconsistently,

Sean Hsuan-Yi Chu created DRILL-4007: Summary: json reader treats empty list inconsistently, Key: DRILL-4007 URL: https://issues.apache.org/jira/browse/DRILL-4007 Project: Apache Drill Is

Re: Review Request 35719: DRILL-3242: Update RPC layer so that requests and response are managed on a secondary thread.

> On July 16, 2015, 6:56 p.m., Sudheesh Katkam wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcBus.java, line 459 > > > > > > Should this just be logged? Previously, we threw the exception. > >

Re: Review Request 35719: DRILL-3242: Update RPC layer so that requests and response are managed on a secondary thread.

> On June 30, 2015, 7:55 p.m., Sudheesh Katkam wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java, > > line 148 > > > > > > The RPC thread could potentially spend quite s

[GitHub] drill pull request: Drill-3242: Offload RPC threads

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/229 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Resolved] (DRILL-3242) Enhance RPC layer to offload all request work onto a separate thread.

[ https://issues.apache.org/jira/browse/DRILL-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved DRILL-3242. --- Resolution: Fixed Resolved in e7db9dc > Enhance RPC layer to offload all request work onto a

Re: Query a field with empty list

There could be inconsistent results: https://issues.apache.org/jira/browse/DRILL-4007 On Mon, Nov 2, 2015 at 8:17 AM, Andries Engelbrecht < aengelbre...@maprtech.com> wrote: > Currently in 1.2 I observe the following. > > 0: jdbc:drill:> select a from dfs.json.`/test.json`; > +---+ > | a

Re: Query a field with empty list

Wouldn't it make more sense to return an empty list in this case? Null is not quite the same as an empty list. So, I would think you'd want a distinction between the two. -- Zelaine On Mon, Nov 2, 2015 at 11:41 AM, Hsuan Yi Chu wrote: > There could be inconsistent results: > https://issues.ap

Re: Query a field with empty list

I agree. I think in any context, empty list should appear as [] in the result. Any opposition? On Mon, Nov 2, 2015 at 11:50 AM, Zelaine Fong wrote: > Wouldn't it make more sense to return an empty list in this case? Null is > not quite the same as an empty list. So, I would think you'd want a

Re: Query a field with empty list

Empty list sounds like the appropriate thing. What would be the result of we do flatten on this list? I think user should get a row back in the results. On Mon, Nov 2, 2015 at 11:53 AM, Hsuan Yi Chu wrote: > I agree. I think in any context, empty list should appear as [] in the > result. > > A

Re: Query a field with empty list

Seems empty list is getting the vote for select. For consistency would it then make sense to have flatten produce a single row with an empty list in the result? —Andries > On Nov 2, 2015, at 12:55 PM, Neeraja Rentachintala > wrote: > > Empty list sounds like the appropriate thing. > What wo

Re: Query a field with empty list

I do agree that we should distinguish between empty lists and nulls. I consider this related to the outstanding task to allow for untyped nulls, as empty lists like this in JSON does not give us enough type information to materialize any known type. This does have the unfortunate property that we

Re: Query a field with empty list

Sounds to me that FLATTEN over an empty list NULL should produce 0 row. That's actually the behavior of UNNEST of empty array, or NULL, in Postgres. Unless we have a strong reason that FLATTEN should behavior differently than UNNEST, I think it makes sense for FLATTEN to produce 0 row over empty

[jira] [Created] (DRILL-4008) Difference in commit ID in sys.version and git.properties file

Khurram Faraaz created DRILL-4008: - Summary: Difference in commit ID in sys.version and git.properties file Key: DRILL-4008 URL: https://issues.apache.org/jira/browse/DRILL-4008 Project: Apache Drill

[jira] [Created] (DRILL-4009) Assert in a query with concat function in a group by and limit in outer query

Victoria Markman created DRILL-4009: --- Summary: Assert in a query with concat function in a group by and limit in outer query Key: DRILL-4009 URL: https://issues.apache.org/jira/browse/DRILL-4009 Pro

[jira] [Resolved] (DRILL-3313) Eliminate redundant #load methods and unit-test loading & exporting of vectors

[ https://issues.apache.org/jira/browse/DRILL-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanifi Gunes resolved DRILL-3313. - Resolution: Fixed Fixed by 77e2b89 > Eliminate redundant #load methods and unit-test loading & ex

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

GitHub user mehant opened a pull request: https://github.com/apache/drill/pull/230 DRILL-3941: Add timing instrumentation in different phases of partiti… …on pruning Add basic timing instrumentation while initializing FileSelection and reading the metadata cache file Y

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

Github user mehant commented on the pull request: https://github.com/apache/drill/pull/230#issuecomment-153201984 @amansinha100 can you please review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[jira] [Created] (DRILL-4010) In HBase reader, create child vectors for referenced HBase columns to avoid spurious schema changes

Daniel Barclay (Drill) created DRILL-4010: - Summary: In HBase reader, create child vectors for referenced HBase columns to avoid spurious schema changes Key: DRILL-4010 URL: https://issues.apache.org/jira/

[GitHub] drill pull request: DRILL-3634: Add number of partitions informati...

GitHub user mehant opened a pull request: https://github.com/apache/drill/pull/231 DRILL-3634: Add number of partitions information in the plan for hive… … scans You can merge this pull request into a Git repository by running: $ git pull https://github.com/mehant/drill DRI

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user dsbos commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43704529 --- Diff: contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java --- @@ -142,10 +148,12 @@ public void setup(OperatorConte

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

Github user amansinha100 commented on a diff in the pull request: https://github.com/apache/drill/pull/230#discussion_r43706195 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java --- @@ -163,7 +163,9 @@ private ParquetTableMetadata_v1 getParq

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

Github user amansinha100 commented on a diff in the pull request: https://github.com/apache/drill/pull/230#discussion_r43706257 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java --- @@ -166,16 +174,28 @@ protected void doOnMa

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

Github user mehant commented on a diff in the pull request: https://github.com/apache/drill/pull/230#discussion_r43706628 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java --- @@ -163,7 +163,9 @@ private ParquetTableMetadata_v1 getParquetTab

[GitHub] drill pull request: DRILL-3941: Add timing instrumentation in diff...

Github user mehant commented on a diff in the pull request: https://github.com/apache/drill/pull/230#discussion_r43706882 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java --- @@ -166,16 +174,28 @@ protected void doOnMatch(Re

[GitHub] drill pull request: DRILL-3871: Off by one error while reading bin...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] drill pull request: DRILL-3229: Implement Union type vector

Github user jacques-n commented on the pull request: https://github.com/apache/drill/pull/180#issuecomment-153219549 This seems very useful to users as an experimental feature. +1 with the default behavior as disabled. --- If your project is set up for it, you can reply to this email

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user jacques-n commented on the pull request: https://github.com/apache/drill/pull/189#issuecomment-153219893 This is a net gain. Agree with @jinfengni to track the SR replacement separately. +1 --- If your project is set up for it, you can reply to this email and have your re

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

GitHub user abhipol opened a pull request: https://github.com/apache/drill/pull/232 DRILL-951:CSV header row should be parsed You can merge this pull request into a Git repository by running: $ git pull https://github.com/abhipol/drill issues/DRILL-951 Alternatively you can r

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/189#discussion_r43714617 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/ClassGenerator.java --- @@ -218,11 +220,16 @@ public HoldingContainer addExpr(LogicalExpres

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/189#discussion_r43714802 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java --- @@ -106,19 +177,30 @@ public HoldingContainer visitFunctionCal

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user StevenMPhillips commented on a diff in the pull request: https://github.com/apache/drill/pull/189#discussion_r43715223 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/EvaluationVisitor.java --- @@ -106,19 +177,30 @@ public HoldingContainer visitFunct

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user StevenMPhillips commented on a diff in the pull request: https://github.com/apache/drill/pull/189#discussion_r43715208 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/ClassGenerator.java --- @@ -218,11 +220,16 @@ public HoldingContainer addExpr(Logical

[GitHub] drill pull request: DRILL-3912: Common subexpression elimination

Github user jinfengni commented on the pull request: https://github.com/apache/drill/pull/189#issuecomment-153236066 The revised code(adds MappingSet to ExpressionHolder) looks good to me. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43716119 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextParsingSettings.java --- @@ -57,6 +56,14 @@ public void set(TextF

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43716151 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java --- @@ -71,15 +79,14 @@ public Complia

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43716276 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java --- @@ -101,16 +108,23 @@ public void

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43717182 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java --- @@ -325,10 +325,13 @@ public AggOutcome doWork

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43717190 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java --- @@ -122,6 +122,7 @@ protected void killIncomi

[GitHub] drill pull request: DRILL-2288: Fix ScanBatch violation of IterOut...

Github user jacques-n commented on a diff in the pull request: https://github.com/apache/drill/pull/228#discussion_r43717313 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java --- @@ -93,13 +93,15 @@ public void ensureAtLeastOneField(Com

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user abhipol commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43721290 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextParsingSettings.java --- @@ -57,6 +56,14 @@ public void set(TextFor

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user abhipol commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43721297 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java --- @@ -101,16 +108,23 @@ public void se

[GitHub] drill pull request: DRILL-951:CSV header row should be parsed

Github user abhipol commented on a diff in the pull request: https://github.com/apache/drill/pull/232#discussion_r43721299 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java --- @@ -71,15 +79,14 @@ public Compliant