[GitHub] [drill] paul-rogers commented on issue #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on issue #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#issuecomment-549198995 @KazydubB, thanks much for the changes. This is a complex area. It looks like you are getting a good understanding.

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341884201 ## File path: exec/vector/src/main/java/org/apache/drill/exec/record/metadata/DictBuilder.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341884645 ## File path: exec/vector/src/main/java/org/apache/drill/exec/record/metadata/MapBuilder.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341884303 ## File path: exec/vector/src/main/java/org/apache/drill/exec/record/metadata/DictBuilder.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341884369 ## File path: exec/vector/src/main/java/org/apache/drill/exec/record/metadata/MetadataUtils.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341882388 ## File path: exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/DictWriter.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341881937 ## File path: exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/DictWriter.java

[GitHub] [drill] paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework

2019-11-03 Thread GitBox
paul-rogers commented on a change in pull request #1870: DRILL-7359: Add support for DICT type in RowSet Framework URL: https://github.com/apache/drill/pull/1870#discussion_r341883957 ## File path:

[GitHub] [drill] paul-rogers commented on issue #1889: DRILL-7436: Fix record count, vector structure issues in several operators

2019-11-03 Thread GitBox
paul-rogers commented on issue #1889: DRILL-7436: Fix record count, vector structure issues in several operators URL: https://github.com/apache/drill/pull/1889#issuecomment-549196906 Sorry for the large number of files changed: the batch count issues are complex and affect many bits of

[GitHub] [drill] paul-rogers opened a new pull request #1889: DRILL-7436: Fix record count, vector structure issues in several operators

2019-11-03 Thread GitBox
paul-rogers opened a new pull request #1889: DRILL-7436: Fix record count, vector structure issues in several operators URL: https://github.com/apache/drill/pull/1889 Adds additional vector checks to the `BatchValidator`. Enables checking for the following operators: *

[jira] [Created] (DRILL-7436) Fix record count, vector structure issues in several operators

2019-11-03 Thread Paul Rogers (Jira)
Paul Rogers created DRILL-7436: -- Summary: Fix record count, vector structure issues in several operators Key: DRILL-7436 URL: https://issues.apache.org/jira/browse/DRILL-7436 Project: Apache Drill

[GitHub] [drill] paul-rogers commented on issue #1887: DRILL-7372: MethodAnalyzer consumes too much memory

2019-11-03 Thread GitBox
paul-rogers commented on issue #1887: DRILL-7372: MethodAnalyzer consumes too much memory URL: https://github.com/apache/drill/pull/1887#issuecomment-549193078 Just a thought: we proved a couple of years back that the modern JVM does a perfectly fine job of scalar replacement. We also

[jira] [Created] (DRILL-7435) JSON reader incorrectly adds a LATE type to union vector

2019-11-03 Thread Paul Rogers (Jira)
Paul Rogers created DRILL-7435: -- Summary: JSON reader incorrectly adds a LATE type to union vector Key: DRILL-7435 URL: https://issues.apache.org/jira/browse/DRILL-7435 Project: Apache Drill

[jira] [Created] (DRILL-7434) TopNBatch constructs Union vector incorrectly

2019-11-03 Thread Paul Rogers (Jira)
Paul Rogers created DRILL-7434: -- Summary: TopNBatch constructs Union vector incorrectly Key: DRILL-7434 URL: https://issues.apache.org/jira/browse/DRILL-7434 Project: Apache Drill Issue Type:

[jira] [Created] (DRILL-7433) Allow parallel computation of metadata aggregating

2019-11-03 Thread Vova Vysotskyi (Jira)
Vova Vysotskyi created DRILL-7433: - Summary: Allow parallel computation of metadata aggregating Key: DRILL-7433 URL: https://issues.apache.org/jira/browse/DRILL-7433 Project: Apache Drill

[jira] [Created] (DRILL-7432) Add support for collecting "Drill partitions" metadata

2019-11-03 Thread Vova Vysotskyi (Jira)
Vova Vysotskyi created DRILL-7432: - Summary: Add support for collecting "Drill partitions" metadata Key: DRILL-7432 URL: https://issues.apache.org/jira/browse/DRILL-7432 Project: Apache Drill

[jira] [Created] (DRILL-7431) Add functionality to auto-collect metadata after CTAS is executed

2019-11-03 Thread Vova Vysotskyi (Jira)
Vova Vysotskyi created DRILL-7431: - Summary: Add functionality to auto-collect metadata after CTAS is executed Key: DRILL-7431 URL: https://issues.apache.org/jira/browse/DRILL-7431 Project: Apache

[jira] [Created] (DRILL-7430) Drill Metastore analyze improvements

2019-11-03 Thread Vova Vysotskyi (Jira)
Vova Vysotskyi created DRILL-7430: - Summary: Drill Metastore analyze improvements Key: DRILL-7430 URL: https://issues.apache.org/jira/browse/DRILL-7430 Project: Apache Drill Issue Type: Task

Re: Use cases for DFDL

2019-11-03 Thread Charles Givre
Hi Julian, It seems like there is a beginning of convergence of the minds here. I went to the Apache Roadshow in DC and that was where I learned about DFDL and immediately thought this was a really interesting possibility. I'd love to see if we could foster some collaboration between the