Re: Help on Performance and Functional regression testing for 2941 and 3242
Great thanks. We'll need to do a perf regression run as well. Should be neutral to positive but just want to check. -- Jacques Nadeau CTO and Co-Founder, Dremio On Sat, Jul 25, 2015 at 8:45 PM, rahul challapalli challapallira...@gmail.com wrote: The run with the last but one commit looks good! On Sat, Jul 25, 2015 at 7:49 PM, Jacques Nadeau jacq...@dremio.com wrote: Can you try again not including the last commit? (the one for 2941) On Jul 25, 2015 7:05 PM, rahul challapalli challapallira...@gmail.com wrote: Jacques, There are quite a few failures (91 execution and 52 timeouts). At a glance, I have seen NPE's, IOOB erros and ChannelClosedException messages. I will get back with more information tomorrow. - Rahul On Sat, Jul 25, 2015 at 5:54 PM, rahul challapalli challapallira...@gmail.com wrote: Jacques, I kicked off a run for functional testing. Will let you know if there are any regressions. - Rahul On Sat, Jul 25, 2015 at 5:07 PM, Jacques Nadeau jacq...@apache.org wrote: I've been working on DRILL-2941 and DRILL-3242. Both are fairly substantial changes. I've been running some regression and performance tests and am getting pretty solid results. However, given their scale, I'd like to get someone else to take a second look from functional regression and performance regression testing to make sure these are a net benefit. Does anyone have a few cycles to run my branches against some other tests? The changes are at [1]. The commit id is: 01d9f129be82a0bc2d90341ae80e7f78eb774942 Thanks, Jacques [1] https://github.com/jacques-n/drill/tree/DRILL-2941
RecordBatch.MAX_BATCH_SIZE = 65536?
In org.apache.drill.exec.record.RecordBatch, MAX_BATCH_SIZE is 65536. Why isn't that 65535? Is a batch size of zero not possible? Daniel -- Daniel Barclay MapR Technologies
Re: Review Request 36630: DRILL-3503: Make PruneScanRule pluggable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36630/ --- (Updated July 25, 2015, 6:18 a.m.) Review request for drill and Aman Sinha. Changes --- Made a few changes to the partition pruning interface. Bugs: DRILL-3503 https://issues.apache.org/jira/browse/DRILL-3503 Repository: drill-git Description --- Added an interface to abstract away partitioning scheme away from the partition pruning rule. Removed some of the redundant logic in PruneScanRule. Diffs (updated) - contrib/storage-hive/core/src/main/java/org/apache/drill/exec/planner/sql/HivePartitionDescriptor.java 8307dff exec/java-exec/src/main/java/org/apache/drill/exec/planner/DFSPartitionLocation.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/FileSystemPartitionDescriptor.java 9ad14b1 exec/java-exec/src/main/java/org/apache/drill/exec/planner/ParquetPartitionDescriptor.java 127e70a exec/java-exec/src/main/java/org/apache/drill/exec/planner/PartitionDescriptor.java 35fdae9 exec/java-exec/src/main/java/org/apache/drill/exec/planner/PartitionLocation.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushPartitionFilterIntoScan.java b83cedd exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java daa7276 exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/ParquetPruneScanRule.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java 5b5e4bc Diff: https://reviews.apache.org/r/36630/diff/ Testing --- Thanks, Mehant Baid
[jira] [Created] (DRILL-3556) Add support for using Json literals directly in SQL syntax
Jacques Nadeau created DRILL-3556:
-
Summary: Add support for using Json literals directly in SQL syntax
Key: DRILL-3556
URL: https://issues.apache.org/jira/browse/DRILL-3556
Project: Apache Drill
Issue Type: New Feature
Components: SQL Parser
Reporter: Jacques Nadeau
Assignee: Jacques Nadeau
Fix For: 1.3.0
This is the basic work to support using JSON literals. We can open other JIRAs
for things such as embedded column references, where clause use, etc.
Example:
{code}
SELECT {my:data, foo:bar} FROM (VALUES 1)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: RecordBatch.MAX_BATCH_SIZE = 65536?
You're confusing how many slots of memory with the precision of the count. We have 65536 memory slots (we use a two byte pointer to point to memory slots). We use a four byte value to maintain the number of populated slots. So the MAX_BATCH_SIZE is, in fact 65536. -- Jacques Nadeau CTO and Co-Founder, Dremio On Fri, Jul 24, 2015 at 11:00 PM, Daniel Barclay dbarc...@maprtech.com wrote: In org.apache.drill.exec.record.RecordBatch, MAX_BATCH_SIZE is 65536. Why isn't that 65535? Is a batch size of zero not possible? Daniel -- Daniel Barclay MapR Technologies
Help on Performance and Functional regression testing for 2941 and 3242
I've been working on DRILL-2941 and DRILL-3242. Both are fairly substantial changes. I've been running some regression and performance tests and am getting pretty solid results. However, given their scale, I'd like to get someone else to take a second look from functional regression and performance regression testing to make sure these are a net benefit. Does anyone have a few cycles to run my branches against some other tests? The changes are at [1]. The commit id is: 01d9f129be82a0bc2d90341ae80e7f78eb774942 Thanks, Jacques [1] https://github.com/jacques-n/drill/tree/DRILL-2941
Re: Help on Performance and Functional regression testing for 2941 and 3242
Jacques, I kicked off a run for functional testing. Will let you know if there are any regressions. - Rahul On Sat, Jul 25, 2015 at 5:07 PM, Jacques Nadeau jacq...@apache.org wrote: I've been working on DRILL-2941 and DRILL-3242. Both are fairly substantial changes. I've been running some regression and performance tests and am getting pretty solid results. However, given their scale, I'd like to get someone else to take a second look from functional regression and performance regression testing to make sure these are a net benefit. Does anyone have a few cycles to run my branches against some other tests? The changes are at [1]. The commit id is: 01d9f129be82a0bc2d90341ae80e7f78eb774942 Thanks, Jacques [1] https://github.com/jacques-n/drill/tree/DRILL-2941
Re: Help on Performance and Functional regression testing for 2941 and 3242
Jacques, There are quite a few failures (91 execution and 52 timeouts). At a glance, I have seen NPE's, IOOB erros and ChannelClosedException messages. I will get back with more information tomorrow. - Rahul On Sat, Jul 25, 2015 at 5:54 PM, rahul challapalli challapallira...@gmail.com wrote: Jacques, I kicked off a run for functional testing. Will let you know if there are any regressions. - Rahul On Sat, Jul 25, 2015 at 5:07 PM, Jacques Nadeau jacq...@apache.org wrote: I've been working on DRILL-2941 and DRILL-3242. Both are fairly substantial changes. I've been running some regression and performance tests and am getting pretty solid results. However, given their scale, I'd like to get someone else to take a second look from functional regression and performance regression testing to make sure these are a net benefit. Does anyone have a few cycles to run my branches against some other tests? The changes are at [1]. The commit id is: 01d9f129be82a0bc2d90341ae80e7f78eb774942 Thanks, Jacques [1] https://github.com/jacques-n/drill/tree/DRILL-2941
Re: Help on Performance and Functional regression testing for 2941 and 3242
Can you try again not including the last commit? (the one for 2941) On Jul 25, 2015 7:05 PM, rahul challapalli challapallira...@gmail.com wrote: Jacques, There are quite a few failures (91 execution and 52 timeouts). At a glance, I have seen NPE's, IOOB erros and ChannelClosedException messages. I will get back with more information tomorrow. - Rahul On Sat, Jul 25, 2015 at 5:54 PM, rahul challapalli challapallira...@gmail.com wrote: Jacques, I kicked off a run for functional testing. Will let you know if there are any regressions. - Rahul On Sat, Jul 25, 2015 at 5:07 PM, Jacques Nadeau jacq...@apache.org wrote: I've been working on DRILL-2941 and DRILL-3242. Both are fairly substantial changes. I've been running some regression and performance tests and am getting pretty solid results. However, given their scale, I'd like to get someone else to take a second look from functional regression and performance regression testing to make sure these are a net benefit. Does anyone have a few cycles to run my branches against some other tests? The changes are at [1]. The commit id is: 01d9f129be82a0bc2d90341ae80e7f78eb774942 Thanks, Jacques [1] https://github.com/jacques-n/drill/tree/DRILL-2941
