work.
Big-picture issues such as this tend to get lost in the 2270 open Jira tickets.
How might the project create some "theme" tickets (or Wiki pages or whatever)
to help pull the main issues out of the wealth of detail in Jira?
Thanks,
- Paul
On Monday, August 13, 2018, 11:07:39
Hi Arina,
Thanks for launching this discussion. A few minor suggestions.
The developers have done a fantastic job stabilizing and improving Drill's core
functionality. Now the opportunity is to expand the use cases for Drill so that
it gets wider adoption within the community. Drill competes
Hi All,
Recall that one of the claimed advantages of value vectors is that we could, in
theory, write operators in C/C++ to use SIMD instructions. Recall that
developers have often attempted to make vectors ever larger in order to benefit
from CPU cache lines.
Since Drill is written in Java,
ill's evolution will be to partition and
> distribute the planning process itself. That will be fun :)
>
>
> On Thu, Aug 9, 2018 at 10:51 AM, Paul Rogers
> wrote:
>
> > Hi Alex,
> >
> > Perhaps Parth can jump in here as he has deeper knowledge of Parquet.
> >
al feature?
Thanks,
Best Regards,
Alex
On Tue, Aug 7, 2018 at 10:25 PM, Paul Rogers
wrote:
> Hi Qiaoyi,
>
> In general, optimal performance occurs when a system knows the schema at
> the start and can fully optimize based on that schema. Think C or C++
> compilers com
Paul Rogers created DRILL-6677:
--
Summary: Check style unused import check conflicts with Eclipse
Key: DRILL-6677
URL: https://issues.apache.org/jira/browse/DRILL-6677
Project: Apache Drill
Paul Rogers created DRILL-6676:
--
Summary: Add Union, List and Repeated List types to Result Set
Loader
Key: DRILL-6676
URL: https://issues.apache.org/jira/browse/DRILL-6676
Project: Apache Drill
Paul Rogers created DRILL-6673:
--
Summary: Drill table functions cannot use plugin configs with a
constructor
Key: DRILL-6673
URL: https://issues.apache.org/jira/browse/DRILL-6673
Project: Apache Drill
Paul Rogers created DRILL-6672:
--
Summary: Drill table functions cannot handle "setFoo" accessors
Key: DRILL-6672
URL: https://issues.apache.org/jira/browse/DRILL-6672
Project: Ap
As far as I can tell, the SQL grammar is based on the ASCII subset of Unicode,
meaning that SQL allows only the "full stop" (.) character, but not the Unicode
variations.
Since the SQL standard defines the meaning of the full stop, Drill probably
does not want to allow variations.
It seems
is the key feature to be advantage
over Spark, is there any performance consideration behind this design except
the techniques of the dynamic codegen and vectorization computation?
Regards,
Qiaoyi
--
发件人:Paul Rogers
发送时间:2018年8月4日(星期六) 02
Paul Rogers created DRILL-6669:
--
Summary: List all sample data sets in Documentation "Sample
Datasets" section
Key: DRILL-6669
URL: https://issues.apache.org/jira/browse/DRILL-6669
Project: Ap
Paul Rogers created DRILL-6668:
--
Summary: In Web Console, highlight options that are different from
default values
Key: DRILL-6668
URL: https://issues.apache.org/jira/browse/DRILL-6668
Project: Apache
Paul Rogers created DRILL-6667:
--
Summary: Include internal data sets in Documentation Sample
Datasets
Key: DRILL-6667
URL: https://issues.apache.org/jira/browse/DRILL-6667
Project: Apache Drill
Paul Rogers created DRILL-:
--
Summary: Doc link to AOL data set is broken
Key: DRILL-
URL: https://issues.apache.org/jira/browse/DRILL-
Project: Apache Drill
Issue Type: Bug
Hi All,
Catching up on this old topic.
One of Drill's main differentiators is the ability to extend Drill with UDFs,
custom storage and format plugins, custom security plugins, etc. I wonder if
the team has considered taking a modular approach to metadata. Perhaps define a
"metadata plugin"
Hi Tim,
Here's a bit more background info from my experience.
As Sorabh said, the downstream operator has visibility to the incoming record
batch regardless of ownership.
If the downstream operator will retain the incoming vectors (buffers), then it
must take ownership of them. This is the
Hi Qiaoyi,
As you noted, Drill and Spark have similar models -- but with important
differences.
Drill is schema-on-read (also called "schema less"). In particular, this means
that Drill does not know the schema of the data until the first row (actually
"record batch") arrives at each operator.
Those I know of are in org.apache.drill.exec.expr.fn.impl.UnionFunctions:
compareType
typeOf
sqlTypeOf
drillTypeOf
modeOf
isnotnull
isnull
Thanks,
- Paul
On Thursday, July 19, 2018, 7:08:25 AM PDT, Charles Givre
wrote:
Hello all,
Is there a list somewhere of all the Drill type
Congratulations Arina!
- Paul
On Wednesday, July 18, 2018, 2:19:44 PM PDT, Aman Sinha
wrote:
Drill developers,
Time flies and it is time for a new PMC chair ! Thank you all for your
support during the past year.
I am very pleased to announce that the Drill PMC has voted to elect
Hi All,
Over the last six months I've been slowly trying to get the "result set loader"
work committed to Drill. As a recap, this was supposed to provide a uniform way
to optimally pack a record batch up to a proscribed memory limit. This
technique is particularly useful in readers which do
Paul Rogers created DRILL-6585:
--
Summary: PartitionSender clones vectors, but shares field metdata
Key: DRILL-6585
URL: https://issues.apache.org/jira/browse/DRILL-6585
Project: Apache Drill
Hi Tim,
My understanding is that Booleans are stored, as you said, as a UInt1 vector.
But, the values are 0 and (positive) 1. As you said, singletonInt will read
this value. If the value is coming back as -1, then this is a bug. I'd be
surprised if it is since I wrote lots of code that assumed
Hi Weijie,
As it turns out, vectorized processing in Drill is more aspirational than
operational at this point in time.
The code used in Drill is not actually vector-based even though the data itself
is columnar. Drill generally does row-wise operations because row-wise
operations fit the SQL
Congratulations Vitalii!
- Paul
On Tuesday, June 26, 2018, 11:12:16 AM PDT, Aman Sinha
wrote:
I am pleased to announce that Drill PMC invited Vitalii Diravka to the PMC
and he has accepted the invitation.
Congratulations Vitalii and thanks for your contributions !
-Aman
(on behalf
Paul Rogers created DRILL-6522:
--
Summary: Admission control information in docs is out of date
Key: DRILL-6522
URL: https://issues.apache.org/jira/browse/DRILL-6522
Project: Apache Drill
Issue
Paul Rogers created DRILL-6521:
--
Summary: When using Hadoop configs, allow dfs connection to be
unset
Key: DRILL-6521
URL: https://issues.apache.org/jira/browse/DRILL-6521
Project: Apache Drill
Paul Rogers created DRILL-6520:
--
Summary: Setting fs.defaultFS in core-site.xml causes startup crash
Key: DRILL-6520
URL: https://issues.apache.org/jira/browse/DRILL-6520
Project: Apache Drill
Paul Rogers created DRILL-6511:
--
Summary: Bootstrap S3 storage plugin is misleading
Key: DRILL-6511
URL: https://issues.apache.org/jira/browse/DRILL-6511
Project: Apache Drill
Issue Type: Bug
[
https://issues.apache.org/jira/browse/DRILL-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers resolved DRILL-6505.
Resolution: Invalid
Fix Version/s: 1.14.0
The problems described here were for a 1.14 build
Paul Rogers created DRILL-6510:
--
Summary: Web console, query page, focus should start in query
editor
Key: DRILL-6510
URL: https://issues.apache.org/jira/browse/DRILL-6510
Project: Apache Drill
Paul Rogers created DRILL-6509:
--
Summary: In web console, failed queries report "No result found."
Key: DRILL-6509
URL: https://issues.apache.org/jira/browse/DRILL-6509
Project: Ap
Paul Rogers created DRILL-6508:
--
Summary: Syntax errors in drillbit.sh and dependents
Key: DRILL-6508
URL: https://issues.apache.org/jira/browse/DRILL-6508
Project: Apache Drill
Issue Type
Paul Rogers created DRILL-6507:
--
Summary: Display name of storage config in web console config edit
page
Key: DRILL-6507
URL: https://issues.apache.org/jira/browse/DRILL-6507
Project: Apache Drill
Paul Rogers created DRILL-6506:
--
Summary: Drill distribution includes unnecessary jars
Key: DRILL-6506
URL: https://issues.apache.org/jira/browse/DRILL-6506
Project: Apache Drill
Issue Type
Paul Rogers created DRILL-6505:
--
Summary: Drill web console query: support back button or add "edit
query"
Key: DRILL-6505
URL: https://issues.apache.org/jira/browse/DRILL-6505
Project: Ap
Paul Rogers created DRILL-6504:
--
Summary: Typos in S3 storage doc pages
Key: DRILL-6504
URL: https://issues.apache.org/jira/browse/DRILL-6504
Project: Apache Drill
Issue Type: Bug
Affects
Congratulations! Well deserved, if just from the number of times you've
reviewed my code.
Thanks,
- Paul
On Friday, June 15, 2018, 9:36:44 AM PDT, Aman Sinha
wrote:
The Project Management Committee (PMC) for Apache Drill has invited Padma
Penumarthy to become a committer, and we
Hi All,
As it turns out, this topic has been discussed, in depth, previously. Can't
recall if it was on this list, or in a JIRA.
We face a number of constraints:
* As was noted, for some data sources, the data source itself has case
insensitive names. (Windows file systems, RDBMSs, etc.)
* In
Paul Rogers created DRILL-6485:
--
Summary: Typo in drill-env.sh
Key: DRILL-6485
URL: https://issues.apache.org/jira/browse/DRILL-6485
Project: Apache Drill
Issue Type: Bug
Affects Versions
+1
I just learned to ignore the ancient PRs; they were not adding much value.
If a PR looks like it could be resurrected, we might consider 1) assigning a
committer to help push it over the line, and 2) check back with submitter to
see if they can update it.
We tried the above a few times
Hi Weijie,
Seeing the discussion about the details of JCodeModel suggests you may be
trying to debug your generated code at the level of the code generator.
Some time ago we added the ability to step through the generated code. Look for
the following line in the generator code:
//
Congrats, Tim! Thanks much for the many PRs to fix unglamorous, but highly
important parts of Drill such as builds, tests and so on. They will benefit all
Drill developers no matter which part of the project they work on.
Thanks,
- Paul
On Friday, May 25, 2018, 11:58:32 AM PDT, Aman
s
> used anywhere.
>
> Thanks,
> Tim
>
>
> From: Paul Rogers <par0...@yahoo.com.INVALID>
> Sent: Thursday, May 24, 2018 7:08:28 PM
> To: dev@drill.apache.org
> Subject: Re: [Question] Using loggers in tests
>
> LogFixture? As illustrated in ExampleTest?
>
>
LogFixture? As illustrated in ExampleTest?
This fixture lets you turn on all or selected loggers for the duration of a
single test. I used it all the time when debugging. Works great.
It works when turning loggers on when the default is that they are off. For
whatever reason, I could not get it
Paul Rogers created DRILL-6383:
--
Summary: View column types, modes are plan-time guesses, not
actual types
Key: DRILL-6383
URL: https://issues.apache.org/jira/browse/DRILL-6383
Project: Apache Drill
Paul Rogers created DRILL-6382:
--
Summary: Views should "close" over options in effect at view
creation time
Key: DRILL-6382
URL: https://issues.apache.org/jira/browse/DRILL-6382
Project: Ap
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1242
@kfaraaz, the logic for all types is the same. But, still added
`BIT`/`BOOLEAN`, `DATE`, `TIME`, `TIMESTAMP`, `INTERVAL`, `VARBINARY`, and
`DECIMAL`.
Found several bugs, but none
Paul Rogers created DRILL-6378:
--
Summary: MajorType passed into UDF with FieldReader has unset
prec, scale for DECIMAL
Key: DRILL-6378
URL: https://issues.apache.org/jira/browse/DRILL-6378
Project
Paul Rogers created DRILL-6377:
--
Summary: typeof() does not return DECIMAL scale, precision
Key: DRILL-6377
URL: https://issues.apache.org/jira/browse/DRILL-6377
Project: Apache Drill
Issue
Paul Rogers created DRILL-6376:
--
Summary: Doc: Return type of ROUND(x, y), TRUNC(x, y), TO_NUMBER
is wrong
Key: DRILL-6376
URL: https://issues.apache.org/jira/browse/DRILL-6376
Project: Apache Drill
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1244
@ppadma, here is the next step in our long saga to check in the batch
sizing work. Please give it a review at your convenience. Thanks!
---
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/1244
DRILL-6373: Refactor Result Set Loader for Union, List support
This PR builds on the previous refactoring of the column accessors to
prepare for Union, (non-repeated) List and Repeated List
Paul Rogers created DRILL-6373:
--
Summary: Refactor the Result Set Loader to prepare for Union, List
support
Key: DRILL-6373
URL: https://issues.apache.org/jira/browse/DRILL-6373
Project: Apache Drill
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1239
Just to be clear, I have no objection Drill enforcing its own cgroup limits.
My point is rather that CPU limits must be integrated with YARN, via the
DoY config file, so that the user
Congrats! Thanks much for your contributions!
- Paul
On Monday, April 30, 2018, 9:01:07 AM PDT, Aman Sinha
wrote:
The Project Management Committee (PMC) for Apache Drill has invited Sorabh
Hamirwasia to become a committer, and we are pleased to announce that
I missed this as a discussion since it had the title of a GitHub discussion.
Comments below.
On Friday, April 27, 2018, 5:42:37 PM PDT, salim achouche
wrote:
> Another point, I don't see a functional benefit from avoiding a change of
ownership for pass-through
Hi Vlad,
More responses.
> The same approach [as for internal operators] applies to senders and
> receivers. Senders gets batches
from the upstream operators taking ownership of those batches and send
data to receivers.
Senders receive data from an "upstream" operator, then serialize over the
Specific answers based on my understanding.
> I did not mean that a pass-through operator should not take the
ownership of a batch it processes. My question was whether they do so
and if they do, when and how.
Yes, operators do take ownership, somewhere in the process of calling next() on
It can certainly be improved (as we are doing with the batch
handling revisions.) But, we must consider the entire system when considering
changes to any one part of the system. It is a complex topic; it is great that
we have someone with your experience exploring our options.
Thanks,
- Paul
[
Paul Rogers created DRILL-6370:
--
Summary: Mod operator % is documented, but not available
Key: DRILL-6370
URL: https://issues.apache.org/jira/browse/DRILL-6370
Project: Apache Drill
Issue Type
mation about how drill
interprets the data was really difficult to get around. Just me .02.
> On Apr 28, 2018, at 18:05, Paul Rogers <par0...@yahoo.com.INVALID> wrote:
>
> Hi Rob,
>
> Thanks for the suggestion. While this works for Hive (as you showed), it doe
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/1242
DRILL-6361: Revised typeOf() function versions
Drill provides the `typeof()` function to return the type of a column.
However, this function has two key limitations:
1. It returns NULL
Paul Rogers created DRILL-6369:
--
Summary: typeof() fails for constants
Key: DRILL-6369
URL: https://issues.apache.org/jira/browse/DRILL-6369
Project: Apache Drill
Issue Type: Bug
Affects
Paul Rogers created DRILL-6368:
--
Summary: "E" operator does not behave as documented "exponent
operator"
Key: DRILL-6368
URL: https://issues.apache.org/jira/browse/DRILL-6368
P
Paul Rogers created DRILL-6367:
--
Summary: Missing line breaks in Drill operator list
Key: DRILL-6367
URL: https://issues.apache.org/jira/browse/DRILL-6367
Project: Apache Drill
Issue Type: Bug
Paul Rogers created DRILL-6366:
--
Summary: Inconsistency between typeof(), CAST types
Key: DRILL-6366
URL: https://issues.apache.org/jira/browse/DRILL-6366
Project: Apache Drill
Issue Type: Bug
.org>
wrote:
> Hey Paul,
>
> You could use the typeof() function for this purpose. It takes a single
> parameter - the column name.
>
> For example:
> > select typeof(c_current_cdemo_sk) from customer limit 1;
> +-+
> | EXPR$0 |
> +-+
> | BIGINT
Paul Rogers created DRILL-6362:
--
Summary: typeof() lies about types
Key: DRILL-6362
URL: https://issues.apache.org/jira/browse/DRILL-6362
Project: Apache Drill
Issue Type: Improvement
Paul Rogers created DRILL-6361:
--
Summary: Provide a dataTypeOf() or modeOf() function
Key: DRILL-6361
URL: https://issues.apache.org/jira/browse/DRILL-6361
Project: Apache Drill
Issue Type
Paul Rogers created DRILL-6360:
--
Summary: Document the typeof() function
Key: DRILL-6360
URL: https://issues.apache.org/jira/browse/DRILL-6360
Project: Apache Drill
Issue Type: Improvement
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1228#discussion_r184590500
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -536,6 +556,11 @@ public ColumnSize getColumn(String
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1184
@parthchandra, the point about the birthday is that is is one of those
dates that is implied relative to where you are. You celebrate it the same day
regardless of where you are in the world
| **nationkey_type **
|*
*+++--+-+*
*| *ALGERIA * | *VARCHAR * | *0 * | *INT * |*
*| *ARGENTINA * | *VARCHAR * | *1 * | *INT * |*
*+++--+-+*
On Wed, Apr 25, 2018 at 9:22 PM, Paul Rogers <par0...@yahoo.com
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1184
One additional note. We noted that JDBC does not support the idea of a
nested tuple (a Drill "map".) JDBC does support columns that return a Java
object. To bridge the gap, Drill ret
Paul Rogers created DRILL-6359:
--
Summary: All-text mode in JSON still reads missing column as
Nullable Int
Key: DRILL-6359
URL: https://issues.apache.org/jira/browse/DRILL-6359
Project: Apache Drill
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1184
Sorry, coming late. There seem to be two problems. The original "nested
column" issue is an artifact of the JDBC driver. In Drill, a Map (the thing
that contains your nested colum
Hi All,
Anyone know if there is a non-code way to display the data types of columns
returned from a Drill query? Sqlline appears to only show the column names and
values. The same is true of the Drill web console.
The EXPLAIN PLAN FOR ... command shows the query plan, but not type (which are
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1228#discussion_r184264961
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -536,6 +556,11 @@ public ColumnSize getColumn(String
Paul Rogers created DRILL-6358:
--
Summary: Null value returned from WHERE a IS NOT NULL query
Key: DRILL-6358
URL: https://issues.apache.org/jira/browse/DRILL-6358
Project: Apache Drill
Issue
Paul Rogers created DRILL-6357:
--
Summary: Unexpected column "**" when reading a JSON file
Key: DRILL-6357
URL: https://issues.apache.org/jira/browse/DRILL-6357
Project: Apache Drill
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1239
One other thing to highlight from an earlier comment. CPU is something that
the user specifies in the DoY config file. That information is passed to YARN
in container requests. This feature asks
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1239
@kkhatua, putting on my Apache hat... Apache Drill is an Apache project
that must work with other Apache projects such as Apache YARN. The Apache Drill
DoY support is designed to work well
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1239#discussion_r184169114
--- Diff: distribution/src/resources/yarn-drillbit.sh ---
@@ -175,4 +209,11 @@ fi
echo "`date` Starting drillbit on `hostname` under YARN, lo
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1239#discussion_r184144439
--- Diff: distribution/src/resources/yarn-drillbit.sh ---
@@ -175,4 +209,11 @@ fi
echo "`date` Starting drillbit on `hostname` under YARN, lo
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1239
There may be some misunderstanding of how DoY works. The only info that
users can pass to DoY is that which is in the DoY config file. We should add
arguments to that file which will be passed
Paul Rogers created DRILL-6354:
--
Summary: Errors from drill-config.sh on Mac
Key: DRILL-6354
URL: https://issues.apache.org/jira/browse/DRILL-6354
Project: Apache Drill
Issue Type: Bug
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1228#discussion_r183264768
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -277,18 +286,29 @@ public boolean isRepeatedList
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1228#discussion_r183264235
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -50,7 +50,7 @@
public class RecordBatchSizer
Github user paul-rogers closed the pull request at:
https://github.com/apache/drill/pull/1206
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1218
@ppadma , this PR contains a subset of the changes from the previous big
PR. Mostly refactoring. Can you take a look? Thanks.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1206
Replacing this PR with a set of smaller ones.
---
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/1218
DRILL-6335: Refactor row set abstractions to prepare for unions
Refactors the column accessors to prepare for adding unions, lists and
repeated lists.
This is a subset of a PR done
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/1206
Going to close this one and try to split this into smaller chunks.
---
Paul Rogers created DRILL-6335:
--
Summary: Refactor row set abstractions to prepare for unions
Key: DRILL-6335
URL: https://issues.apache.org/jira/browse/DRILL-6335
Project: Apache Drill
Issue
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/1213
Minor code cleanup
Pulled the remaining code cleanup items out of the Result Set work into
this simple PR.
You can merge this pull request into a Git repository by running:
$ git pull
Paul Rogers created DRILL-6334:
--
Summary: Code cleanup
Key: DRILL-6334
URL: https://issues.apache.org/jira/browse/DRILL-6334
Project: Apache Drill
Issue Type: Improvement
Reporter
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1210#discussion_r181799954
--- Diff: distribution/src/resources/runbit ---
@@ -65,6 +65,47 @@ drill_rotate_log ()
fi
}
+args=( $@ )
+RBARGS
com/apache/drill/blob/master/exec/java-
> exec/src/main/java/org/apache/drill/exec/expr/fn/impl/Mappify.java#L55
> [3] https://github.com/apache/drill/blob/master/exec/java-
> exec/src/main/java/org/apache/drill/exec/expr/fn/ValueReference.java#L76
>
> -Aman
>
> On Fri, Apr 13
, only fixed-width types can be updated in random
order. DRILL-6087 describes this issue.
Thanks,
- Paul
[1] https://github.com/paul-rogers/drill/wiki/UDFs-Background-Information
On Wednesday, April 11, 2018, 4:09:47 PM PDT, Aman Sinha
<amansi...@apache.org> wrote:
Here's
401 - 500 of 3384 matches
Mail list logo