[GitHub] drill pull request #895: DRILL-5704: Improve error message on client side wh...

[sohami]

GitHub user sohami opened a pull request:

https://github.com/apache/drill/pull/895

DRILL-5704: Improve error message on client side when queries fail wi…

…th "Failed to create schema tree." when Impersonation is enabled and 
logins are anonymous

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sohami/drill DRILL-5704

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/895.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #895


commit 7950826cb1ae28c33007aac22834ca8f3b33846a
Author: Sorabh Hamirwasia 
Date:   2017-08-04T01:23:02Z

DRILL-5704: Improve error message on client side when queries fail with 
"Failed to create schema tree." when Impersonation is enabled and logins are 
anonymous




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5704) Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous

[Sorabh Hamirwasia (JIRA)]

Sorabh Hamirwasia created DRILL-5704:


 Summary: Improve error message on client side when queries fail 
with "Failed to create schema tree." when Impersonation is enabled and logins 
are anonymous
 Key: DRILL-5704
 URL: https://issues.apache.org/jira/browse/DRILL-5704
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Sorabh Hamirwasia


When username is not specified then Drill set's the session user as anonymous 
if impersonation is enabled. During query execution Drill tries to build schema 
tree and as part of that it validates if the user has access to the workspace 
or not by using FileClient Api liststatus which verifies the user from the OS 
user. Since impersonation is only enabled here without authentication and we 
don't specify any user in connection string, Drill will use default user which 
is "anonymous" and pass that to check workspace permission which will fail as 
node doesn't have any valid user with that name.


{code:java}
Caused by: java.io.IOException: Error getting user info for current user, 
anonymous
   ..
   ..
at 
org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:157)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.(FileSystemSchemaFactory.java:78)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
 ~[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
at 
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
 [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
... 10 common frames omitted
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Draft board report

[Parth Chandra]

+1

On Thu, Aug 3, 2017 at 3:03 PM, Julian Hyde  wrote:

> +1
>
> > On Aug 3, 2017, at 2:58 PM, Aman Sinha  wrote:
> >
> > Drill developers,
> > The quarterly board report for Drill is due in the next week or so.  Pls
> > take a look at the draft report below send me your comments if any.  I
> > would like to send it by tomorrow since I will be on vacation next week.
> >
> > Thank you,
> > Aman
> >
> >  Apache Drill Board Report ==
> >
> > ## Description:
> > - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
> > Storage
> >
> > ## Issues:
> > - There are no issues requiring board attention at this time
> >
> > ## Activity:
> > - Since the last board report, Drill has released version 1.11.0. The
> > following new features/enhancements
> > were added in addition to many other bug fixes:
> >- Cryptography-related functions.
> >- Spill to disk for the hash aggregate operator.
> >- Format plugin support for PCAP files.
> >- Ability to change the HDFS block Size for Parquet files.
> >- Ability to store query profiles in memory.
> >- Configurable CTAS directory and file permissions option.
> >- Support for network encryption.
> >- Relative paths stored in the metadata file.
> >- Support for ANSI_QUOTES.
> >
> > ## Health report:
> >
> > - The project is healthy. Development activity as reflected in the pull
> > requests and JIRAs
> >   is good.  Activity on the dev and user mailing lists has shown a slight
> > increase compared
> >   to previous period. Three new committers and one new PMC member were
> > added in the last period.
> >
> > ## PMC changes:
> >
> > - Currently 18 PMC members.
> > - Arina Ielchiieva was added to the PMC on Tue Aug 01 2017
> >
> > ## Committer base changes:
> >
> > - Currently 36 committers.
> > - New commmitters:
> >- Charles Givre was added as a committer on Mon Jun 12 2017
> >- Laurent Goujon was added as a committer on Thu Jun 08 2017
> >- Paul Rogers was added as a committer on Fri May 19 2017
> >
> > ## Releases:
> >
> > - 1.11.0 was released on Thu Jul 27 2017
> >
> > ## Mailing list activity:
> >
> > - dev@drill.apache.org:
> >- 444 subscribers (up 8 in the last 3 months):
> >- 1928 emails sent to list (1918 in previous quarter)
> >
> > - iss...@drill.apache.org:
> >- 18 subscribers (down -2 in the last 3 months):
> >- 2748 emails sent to list (2964 in previous quarter)
> >
> > - u...@drill.apache.org:
> >- 609 subscribers (up 23 in the last 3 months):
> >- 454 emails sent to list (362 in previous quarter)
> >
> >
> > ## JIRA activity:
> >
> > - 234 JIRA tickets created in the last 3 months
> > - 101 JIRA tickets closed/resolved in the last 3 months
>
>

[GitHub] drill pull request #894: DRILL-5701: drill.connections.rpc.

[sohami]

GitHub user sohami opened a pull request:

https://github.com/apache/drill/pull/894

DRILL-5701: drill.connections.rpc.. metric not behaving correctly

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sohami/drill DRILL-5701

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/894.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #894


commit e4d84d1f555a8abb383c72bf947c6787ecea055d
Author: Sorabh Hamirwasia 
Date:   2017-08-02T21:25:21Z

DRILL-5701: 
drill.connections.rpc.. metric not 
behaving correctly




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[DISCUSS] Draft board report

[Aman Sinha]

Drill developers,
The quarterly board report for Drill is due in the next week or so.  Pls
take a look at the draft report below send me your comments if any.  I
would like to send it by tomorrow since I will be on vacation next week.

Thank you,
Aman

 Apache Drill Board Report ==

## Description:
 - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
Storage

## Issues:
 - There are no issues requiring board attention at this time

## Activity:
 - Since the last board report, Drill has released version 1.11.0. The
following new features/enhancements
 were added in addition to many other bug fixes:
- Cryptography-related functions.
- Spill to disk for the hash aggregate operator.
- Format plugin support for PCAP files.
- Ability to change the HDFS block Size for Parquet files.
- Ability to store query profiles in memory.
- Configurable CTAS directory and file permissions option.
- Support for network encryption.
- Relative paths stored in the metadata file.
- Support for ANSI_QUOTES.

## Health report:

 - The project is healthy. Development activity as reflected in the pull
requests and JIRAs
   is good.  Activity on the dev and user mailing lists has shown a slight
increase compared
   to previous period. Three new committers and one new PMC member were
added in the last period.

## PMC changes:

 - Currently 18 PMC members.
 - Arina Ielchiieva was added to the PMC on Tue Aug 01 2017

## Committer base changes:

 - Currently 36 committers.
 - New commmitters:
- Charles Givre was added as a committer on Mon Jun 12 2017
- Laurent Goujon was added as a committer on Thu Jun 08 2017
- Paul Rogers was added as a committer on Fri May 19 2017

## Releases:

 - 1.11.0 was released on Thu Jul 27 2017

## Mailing list activity:

 - dev@drill.apache.org:
- 444 subscribers (up 8 in the last 3 months):
- 1928 emails sent to list (1918 in previous quarter)

 - iss...@drill.apache.org:
- 18 subscribers (down -2 in the last 3 months):
- 2748 emails sent to list (2964 in previous quarter)

 - u...@drill.apache.org:
- 609 subscribers (up 23 in the last 3 months):
- 454 emails sent to list (362 in previous quarter)


## JIRA activity:

 - 234 JIRA tickets created in the last 3 months
 - 101 JIRA tickets closed/resolved in the last 3 months

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[sindhurirayavaram]

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/891
  
@arina! I am planning to keep all the files in the local resources file! I 
have mentioned the reasons above. Yes, onload should be used for IE.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
@sindhurirayavaram, one more question about browser compatibility:
will this work in Chrome, Mozilla, Safari etc?
For example, quick search shows that there are can be issues with different 
browsers when using `link onerror` [1].

[1] 
https://stackoverflow.com/questions/30171270/link-onerror-do-not-work-in-ie


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #893: DRILL-5703 Add Syntax Highlighting & Autocompletion...

[cgivre]

GitHub user cgivre opened a pull request:

https://github.com/apache/drill/pull/893

DRILL-5703 Add Syntax Highlighting & Autocompletion to Query Form

This PR (https://issues.apache.org/jira/browse/DRILL-5703) adds syntax 
highlighting and autocompletion to the query form using the Ace code editor.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cgivre/drill master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/893.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #893


commit a5accbd0d9d2defb6de488680b6c650021ae9931
Author: cgivre 
Date:   2017-07-03T03:13:02Z

Added Crypto Functions

commit 35f8002cf95a9376da3bcf930e9b0de585e6c952
Author: cgivre 
Date:   2017-07-03T03:18:39Z

Removed extraneous code

commit 0d71fb56c33e281788f4a8d8b97534132819d5b4
Author: cgivre 
Date:   2017-07-04T02:39:22Z

Merge branch 'master' of https://github.com/apache/drill

commit 58532a552d48a3a8272c6975406e5e948433666f
Author: cgivre 
Date:   2017-07-04T03:41:49Z

Fixed various errors and coding convention, removed helper class.

commit a33e76028508a72018d1034a7268cd271980e5ff
Author: cgivre 
Date:   2017-07-04T03:58:32Z

Fixed spacing and indents

commit ac34c0ecdb5639a2b9dc0d739d495afca78f
Author: cgivre 
Date:   2017-07-04T04:00:28Z

Fixed class names

commit 28a38c57b7f9619046bbd2715cd983b779b29490
Author: cgivre 
Date:   2017-07-04T15:11:09Z

Added Javadoc, MD2, and fixed style errors

commit d19700250e90560136235f6bb182c8dcfcc5974f
Author: cgivre 
Date:   2017-07-04T19:49:47Z

Removed unused workspace variables

commit 386604e257ab73323de86a39ab33c593b84a040f
Author: cgivre 
Date:   2017-07-14T04:17:58Z

Merge branch 'master' of https://github.com/apache/drill

commit 3e449bbbe176956c489a829383c1b3c62632e432
Author: cgivre 
Date:   2017-07-14T04:54:57Z

Fixed misc errata

commit 5d9a8bb12cee671f2372f21f06ae3ef5954b
Author: cgivre 
Date:   2017-08-03T19:50:05Z

Added code editor for query page

commit 4dfb6254903796b3c122fffc2e5193dcc3558dd2
Author: cgivre 
Date:   2017-08-03T20:09:44Z

Removed extra javascript files for code editor

commit f8bf920dc7ba704413e812ffe650e6d46b985201
Author: cgivre 
Date:   2017-08-03T20:12:46Z

Merge branch 'master' of https://github.com/apache/drill




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-5703) Add Syntax Highlighting & Autocompletion to Query Form

[Charles Givre (JIRA)]

Charles Givre created DRILL-5703:


 Summary: Add Syntax Highlighting & Autocompletion to Query Form
 Key: DRILL-5703
 URL: https://issues.apache.org/jira/browse/DRILL-5703
 Project: Apache Drill
  Issue Type: Improvement
  Components: Web Server
Affects Versions: 1.11.0
Reporter: Charles Givre


The UI could really benefit from having syntax highlighting and autocompletion 
in the query window as well as the form to update storage plugins.  This PR 
adds that capability to the query form using the Ace code editor 
(https://ace.c9.io). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New PMC member: Arina Ielchiieva

[yuliya Feldman]

Congrats Arina!!!
Very glad to see this happening.
Yuliya

  From: Arina Yelchiyeva 
 To: dev@drill.apache.org 
 Sent: Thursday, August 3, 2017 2:53 AM
 Subject: Re: [ANNOUNCE] New PMC member: Arina Ielchiieva
   
Thank all you!

Kind regards
Arina

On Thu, Aug 3, 2017 at 5:58 AM, Sudheesh Katkam  wrote:

> Congratulations and thank you, Arina.
>
> On Wed, Aug 2, 2017 at 1:38 PM, Paul Rogers  wrote:
>
> > The success of the Drill 1.11 release proves this is a well-deserved
> move.
> > Congratulations!
> >
> > - Paul
> >
> > > On Aug 2, 2017, at 11:23 AM, Aman Sinha  wrote:
> > >
> > > I am pleased to announce that Drill PMC invited Arina Ielchiieva to the
> > PMC
> > > and she has accepted the invitation.
> > >
> > > Congratulations Arina and thanks for your contributions !
> > >
> > > -Aman
> > > (on behalf of Drill PMC)
> >
> >
>


   

[GitHub] drill pull request #874: DRILL-5663: Drillbit fails to start when only keyst...

[sindhurirayavaram]

Github user sindhurirayavaram commented on a diff in the pull request:

https://github.com/apache/drill/pull/874#discussion_r131238139
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
@@ -122,10 +122,10 @@
   String HTTP_SESSION_MEMORY_RESERVATION = 
"drill.exec.http.session.memory.reservation";
   String HTTP_SESSION_MEMORY_MAXIMUM = 
"drill.exec.http.session.memory.maximum";
   String HTTP_SESSION_MAX_IDLE_SECS = 
"drill.exec.http.session_max_idle_secs";
-  String HTTP_KEYSTORE_PATH = "javax.net.ssl.keyStore";
--- End diff --

Using the _javax_ property name for Drill property might be confusing for 
users, since changing that value inside Drill configuration will not reflect on 
the actual System property. So to avoid that confusion this property was 
renamed taking example from 
[Hadoop](https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html)
 and 
[IBM](https://www.ibm.com/support/knowledgecenter/en/SSZH4A_5.0.6/com.ibm.worklight.help.doc/admin/r_ssl_certificate_keystore_setup.html).
 Though Hadoop provides separate parameter for client/server whereas Drill has 
only server side config, these properties can be renamed to include _.server_ 
if needed. But making this name change will not affect Drill users already 
using the older property since backward compatibility is provided here. It will 
work seamlessly for them. All the below scenarios are tested. 
1) Setting javax.ssl.* properties in drill-override.conf
2) Setting javax.ssl.* properties as system options with -D option.
3) Setting ssl.* properties in drill-override.conf

So if a user provide the property values only as system property, then both 
Drill and JSSE will still be consuming it and work as expected.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #877: DRILL-5660: Drill 1.10 queries fail due to Parquet ...

[vdiravka]

Github user vdiravka commented on a diff in the pull request:

https://github.com/apache/drill/pull/877#discussion_r131164453
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/MetadataVersions.java
 ---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import com.google.common.collect.ImmutableSortedSet;
+import org.apache.drill.common.exceptions.DrillRuntimeException;
+
+import java.util.NavigableSet;
+
+/**
+ * Supported metadata versions.
+ * 
+ * String metadata version consists of the following characters:
+ * optional "v" letter,
+ * major metadata version (any number of digits),
+ * optional "." delimiter (used if minor metadata version is 
specified),
+ * minor metadata version (one digit number)
+ * 
+ * Note: keep them synchronized with {@link 
Metadata.ParquetTableMetadataBase} versions
+ */
+public class MetadataVersions {
+  /**
+   * Version 1: Introduces parquet file metadata caching.
+   * See DRILL-2743
+   */
+  public static final String V1 = "v1";
+  /**
+   * Version 2: Metadata cache file size is reduced.
+   * See DRILL-4053
+   */
+  public static final String V2 = "v2";
+  /**
+   * Version 3: Difference between v3 and v2 : min/max, type_length, 
precision, scale, repetitionLevel, definitionLevel.
+   * Filter pushdown for Parquet is implemented. 
+   * See DRILL-1950
+   */
+  public static final String V3 = "v3";
+  /**
+   * Version 3.1: Absolute paths of files and directories are replaced 
with relative ones. Metadata version value
+   * doesn't contain `v` letter
+   * See DRILL-3867, DRILL-5660
+   */
+  public static final String V3_1 = "3.1";
+
+  /**
+   * Helper method to parse string metadata version into float.
+   *
+   * @param stringVersion text metadata version
+   * @return parsed Float metadata version
+   */
+  public static Float parseStringMetadataVersion(String stringVersion) {
+try {
+  if (stringVersion.contains(".") && 
stringVersion.split("\\.")[1].length() != 1) {
+throw new DrillRuntimeException("Minor metadata version shouldn't 
be greater than 9 or contain more than one digit");
+  }
+  return stringVersion.charAt(0) == 'v' ? 
Float.valueOf(stringVersion.substring(1)) : Float.valueOf(stringVersion);
+} catch (Exception e) {
+  throw new DrillRuntimeException(String.format("Could not parse 
metadata version '%s'", stringVersion), e);
+}
+  }
+
+  /**
+   * All historical versions of the Drill metadata cache files
+   */
+  public static final NavigableSet SUPPORTED_VERSIONS = 
ImmutableSortedSet.of(
--- End diff --

Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Question on building drill & developing a new storage plugin

[Muhammad Gelbana]

This wiki  should be very
helpful for understanding plugins development process and the unit testing
architecture among many other things.

I'm facing difficulties with running the test cases successfully too using
a fresh clone: https://issues.apache.org/jira/browse/DRILL-5606
If you get over this, please share the knowledge.

Thanks,
Gelbana

On Thu, Aug 3, 2017 at 8:57 PM, Paul Rogers  wrote:

> Hi Bob,
>
> Thanks for tackling the new plugin!
>
> Drill is a huge piece of software build, in part, by combining a wide
> variety of libraries and packages. As it turns out, the package authors
> have used an even wider variety of loggers, Guava versions and so on.
>
> Maven provides “dependency management” to control the chaos. Basically,
> the pom.xml file first adds library X that, say, uses log4j or commons
> logging. Then, elsewhere in the pom file, dependency management removes
> these dependencies. Maven provides (very hard to read) documentation. The
> “dependency tree” is your friend here.
>
> From your e-mail, it is unclear if these errors appeared with a “stock”
> Drill build, or after you started adding the libraries for WARC. If in
> “stock” Drill, then somehow something slipped through the cracks & we need
> to fix it. If after adding new jars, then you have to tackle the dependency
> management aspect.
>
> Note that, depending on where you add the code, you may also find yourself
> fighting with the “JDBC All” project. That project imports all of Drill
> into the JDBC package, then removes a bunch of stuff to keep the file size
> under a defined maximum. If your code pushes JDBC over the limit, you’ll
> have to find things to throw overboard to get below the limit again. (This
> is why storage plugins should, ideally, be in the contrib directory, not in
> exec.)
>
> At present, Drill provides no “SDK”: no way to build plugins except as
> part of the Drill source tree. At one point I was able to get this working
> for some aspects. Storage plugins, however, do require changes in core
> Drill code (for the bootstrap file, for registering the reader, etc.) That
> is the bit that must be fixed to allow a true external plugin development.
> This is a shame, and should be fixed. If you have time to do so, we’re
> always looking for contributions!
>
> Thanks,
>
> - Paul
>
>
> > On Aug 3, 2017, at 10:29 AM, Bob Rudis  wrote:
> >
> > Hey folks,
> >
> > First:
> >
> > Inspired by the PCAP support in 1.11.0 I started down the path of
> > cloning drill and just doing a test build before I started looking at
> > 2 of the issues I posted for the PCAP storage format and also working
> > to incorporate a similar WARC plugin (via the jwat jars).
> >
> > `mvn package -DskipTests` works fine but `mvn install -DskipTests`
> generates:
> >
> > Found Banned Dependency: log4j:log4j:jar:1.2.17
> > Found Banned Dependency: commons-logging:commons-logging:jar:1.1.1
> >
> > warnings which causes:
> >
> > Apache Drill Root POM .. FAILURE
> >
> > and (hence) the overall build to fail.
> >
> > I'm hoping that I'm just missing something obvious after some failed
> googling.
> >
> >
> > Second:
> >
> > Is there a way to develop the WARC format plugin outside of
> > Drill-source-proper vs having to build into it. If so, is there a
> > small, example GH repo someone cld point me to (that also has the "how
> > to get it into Drill" part). From a scan of the source, it looks like
> > the formats are embedded into a few source files in the
> > Drill-source-proper. If there's no way to develop one outside of it,
> > that's not a problem, I just figured it'd be easier to do it outside
> > of modifying Drill source first.
> >
> > thx,
> >
> > -Bob
>
>

Re: Question on building drill & developing a new storage plugin

[Paul Rogers]

Hi Bob,

Thanks for tackling the new plugin!

Drill is a huge piece of software build, in part, by combining a wide variety 
of libraries and packages. As it turns out, the package authors have used an 
even wider variety of loggers, Guava versions and so on.

Maven provides “dependency management” to control the chaos. Basically, the 
pom.xml file first adds library X that, say, uses log4j or commons logging. 
Then, elsewhere in the pom file, dependency management removes these 
dependencies. Maven provides (very hard to read) documentation. The “dependency 
tree” is your friend here.

From your e-mail, it is unclear if these errors appeared with a “stock” Drill 
build, or after you started adding the libraries for WARC. If in “stock” Drill, 
then somehow something slipped through the cracks & we need to fix it. If after 
adding new jars, then you have to tackle the dependency management aspect.

Note that, depending on where you add the code, you may also find yourself 
fighting with the “JDBC All” project. That project imports all of Drill into 
the JDBC package, then removes a bunch of stuff to keep the file size under a 
defined maximum. If your code pushes JDBC over the limit, you’ll have to find 
things to throw overboard to get below the limit again. (This is why storage 
plugins should, ideally, be in the contrib directory, not in exec.)

At present, Drill provides no “SDK”: no way to build plugins except as part of 
the Drill source tree. At one point I was able to get this working for some 
aspects. Storage plugins, however, do require changes in core Drill code (for 
the bootstrap file, for registering the reader, etc.) That is the bit that must 
be fixed to allow a true external plugin development. This is a shame, and 
should be fixed. If you have time to do so, we’re always looking for 
contributions!

Thanks,

- Paul


> On Aug 3, 2017, at 10:29 AM, Bob Rudis  wrote:
> 
> Hey folks,
> 
> First:
> 
> Inspired by the PCAP support in 1.11.0 I started down the path of
> cloning drill and just doing a test build before I started looking at
> 2 of the issues I posted for the PCAP storage format and also working
> to incorporate a similar WARC plugin (via the jwat jars).
> 
> `mvn package -DskipTests` works fine but `mvn install -DskipTests` generates:
> 
> Found Banned Dependency: log4j:log4j:jar:1.2.17
> Found Banned Dependency: commons-logging:commons-logging:jar:1.1.1
> 
> warnings which causes:
> 
> Apache Drill Root POM .. FAILURE
> 
> and (hence) the overall build to fail.
> 
> I'm hoping that I'm just missing something obvious after some failed googling.
> 
> 
> Second:
> 
> Is there a way to develop the WARC format plugin outside of
> Drill-source-proper vs having to build into it. If so, is there a
> small, example GH repo someone cld point me to (that also has the "how
> to get it into Drill" part). From a scan of the source, it looks like
> the formats are embedded into a few source files in the
> Drill-source-proper. If there's no way to develop one outside of it,
> that's not a problem, I just figured it'd be easier to do it outside
> of modifying Drill source first.
> 
> thx,
> 
> -Bob

Re: Question on building drill & developing a new storage plugin

[Charles Givre]

Hi Bob, 
You can actually build your storage plugin outside of drill, package it using 
maven, then copy the resultant JAR files into the /jars/3rdparty folder. 
Here are some examples…

https://github.com/bizreach/drill-ltsv-plugin 

https://github.com/cgivre/drill-logfile-plugin 


IMHO this is much easier than trying to rebuild Drill every time you need to 
rebuild. 

Best,
— C


> On Aug 3, 2017, at 13:29, Bob Rudis  wrote:
> 
> Hey folks,
> 
> First:
> 
> Inspired by the PCAP support in 1.11.0 I started down the path of
> cloning drill and just doing a test build before I started looking at
> 2 of the issues I posted for the PCAP storage format and also working
> to incorporate a similar WARC plugin (via the jwat jars).
> 
> `mvn package -DskipTests` works fine but `mvn install -DskipTests` generates:
> 
> Found Banned Dependency: log4j:log4j:jar:1.2.17
> Found Banned Dependency: commons-logging:commons-logging:jar:1.1.1
> 
> warnings which causes:
> 
> Apache Drill Root POM .. FAILURE
> 
> and (hence) the overall build to fail.
> 
> I'm hoping that I'm just missing something obvious after some failed googling.
> 
> 
> Second:
> 
> Is there a way to develop the WARC format plugin outside of
> Drill-source-proper vs having to build into it. If so, is there a
> small, example GH repo someone cld point me to (that also has the "how
> to get it into Drill" part). From a scan of the source, it looks like
> the formats are embedded into a few source files in the
> Drill-source-proper. If there's no way to develop one outside of it,
> that's not a problem, I just figured it'd be easier to do it outside
> of modifying Drill source first.
> 
> thx,
> 
> -Bob

Question on building drill & developing a new storage plugin

[Bob Rudis]

Hey folks,

First:

Inspired by the PCAP support in 1.11.0 I started down the path of
cloning drill and just doing a test build before I started looking at
2 of the issues I posted for the PCAP storage format and also working
to incorporate a similar WARC plugin (via the jwat jars).

`mvn package -DskipTests` works fine but `mvn install -DskipTests` generates:

Found Banned Dependency: log4j:log4j:jar:1.2.17
Found Banned Dependency: commons-logging:commons-logging:jar:1.1.1

warnings which causes:

Apache Drill Root POM .. FAILURE

and (hence) the overall build to fail.

I'm hoping that I'm just missing something obvious after some failed googling.


Second:

Is there a way to develop the WARC format plugin outside of
Drill-source-proper vs having to build into it. If so, is there a
small, example GH repo someone cld point me to (that also has the "how
to get it into Drill" part). From a scan of the source, it looks like
the formats are embedded into a few source files in the
Drill-source-proper. If there's no way to develop one outside of it,
that's not a problem, I just figured it'd be easier to do it outside
of modifying Drill source first.

thx,

-Bob

[jira] [Created] (DRILL-5702) Jdbc Driver Class not found

[Holger Kiel (JIRA)]

Holger Kiel created DRILL-5702:
--

 Summary: Jdbc Driver Class not found
 Key: DRILL-5702
 URL: https://issues.apache.org/jira/browse/DRILL-5702
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.11.0
Reporter: Holger Kiel
Priority: Critical


Cannot connect to drill cluster after upgrade to new Jar 
drill-jdbc-all-1.11.0.jar. When replacing Jar file with older release 
drill-jdbc-all-1.10.0.jar, connection works again. Tested with various client 
applications:

{code}
java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
${package.namespace.prefix}org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
 not found
at 
oadd.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
at oadd.org.apache.hadoop.security.Groups.(Groups.java:80)
at oadd.org.apache.hadoop.security.Groups.(Groups.java:74)
at 
oadd.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
at 
oadd.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283)
at 
oadd.org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
at 
oadd.org.apache.drill.exec.rpc.security.plain.PlainFactory.createAndLoginUser(PlainFactory.java:63)
at 
oadd.org.apache.drill.exec.rpc.user.UserClient.authenticate(UserClient.java:244)
at 
oadd.org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:171)
at 
oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:432)
at 
oadd.org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:379)
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:158)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:72)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
oadd.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:143)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
{code}

Workaround is using the old driver version.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (DRILL-3665) Deadlock while executing CTAS that runs out of memory

[Roman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman resolved DRILL-3665.
--
   Resolution: Duplicate
Fix Version/s: (was: Future)
   1.11.0

> Deadlock while executing CTAS that runs out of memory
> -
>
> Key: DRILL-3665
> URL: https://issues.apache.org/jira/browse/DRILL-3665
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Roman
>Priority: Critical
> Fix For: 1.11.0
>
> Attachments: drillbit.log.drill-3665, jstack.txt
>
>
> I had a query running out of memory during CTAS and after that drillbit was 
> rendered unusable:
> {code}
> 0: jdbc:drill:schema=dfs> create table lineitem as select
> . . . . . . . . . . . . > cast(columns[0] as int) l_orderkey,
> . . . . . . . . . . . . > cast(columns[1] as int) l_partkey,
> . . . . . . . . . . . . > cast(columns[2] as int) l_suppkey,
> . . . . . . . . . . . . > cast(columns[3] as int) l_linenumber,
> . . . . . . . . . . . . > cast(columns[4] as double) l_quantity,
> . . . . . . . . . . . . > cast(columns[5] as double) l_extendedprice,
> . . . . . . . . . . . . > cast(columns[6] as double) l_discount,
> . . . . . . . . . . . . > cast(columns[7] as double) l_tax,
> . . . . . . . . . . . . > cast(columns[8] as varchar(200)) l_returnflag,
> . . . . . . . . . . . . > cast(columns[9] as varchar(200)) l_linestatus,
> . . . . . . . . . . . . > cast(columns[10] as date) l_shipdate,
> . . . . . . . . . . . . > cast(columns[11] as date) l_commitdate,
> . . . . . . . . . . . . > cast(columns[12] as date) l_receiptdate,
> . . . . . . . . . . . . > cast(columns[13] as varchar(200)) 
> l_shipinstruct,
> . . . . . . . . . . . . > cast(columns[14] as varchar(200)) l_shipmode,
> . . . . . . . . . . . . > cast(columns[15] as varchar(200)) l_comment
> . . . . . . . . . . . . > from `lineitem.dat`;
> Error: RESOURCE ERROR: One or more nodes ran out of memory while executing 
> the query.
> Fragment 1:10
> [Error Id: 11084315-5388-4500-b165-642a5f595ebf on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> Here is drill's behavior after that:
> 1. Tried to run: "select * from sys.options" in the same sqlline session - 
> hangs.
> 2. Was able to start sqlline and connect to drillbit:
> - If you try running anything on this connection: it hangs.
> - Issue ^C --> you will get result if you are lucky (these queries 
> will appear as: "CANCELLATION_REQUESTED" on WebUI)
>   (I only tried querying sys.memory, sys.options which possibly have 
> a different code path than queries from actual user data)
> - If you are not lucky, you will get this error below:
> {code}
> 0: jdbc:drill:schema=dfs> show files;
> java.lang.RuntimeException: java.sql.SQLException: Unexpected 
> RuntimeException: java.lang.IllegalArgumentException: Buffer has negative 
> reference count.
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> or maybe something like this:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from nation group by n_regionkey;
> Error: CONNECTION ERROR: Exceeded timeout (5000) while waiting send 
> intermediate work fragments to remote nodes. Sent 1 and only heard response 
> back from 0 nodes.
> [Error Id: 6abce8e9-78a1-4b3d-bcec-503930482b40 on atsqa4-133.qa.lab:31010] 
> (state=,code=0)
> {code}
> I'm attaching results of a jstack  and drillbit.log and so far I was not able 
> to reproduce this problem again (working on it).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill issue #874: DRILL-5663: Drillbit fails to start when only keystore pat...

[sindhurirayavaram]

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/874
  
Hadoop uses different property names for configuring ssl properties. We can 
provide backward compatibility where we can still use the java properties in 
drill-override.conf.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New PMC member: Arina Ielchiieva

[Robert Hou]

Congratulations!  Thanks for your contributions.


--Robert


From: Vitalii Diravka 
Sent: Thursday, August 3, 2017 3:32 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New PMC member: Arina Ielchiieva

Congratulations! Well deserved.

Kind regards
Vitalii

On Thu, Aug 3, 2017 at 2:53 AM, Arina Yelchiyeva  wrote:

> Thank all you!
>
> Kind regards
> Arina
>
> On Thu, Aug 3, 2017 at 5:58 AM, Sudheesh Katkam 
> wrote:
>
> > Congratulations and thank you, Arina.
> >
> > On Wed, Aug 2, 2017 at 1:38 PM, Paul Rogers  wrote:
> >
> > > The success of the Drill 1.11 release proves this is a well-deserved
> > move.
> > > Congratulations!
> > >
> > > - Paul
> > >
> > > > On Aug 2, 2017, at 11:23 AM, Aman Sinha 
> wrote:
> > > >
> > > > I am pleased to announce that Drill PMC invited Arina Ielchiieva to
> the
> > > PMC
> > > > and she has accepted the invitation.
> > > >
> > > > Congratulations Arina and thanks for your contributions !
> > > >
> > > > -Aman
> > > > (on behalf of Drill PMC)
> > >
> > >
> >
>

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
Regarding build failure, it will fail only if we did not download these 
files before. If we did, it will take them from the cache:
`[INFO] --- download-maven-plugin:1.3.0:wget 
(add-datatables-css-as-resource) @ drill-java-exec ---
[INFO] Got from cache: 
.m2\repository\.cache\download-maven-plugin\dataTables.colVis.css_050ee51b8fd84b79ae0240ce840501c6
`
Basically we need internet only when we build the project for the first 
time, the next time having internet access is not necessary,  So I guess we are 
fine here.

+1, LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
Actually you are right, there are benefits of using CDN first [1].

[1] 
https://stackoverflow.com/questions/1014203/best-way-to-use-googles-hosted-jquery-but-fall-back-to-my-hosted-library-on-go


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
> Even if we have local files it's better to check the browser's cache 
first. If that fails, we can load them locally.
But why? javascript and css files have stable versions. If we download them 
during build time, they won't change after that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[sindhurirayavaram]

Github user sindhurirayavaram commented on the issue:

https://github.com/apache/drill/pull/891
  
@arina-ielchiieva Thanks for the comments! Yes drillbit fails if it can't 
download them during build time. These are pretty large fails. Is it good to 
add them in the source tree? 
To answer your second question! Even if we have local files it's better to 
check the browser's cache first. If that fails, we can load them locally.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
I have checked and build will fail: `Failed to execute goal 
com.googlecode.maven-download-plugin:download-maven-plugin:1.3.0:wget 
(add-jquery-as-resource) on project drill-java-exec: IO Error: Could not get 
content -> [Help 1]`, so I guess we just can refer to static resources.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #891: DRILL-5699: Drill Web UI Page Source Has Links To External...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/891
  
@sindhurirayavaram, thanks for the PR! I have two questions though:
1. Will Drill build fail if we could not download these the javascript and 
css files during build time?
2. If the answer to the first question is no than why we try to download 
the javascript and css files from the internet if we have already loaded them 
locally when loading the result page?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New PMC member: Arina Ielchiieva

[Vitalii Diravka]

Congratulations! Well deserved.

Kind regards
Vitalii

On Thu, Aug 3, 2017 at 2:53 AM, Arina Yelchiyeva  wrote:

> Thank all you!
>
> Kind regards
> Arina
>
> On Thu, Aug 3, 2017 at 5:58 AM, Sudheesh Katkam 
> wrote:
>
> > Congratulations and thank you, Arina.
> >
> > On Wed, Aug 2, 2017 at 1:38 PM, Paul Rogers  wrote:
> >
> > > The success of the Drill 1.11 release proves this is a well-deserved
> > move.
> > > Congratulations!
> > >
> > > - Paul
> > >
> > > > On Aug 2, 2017, at 11:23 AM, Aman Sinha 
> wrote:
> > > >
> > > > I am pleased to announce that Drill PMC invited Arina Ielchiieva to
> the
> > > PMC
> > > > and she has accepted the invitation.
> > > >
> > > > Congratulations Arina and thanks for your contributions !
> > > >
> > > > -Aman
> > > > (on behalf of Drill PMC)
> > >
> > >
> >
>

[GitHub] drill issue #877: DRILL-5660: Drill 1.10 queries fail due to Parquet Metadat...

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/877
  
@vdiravka 
limiting minor version to 9 is breaking the whole concept of major and 
minor versions usage. Since when we reach minor max we'll need to bump up the 
major though structure changes did not occur.
If using float is not working for major and minor version (I believe you 
having trouble comparing 1.1 and 1.10) then let's leave our `MetadataVersion` 
class, just get rid of `v` in version name.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #877: DRILL-5660: Drill 1.10 queries fail due to Parquet ...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/877#discussion_r131101557
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/MetadataVersions.java
 ---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import com.google.common.collect.ImmutableSortedSet;
+import org.apache.drill.common.exceptions.DrillRuntimeException;
+
+import java.util.NavigableSet;
+
+/**
+ * Supported metadata versions.
+ * 
+ * String metadata version consists of the following characters:
+ * optional "v" letter,
+ * major metadata version (any number of digits),
+ * optional "." delimiter (used if minor metadata version is 
specified),
+ * minor metadata version (one digit number)
+ * 
+ * Note: keep them synchronized with {@link 
Metadata.ParquetTableMetadataBase} versions
+ */
+public class MetadataVersions {
+  /**
+   * Version 1: Introduces parquet file metadata caching.
+   * See DRILL-2743
+   */
+  public static final String V1 = "v1";
+  /**
+   * Version 2: Metadata cache file size is reduced.
+   * See DRILL-4053
+   */
+  public static final String V2 = "v2";
+  /**
+   * Version 3: Difference between v3 and v2 : min/max, type_length, 
precision, scale, repetitionLevel, definitionLevel.
+   * Filter pushdown for Parquet is implemented. 
+   * See DRILL-1950
+   */
+  public static final String V3 = "v3";
+  /**
+   * Version 3.1: Absolute paths of files and directories are replaced 
with relative ones. Metadata version value
+   * doesn't contain `v` letter
+   * See DRILL-3867, DRILL-5660
+   */
+  public static final String V3_1 = "3.1";
+
+  /**
+   * Helper method to parse string metadata version into float.
+   *
+   * @param stringVersion text metadata version
+   * @return parsed Float metadata version
+   */
+  public static Float parseStringMetadataVersion(String stringVersion) {
+try {
+  if (stringVersion.contains(".") && 
stringVersion.split("\\.")[1].length() != 1) {
+throw new DrillRuntimeException("Minor metadata version shouldn't 
be greater than 9 or contain more than one digit");
+  }
+  return stringVersion.charAt(0) == 'v' ? 
Float.valueOf(stringVersion.substring(1)) : Float.valueOf(stringVersion);
+} catch (Exception e) {
+  throw new DrillRuntimeException(String.format("Could not parse 
metadata version '%s'", stringVersion), e);
+}
+  }
+
+  /**
+   * All historical versions of the Drill metadata cache files
+   */
+  public static final NavigableSet SUPPORTED_VERSIONS = 
ImmutableSortedSet.of(
--- End diff --

You can use `SortedSet`, more abstract implementation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New PMC member: Arina Ielchiieva

[Arina Yelchiyeva]

Thank all you!

Kind regards
Arina

On Thu, Aug 3, 2017 at 5:58 AM, Sudheesh Katkam  wrote:

> Congratulations and thank you, Arina.
>
> On Wed, Aug 2, 2017 at 1:38 PM, Paul Rogers  wrote:
>
> > The success of the Drill 1.11 release proves this is a well-deserved
> move.
> > Congratulations!
> >
> > - Paul
> >
> > > On Aug 2, 2017, at 11:23 AM, Aman Sinha  wrote:
> > >
> > > I am pleased to announce that Drill PMC invited Arina Ielchiieva to the
> > PMC
> > > and she has accepted the invitation.
> > >
> > > Congratulations Arina and thanks for your contributions !
> > >
> > > -Aman
> > > (on behalf of Drill PMC)
> >
> >
>