[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread amansinha100
Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/376#issuecomment-185074140
  
Simplified the state management in FileSelection.  @jacques-n is this close 
enough to what you intended ?  Also, want to note that this JIRA is motivated 
by a performance enhancement, so I haven't added a new unit test.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread jacques-n
Github user jacques-n commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53121603
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSelection.java 
---
@@ -47,6 +49,14 @@
   public List files;
   public final String selectionRoot;
 
+  private enum StatusType {
+CHECKED_DIRS,// whether we have already checked for directories
+HAS_DIRS,// whether directories were found in the selection
+EXPANDED_DIRS// whether this selection has been expanded to 
files
+  }
+
+  private final BitSet dirStatus;
--- End diff --

Sorry I wasn't clearer. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread amansinha100
Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53120105
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSelection.java 
---
@@ -47,6 +49,14 @@
   public List files;
   public final String selectionRoot;
 
+  private enum StatusType {
+CHECKED_DIRS,// whether we have already checked for directories
+HAS_DIRS,// whether directories were found in the selection
+EXPANDED_DIRS// whether this selection has been expanded to 
files
+  }
+
+  private final BitSet dirStatus;
--- End diff --

Yeah, there was probably some disconnect since your previous comment  was a 
little succinct :).  I was trying to keep track of multiple states, but agree 
it can be simplified.  I may not need to use enumerator constructors. I will 
post an updated patch once I get a clean test run.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Resolved] (DRILL-4354) Remove sessions in anonymous (user auth disabled) mode in WebUI server

2016-02-16 Thread Jacques Nadeau (JIRA)

 [ 
https://issues.apache.org/jira/browse/DRILL-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacques Nadeau resolved DRILL-4354.
---
Resolution: Fixed

> Remove sessions in anonymous (user auth disabled) mode in WebUI server
> --
>
> Key: DRILL-4354
> URL: https://issues.apache.org/jira/browse/DRILL-4354
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server
>Affects Versions: 1.5.0
>Reporter: Venki Korukanti
>Assignee: Venki Korukanti
> Fix For: 1.5.0
>
>
> Currently we open anonymous sessions when user auth disabled. These sessions 
> are cleaned up when they expire (controlled by boot config 
> {{drill.exec.http.session_max_idle_secs}}). This may lead to unnecessary 
> resource accumulation. This JIRA is to remove anonymous sessions and only 
> have sessions when user authentication is enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[ANNOUNCE] Apache Drill 1.5.0 released

2016-02-16 Thread Jason Altekruse
On behalf of *Apache* *Drill* community, I am happy to *announce* the
release of
*Apache* *Drill* 1.5.0.

The source and binary artifacts are available at [1]
Review a complete list of fixes and enhancements at [2]

This release of *Drill* fixes many issues and introduces a number of
enhancements, including the following highlights:

Web Authentication

Drill 1.5 extends Drill user authentication to the Web Console and
underlying REST API so administrators can control the extent of access to
the Web Console and REST API client applications.

Kudu Support

Drill now includes experimental support for querying the Apache Kudu
(incubating) scalable columnar database.

Improved Memory Allocator

Drill uses a new allocator that improves an operator’s use of direct memory
and tracks the memory use more accurately.

Configurable Caching of Hive Metadata

You can now configure the TTL for the Hive metadata client cache depending
on how frequently the Hive metadata is updated.

Thanks to everyone in the community who contributed in this release.

[1] http://drill.apache.org/download/
[2] http://drill.apache.org/docs/apache-drill-1-5-0-release-notes/

- Jason


[GitHub] drill pull request: DRILL-4390: Uses Resource where Drill favicon ...

2016-02-16 Thread jacques-n
Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/378#issuecomment-185001567
  
LGTM +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (DRILL-4391) browsing metadata via SQLSquirrel shows Postgres indexes, primary and foreign keys as tables

2016-02-16 Thread N Campbell (JIRA)
N Campbell created DRILL-4391:
-

 Summary: browsing metadata via SQLSquirrel shows Postgres indexes, 
primary and foreign keys as tables
 Key: DRILL-4391
 URL: https://issues.apache.org/jira/browse/DRILL-4391
 Project: Apache Drill
  Issue Type: Bug
  Components: Metadata
Affects Versions: 1.4.0
Reporter: N Campbell


Apache Drill has storage defined to access a Postgres database 
A schema in the database has several tables which either have indexes, primary 
keys, foreign keys or combination of them all. 
When SQLSquirrel presents metadata from the Drill JDBC driver the list of 
tables will include entries which correspond to the indexes, primary or foreign 
keys in the schema. The implication being that non-standard JDBC metadata 
methods to obtain information is being used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] drill pull request: DRILL-4390: Uses Resource where Drill favicon ...

2016-02-16 Thread laurentgo
GitHub user laurentgo opened a pull request:

https://github.com/apache/drill/pull/378

DRILL-4390: Uses Resource where Drill favicon is located for static assets

Drill Webserver uses the first jar containing a rest/static directory to
find its static assets. In case of another jar containing this directory, it
might cause the webserver to return 404 errors.

This configures the server to use the resource containing the Drill favicon
as the place to look for all static resources.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/laurentgo/drill laurent/DRILL-4390

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/378.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #378


commit a97c5506cfa800b37258962a8d049fd1b72852e4
Author: Laurent Goujon 
Date:   2016-02-17T00:47:17Z

DRILL-4390: Uses Resource where Drill favicon is located for static assets

Drill Webserver uses the first jar containing a rest/static directory to
find its static assets. In case of another jar containing this directory, it
might cause the webserver to return 404 errors.

This configures the server to use the resource containing the Drill favicon
as the place to look for all static resources.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (DRILL-4390) Drill webserver cannot get static assets if another jar contains a rest/static directory

2016-02-16 Thread Laurent Goujon (JIRA)
Laurent Goujon created DRILL-4390:
-

 Summary: Drill webserver cannot get static assets if another jar 
contains a rest/static directory
 Key: DRILL-4390
 URL: https://issues.apache.org/jira/browse/DRILL-4390
 Project: Apache Drill
  Issue Type: Bug
  Components: Web Server
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Priority: Minor


WebServer is configured to serve static/ URL from a resource containing 
"rest/static" but if another jar than drill java-exec jar contains the same 
directory, all queries to static/ will result most likely in 404s.

I propose to be more precise regarding the resource to use, and find the one 
containing the Drill favicon under rest/static.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] drill pull request: Drill 4372 for review

2016-02-16 Thread hsuanyi
GitHub user hsuanyi opened a pull request:

https://github.com/apache/drill/pull/377

Drill 4372 for review

@jinfengni, @amansinha100  
I just updated and rebased the initial patch. Can you start review it?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hsuanyi/incubator-drill DRILL-4372_For_Review

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/377.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #377


commit 9f6fdca69ef3c652bce0961dd76ad8bbdf1cfaa9
Author: Sudheesh Katkam 
Date:   2015-12-22T04:38:59Z

Validate Drill functions (argument and return types). WIP.

commit 3c9bb47f13488bca5b3c528c1c34af4a9b22c5e7
Author: Hsuan-Yi Chu 
Date:   2015-12-30T22:21:10Z

DRILL-4372: Expose the functions return type to Drill




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4372: Expose the functions return type t...

2016-02-16 Thread hsuanyi
Github user hsuanyi commented on the pull request:

https://github.com/apache/drill/pull/370#issuecomment-184940494
  
Will send out a rebased and updated code shortly


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4372: Expose the functions return type t...

2016-02-16 Thread hsuanyi
Github user hsuanyi closed the pull request at:

https://github.com/apache/drill/pull/370


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread jacques-n
Github user jacques-n commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53083386
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSelection.java 
---
@@ -47,6 +49,14 @@
   public List files;
   public final String selectionRoot;
 
+  private enum StatusType {
+CHECKED_DIRS,// whether we have already checked for directories
+HAS_DIRS,// whether directories were found in the selection
+EXPANDED_DIRS// whether this selection has been expanded to 
files
+  }
+
+  private final BitSet dirStatus;
--- End diff --

You took this differently than I meant it. My proposal was that 
FileSelection has various states:

NOT_CHECKED_DIRS => (HAS_DIRS | NO_DIRS) => EXPANDED

Doesn this lifecycle describe the state of FileSelection? This way you 
don't have the multi-state-management problem you currently have below with 
this kind of construct: 

fileSel.setExpanded(true);
fileSel.setCheckedForDirectories(true);
fileSel.setHasDirectories(false);  // already expanded

For each of the enumerations, we can return the right booleans that you 
need through enumerator constructors. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4383: Allow custom configurations to be ...

2016-02-16 Thread jacques-n
Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/375#issuecomment-184883511
  
that lgtm => +1 :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4275: create TransientStore for short-li...

2016-02-16 Thread jacques-n
Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/374#issuecomment-184883435
  
I'm +1 on my patch assuming the manually testing that Sudheesh is asking 
about.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread amansinha100
Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/376#issuecomment-184848577
  
Updated PR after addressing review comment from @jacques-n


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4275: create TransientStore for short-li...

2016-02-16 Thread sudheeshkatkam
Github user sudheeshkatkam commented on the pull request:

https://github.com/apache/drill/pull/374#issuecomment-184833965
  
Since there are no unit tests for web UI, can you do some manual testing to 
ensure there are no regressions?

Is there a simple way to test all persistent and transient stores? I see 
tests for a few implementations only.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4275: create TransientStore for short-li...

2016-02-16 Thread sudheeshkatkam
Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/374#discussion_r53059667
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/store/provider/CachingPersistentStoreProvider.java
 ---
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.sys.store.provider;
+
+import java.util.List;
+import java.util.concurrent.ConcurrentMap;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import org.apache.drill.common.AutoCloseables;
+import org.apache.drill.exec.exception.StoreException;
+import org.apache.drill.exec.store.sys.PersistentStore;
+import org.apache.drill.exec.store.sys.PersistentStoreConfig;
+import org.apache.drill.exec.store.sys.PersistentStoreProvider;
+
+public class CachingPersistentStoreProvider extends 
BasePersistentStoreProvider {
+  private static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(CachingPersistentStoreProvider.class);
+
+  private final ConcurrentMap storeCache = Maps.newConcurrentMap();
+  private final PersistentStoreProvider provider;
+
+  public CachingPersistentStoreProvider(PersistentStoreProvider provider) {
+this.provider = provider;
+  }
+
+  @SuppressWarnings("unchecked")
+  public  PersistentStore getOrCreateStore(final 
PersistentStoreConfig config) throws StoreException {
--- End diff --

override


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread amansinha100
Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53056521
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
 ---
@@ -529,6 +549,36 @@ public long getRowCount() {
 
   }
 
+
+  // Create and return a new file selection based on reading the metadata 
cache file.
+  // This function also initializes a few of ParquetGroupScan's fields as 
appropriate.
+  private FileSelection
+  initFromMetadataCache(DrillFileSystem fs, FileSelection selection) 
throws IOException {
+FileStatus metaRootDir = selection.getFirstPath(fs);
+Path metaFilePath = new Path(metaRootDir.getPath(), 
Metadata.METADATA_FILENAME);
+
+// get (and set internal field) the metadata for the directory by 
reading the metadata file
+this.parquetTableMetadata = Metadata.readBlockMeta(fs, 
metaFilePath.toString());
+List fileNames = Lists.newArrayList();
+for (Metadata.ParquetFileMetadata file : 
parquetTableMetadata.getFiles()) {
+  fileNames.add(file.getPath());
+}
+// when creating the file selection, set the selection root in the 
form /a/b instead of
+// file:/a/b.  The reason is that the file names above have been 
created in the form
+// /a/b/c.parquet and the format of the selection root must match that 
of the file names
+// otherwise downstream operations such as partition pruning can break.
+final Path metaRootPath = 
Path.getPathWithoutSchemeAndAuthority(metaRootDir.getPath());
+this.selectionRoot = metaRootPath.toString();
+
+// Use the FileSelection constructor directly here instead of the 
FileSelection.create() method
+// because create() changes the root to include the scheme and 
authority; In future, if create()
+// is the preferred way to instantiate a file selection, we may need 
to do something different...
+FileSelection newSelection = new 
FileSelection(selection.getStatuses(fs), fileNames, metaRootPath.toString());
--- End diff --

@jinfengni, as part of this PR I moved the expansion from 
ParquetFormatPlugin to ParquetGroupScan for the metadata cache but did not 
change the call to create FileSelection.  I agree that the inconsistency could 
be a problem.  As @adeneche points out, DRILL-4380 has the background for the 
change, but we really need to fix the FileSelection.create() interface, which 
means we should prioritize DRILL-4381.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Hangout happening now!

2016-02-16 Thread Jason Altekruse
Join us to hear the latest news on Drill, anyone with an interest in Drill
is welcome to join.

https://plus.google.com/hangouts/_/dremio.com/drillhangout

- Jason


[jira] [Created] (DRILL-4389) ResultSetMetadata for DatabaseMetadata.getSchemas returns TABLE_CAT and not TABLE_CATALOG

2016-02-16 Thread N Campbell (JIRA)
N Campbell created DRILL-4389:
-

 Summary: ResultSetMetadata for DatabaseMetadata.getSchemas returns 
TABLE_CAT and not TABLE_CATALOG 
 Key: DRILL-4389
 URL: https://issues.apache.org/jira/browse/DRILL-4389
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.4.0
Reporter: N Campbell
Priority: Minor


If you inspect the ResultSetMetadata returned by DatabaseMetadata.getSchemas it 
returns a column TABLE_CAT instead of TABLE_CATALOG

https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getSchemas()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread amansinha100
Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53047947
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSelection.java 
---
@@ -68,6 +79,7 @@ protected FileSelection(final FileSelection selection) {
 this.statuses = selection.statuses;
 this.files = selection.files;
 this.selectionRoot = selection.selectionRoot;
+this.dirStatus = new BitSet(StatusType.values().length);
--- End diff --

Yes, it should...I will change it.  Thanks for catching. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (DRILL-4388) A large # of DatabaseMetadata methods return can not access a member of class org.apache.drill.jdbc.impl.DrillDatabaseMetaDataImpl with modifiers "public"

2016-02-16 Thread N Campbell (JIRA)
N Campbell created DRILL-4388:
-

 Summary: A large # of DatabaseMetadata methods return can not 
access a member of class org.apache.drill.jdbc.impl.DrillDatabaseMetaDataImpl 
with modifiers "public"
 Key: DRILL-4388
 URL: https://issues.apache.org/jira/browse/DRILL-4388
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.4.0
 Environment: Drill 1.4.1
Reporter: N Campbell


If you were to enumerate all the methods re "get" and "is" on the 
DatabaseMetadata interface, you will find a large # which cause

can not access a member of class 
org.apache.drill.jdbc.impl.DrillDatabaseMetaDataImpl with modifiers "public"





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] drill pull request: DRILL-4287: During initial DrillTable creation...

2016-02-16 Thread adeneche
Github user adeneche commented on a diff in the pull request:

https://github.com/apache/drill/pull/376#discussion_r53043516
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
 ---
@@ -529,6 +549,36 @@ public long getRowCount() {
 
   }
 
+
+  // Create and return a new file selection based on reading the metadata 
cache file.
+  // This function also initializes a few of ParquetGroupScan's fields as 
appropriate.
+  private FileSelection
+  initFromMetadataCache(DrillFileSystem fs, FileSelection selection) 
throws IOException {
+FileStatus metaRootDir = selection.getFirstPath(fs);
+Path metaFilePath = new Path(metaRootDir.getPath(), 
Metadata.METADATA_FILENAME);
+
+// get (and set internal field) the metadata for the directory by 
reading the metadata file
+this.parquetTableMetadata = Metadata.readBlockMeta(fs, 
metaFilePath.toString());
+List fileNames = Lists.newArrayList();
+for (Metadata.ParquetFileMetadata file : 
parquetTableMetadata.getFiles()) {
+  fileNames.add(file.getPath());
+}
+// when creating the file selection, set the selection root in the 
form /a/b instead of
+// file:/a/b.  The reason is that the file names above have been 
created in the form
+// /a/b/c.parquet and the format of the selection root must match that 
of the file names
+// otherwise downstream operations such as partition pruning can break.
+final Path metaRootPath = 
Path.getPathWithoutSchemeAndAuthority(metaRootDir.getPath());
+this.selectionRoot = metaRootPath.toString();
+
+// Use the FileSelection constructor directly here instead of the 
FileSelection.create() method
+// because create() changes the root to include the scheme and 
authority; In future, if create()
+// is the preferred way to instantiate a file selection, we may need 
to do something different...
+FileSelection newSelection = new 
FileSelection(selection.getStatuses(fs), fileNames, metaRootPath.toString());
--- End diff --

Unfortunately, trying to fix this will introduce a performance regression, 
see [DRILL-4380](https://issues.apache.org/jira/browse/DRILL-4380) for more 
details.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (DRILL-4387) Fix execution side when it handles skip

2016-02-16 Thread Jinfeng Ni (JIRA)
Jinfeng Ni created DRILL-4387:
-

 Summary: Fix execution side when it handles skip
 Key: DRILL-4387
 URL: https://issues.apache.org/jira/browse/DRILL-4387
 Project: Apache Drill
  Issue Type: Bug
Reporter: Jinfeng Ni






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)