date:20160310

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

2016-03-10 Thread amansinha100

Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/424#issuecomment-195140460
  
LGTM.  +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

2016-03-10 Thread amansinha100

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/424#discussion_r55780247
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
 ---
@@ -547,29 +559,72 @@ public long getRowCount() {
   }
 
 
-  // Create and return a new file selection based on reading the metadata 
cache file.
-  // This function also initializes a few of ParquetGroupScan's fields as 
appropriate.
+  /**
+   * Create and return a new file selection based on reading the metadata 
cache file.
+   *
+   * This function also initializes a few of ParquetGroupScan's fields as 
appropriate.
+   *
+   * @param selection initial file selection
+   * @param metaFilePath metadata cache file path
+   * @return file selection read from cache
+   *
+   * @throws IOException
+   * @throws UserException when the updated selection is empty, this 
happens if the user selects an empty folder.
+   */
   private FileSelection
-  initFromMetadataCache(DrillFileSystem fs, FileSelection selection) 
throws IOException {
-FileStatus metaRootDir = selection.getFirstPath(fs);
-Path metaFilePath = new Path(metaRootDir.getPath(), 
Metadata.METADATA_FILENAME);
+  initFromMetadataCache(FileSelection selection, Path metaFilePath) throws 
IOException {
+// get the metadata for the root directory by reading the metadata file
+// parquetTableMetadata contains the metadata for all files in the 
selection root folder, but we need to make sure
+// we only select the files that are part of selection (by setting 
fileSet appropriately)
 
 // get (and set internal field) the metadata for the directory by 
reading the metadata file
 this.parquetTableMetadata = Metadata.readBlockMeta(fs, 
metaFilePath.toString());
 List fileNames = Lists.newArrayList();
-for (Metadata.ParquetFileMetadata file : 
parquetTableMetadata.getFiles()) {
-  fileNames.add(file.getPath());
+List fileStatuses = selection.getStatuses(fs);
+
+final Path first = fileStatuses.get(0).getPath();
+if (fileStatuses.size() == 1 && 
selection.getSelectionRoot().equals(first.toString())) {
+  // we are selecting all files from selection root. Expand the file 
list from the cache
+  for (Metadata.ParquetFileMetadata file : 
parquetTableMetadata.getFiles()) {
+fileNames.add(file.getPath());
+  }
+  // we don't need to populate fileSet as all files are selected
+} else {
+  // we need to expand the files from fileStatuses
+  for (FileStatus status : fileStatuses) {
+if (status.isDirectory()) {
+  //TODO read the metadata cache files in parallel
--- End diff --

Could you file a JIRA for this TODO ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

2016-03-10 Thread amansinha100

Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/407#issuecomment-195138369
  
Updated changes LGTM.  +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

2016-03-10 Thread amansinha100

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/407#discussion_r55779800
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
 ---
@@ -294,13 +313,41 @@ public UnionAllInput(UnionAllRecordBatch 
unionAllRecordBatch, RecordBatch left,
   rightSide = new OneSideInput(right);
 }
 
+private boolean isBothSideEmpty() {
+  return leftIsFinish && rightIsFinish;
+}
+
 public IterOutcome nextBatch() throws SchemaChangeException {
   if(upstream == RecordBatch.IterOutcome.NOT_YET) {
 IterOutcome iterLeft = leftSide.nextBatch();
 switch(iterLeft) {
   case OK_NEW_SCHEMA:
-break;
+whileLoop:
+while(leftSide.getRecordBatch().getRecordCount() == 0) {
+  iterLeft = leftSide.nextBatch();
+
+  switch(iterLeft) {
+case STOP:
+case OUT_OF_MEMORY:
+  return iterLeft;
+
+case NONE:
--- End diff --

Could you add comments here about why we need to keep reading after getting 
OK_NEW_SCHEMA.  Also, do any of the union-all tests (either unit or functional) 
exercise this code ? i.e where the left input has 2 or more empty files ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Time for the 1.6 Release

2016-03-10 Thread Parth Chandra

Done - Version in master branch updated to 1.7.0-SNAPSHOT


On Thu, Mar 10, 2016 at 4:46 PM, Jacques Nadeau  wrote:

> You want to roll forward the current branch to 1.7.0-SNAPSHOT so we can
> continue developing/merging stuff?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
>
>

Re: Time for the 1.6 Release

2016-03-10 Thread Jacques Nadeau

You want to roll forward the current branch to 1.7.0-SNAPSHOT so we can
continue developing/merging stuff?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Mar 10, 2016 at 4:09 PM, Parth Chandra  wrote:

> Hi guys,
>
>   I've created the 1.6.0 branch and will be rolling out the first release
> candidate as soon as I get the go ahead from the QA team, in the next
> couple of hours.
>
> Thanks
>
> Parth
>
> On Thu, Mar 10, 2016 at 3:20 PM, Parth Chandra 
> wrote:
>
> > It is usually not a good idea to try to rush in a patch at the last
> > minute. One of the reasons for having monthly releases is so people don't
> > have to wait too long for fixes and developers don't rush fixes in.
> > QA is almost done with their validation, so I'm afraid this might have to
> > go into the next release.
> >
> > On Thu, Mar 10, 2016 at 1:19 PM, Jason Altekruse 
> wrote:
> >
> >> I hadn't actually tested out the patch, what I had said was that I could
> >> add a flag to make avro files behave like parquet and JSON, without
> schema
> >> validation. The patch made it so the behavior of directories would be
> >> different from that of individual files, removing the schema
> validation. I
> >> tried applying it just now and it still doesn't appear to make the dirN
> >> columns work, but I don't understand why. I will try to take a look
> >> tonight
> >> and post a patch. It will be up to Parth if he wants to put it in the
> >> release once the full fix is merged.
> >>
> >> Jason Altekruse
> >> Software Engineer at Dremio
> >> Apache Drill Committer
> >>
> >> On Thu, Mar 10, 2016 at 1:09 AM, Stefán Baxter <
> ste...@activitystream.com
> >> >
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > This issue is still unresolved:
> >> > https://issues.apache.org/jira/browse/DRILL-4120
> >> >
> >> > It would mean a great deal to us if it was.
> >> > The solution is, as I understood Jason and Jacques, ready and only
> >> needs to
> >> > be merged.
> >> >
> >> > Regards,
> >> >  -Stefán
> >> >
> >> > On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra 
> >> wrote:
> >> >
> >> > > Hi everyone,
> >> > >
> >> > >   Just a note to  update everyone that the QA team is testing out
> the
> >> > build
> >> > > from master.
> >> > >   There are no further commits expected for the 1.6.0 release.
> >> > >   The repo is open for commits but try not to break anything :)
> >> > >
> >> > >
> >> > > Parth
> >> > >
> >> > > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra 
> >> wrote:
> >> > >
> >> > > > Okay we are down to the final one -
> >> > > >
> >> > > > DRILL-4482 - Avro no longer selects data correctly from a
> >> > > > sub-structure.(Jason)
> >> > > >
> >> > > > Note that MapR QA team is going to start testing 1.6 snapshot now
> >> > before
> >> > > I
> >> > > > roll out the release candidate. DRILL-4482 can be merged in later
> >> as it
> >> > > is
> >> > > > not likely to affect the  Hopefully there will be no show stoppers
> >> (.
> >> > > >
> >> > > > The plan is to roll out the release candidate by Thursday.
> >> > > >
> >> > > > Thanks
> >> > > >
> >> > > > Parth
> >> > > >
> >> > > >
> >> > > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> >> > wrote:
> >> > > >
> >> > > >> OK, let's leave it out then.
> >> > > >>
> >> > > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> >> > > altekruseja...@gmail.com
> >> > > >> > wrote:
> >> > > >>
> >> > > >>> To be honest I was expecting a longer review cycle so I hadn't
> run
> >> > the
> >> > > >>> unit
> >> > > >>> tests before posting it for review. There were only very minor
> >> > > functional
> >> > > >>> changes, so I wasn't thinking it would be an issue, and I was
> >> > > >>> anticipating
> >> > > >>> having to update the patch before merging it. I could update the
> >> test
> >> > > >>> that
> >> > > >>> is failing but I don't see much sense in trying to get it into
> the
> >> > > >>> release
> >> > > >>> because it only introduces new tests and some small core
> >> refactoring.
> >> > > >>>
> >> > > >>> I'm all for getting it merged so everyone can start using it, I
> >> just
> >> > > >>> think
> >> > > >>> it doesn't really matter if it happens on the release branch or
> >> back
> >> > on
> >> > > >>> master once we cut a release branch.
> >> > > >>>
> >> > > >>> I would rather try to focus on getting the Avro issues resolved,
> >> > which
> >> > > is
> >> > > >>> what I'm working on right now.
> >> > > >>>
> >> > > >>> - Jason
> >> > > >>>
> >> > > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra <
> par...@apache.org>
> >> > > wrote:
> >> > > >>>
> >> > > >>> > Sounds good Jason. Let's finalize this in the hangout.
> >> > > >>> > Do you have the expected plans for the failing tests? If so
> can
> >> you
> >> > > >>> update
> >> > > >>> > those and put in a pull request and we'll merge and run the
> >> tests.
> >> > > >>> > Any reason for the operator test framework to be

Re: Time for the 1.6 Release

2016-03-10 Thread Parth Chandra

Hi guys,

  I've created the 1.6.0 branch and will be rolling out the first release
candidate as soon as I get the go ahead from the QA team, in the next
couple of hours.

Thanks

Parth

On Thu, Mar 10, 2016 at 3:20 PM, Parth Chandra 
wrote:

> It is usually not a good idea to try to rush in a patch at the last
> minute. One of the reasons for having monthly releases is so people don't
> have to wait too long for fixes and developers don't rush fixes in.
> QA is almost done with their validation, so I'm afraid this might have to
> go into the next release.
>
> On Thu, Mar 10, 2016 at 1:19 PM, Jason Altekruse  wrote:
>
>> I hadn't actually tested out the patch, what I had said was that I could
>> add a flag to make avro files behave like parquet and JSON, without schema
>> validation. The patch made it so the behavior of directories would be
>> different from that of individual files, removing the schema validation. I
>> tried applying it just now and it still doesn't appear to make the dirN
>> columns work, but I don't understand why. I will try to take a look
>> tonight
>> and post a patch. It will be up to Parth if he wants to put it in the
>> release once the full fix is merged.
>>
>> Jason Altekruse
>> Software Engineer at Dremio
>> Apache Drill Committer
>>
>> On Thu, Mar 10, 2016 at 1:09 AM, Stefán Baxter > >
>> wrote:
>>
>> > Hi,
>> >
>> > This issue is still unresolved:
>> > https://issues.apache.org/jira/browse/DRILL-4120
>> >
>> > It would mean a great deal to us if it was.
>> > The solution is, as I understood Jason and Jacques, ready and only
>> needs to
>> > be merged.
>> >
>> > Regards,
>> >  -Stefán
>> >
>> > On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra 
>> wrote:
>> >
>> > > Hi everyone,
>> > >
>> > >   Just a note to  update everyone that the QA team is testing out the
>> > build
>> > > from master.
>> > >   There are no further commits expected for the 1.6.0 release.
>> > >   The repo is open for commits but try not to break anything :)
>> > >
>> > >
>> > > Parth
>> > >
>> > > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra 
>> wrote:
>> > >
>> > > > Okay we are down to the final one -
>> > > >
>> > > > DRILL-4482 - Avro no longer selects data correctly from a
>> > > > sub-structure.(Jason)
>> > > >
>> > > > Note that MapR QA team is going to start testing 1.6 snapshot now
>> > before
>> > > I
>> > > > roll out the release candidate. DRILL-4482 can be merged in later
>> as it
>> > > is
>> > > > not likely to affect the  Hopefully there will be no show stoppers
>> (.
>> > > >
>> > > > The plan is to roll out the release candidate by Thursday.
>> > > >
>> > > > Thanks
>> > > >
>> > > > Parth
>> > > >
>> > > >
>> > > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
>> > wrote:
>> > > >
>> > > >> OK, let's leave it out then.
>> > > >>
>> > > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
>> > > altekruseja...@gmail.com
>> > > >> > wrote:
>> > > >>
>> > > >>> To be honest I was expecting a longer review cycle so I hadn't run
>> > the
>> > > >>> unit
>> > > >>> tests before posting it for review. There were only very minor
>> > > functional
>> > > >>> changes, so I wasn't thinking it would be an issue, and I was
>> > > >>> anticipating
>> > > >>> having to update the patch before merging it. I could update the
>> test
>> > > >>> that
>> > > >>> is failing but I don't see much sense in trying to get it into the
>> > > >>> release
>> > > >>> because it only introduces new tests and some small core
>> refactoring.
>> > > >>>
>> > > >>> I'm all for getting it merged so everyone can start using it, I
>> just
>> > > >>> think
>> > > >>> it doesn't really matter if it happens on the release branch or
>> back
>> > on
>> > > >>> master once we cut a release branch.
>> > > >>>
>> > > >>> I would rather try to focus on getting the Avro issues resolved,
>> > which
>> > > is
>> > > >>> what I'm working on right now.
>> > > >>>
>> > > >>> - Jason
>> > > >>>
>> > > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
>> > > wrote:
>> > > >>>
>> > > >>> > Sounds good Jason. Let's finalize this in the hangout.
>> > > >>> > Do you have the expected plans for the failing tests? If so can
>> you
>> > > >>> update
>> > > >>> > those and put in a pull request and we'll merge and run the
>> tests.
>> > > >>> > Any reason for the operator test framework to be punted? You
>> have a
>> > > +1
>> > > >>> to
>> > > >>> > merge it.
>> > > >>> >
>> > > >>> >
>> > > >>> >
>> > > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
>> > kfar...@maprtech.com
>> > > >
>> > > >>> > wrote:
>> > > >>> >
>> > > >>> > > We should update the expected results (i.e. the expected query
>> > plan
>> > > >>> in
>> > > >>> > this
>> > > >>> > > case) and not mark them as Failing. We do not have a Failing
>> test
>> > > >>> > directory
>> > > >>> > > today.
>> > > >>> > >
>> > > >>> > > If

Re: Time for the 1.6 Release

2016-03-10 Thread Parth Chandra

It is usually not a good idea to try to rush in a patch at the last minute.
One of the reasons for having monthly releases is so people don't have to
wait too long for fixes and developers don't rush fixes in.
QA is almost done with their validation, so I'm afraid this might have to
go into the next release.

On Thu, Mar 10, 2016 at 1:19 PM, Jason Altekruse  wrote:

> I hadn't actually tested out the patch, what I had said was that I could
> add a flag to make avro files behave like parquet and JSON, without schema
> validation. The patch made it so the behavior of directories would be
> different from that of individual files, removing the schema validation. I
> tried applying it just now and it still doesn't appear to make the dirN
> columns work, but I don't understand why. I will try to take a look tonight
> and post a patch. It will be up to Parth if he wants to put it in the
> release once the full fix is merged.
>
> Jason Altekruse
> Software Engineer at Dremio
> Apache Drill Committer
>
> On Thu, Mar 10, 2016 at 1:09 AM, Stefán Baxter 
> wrote:
>
> > Hi,
> >
> > This issue is still unresolved:
> > https://issues.apache.org/jira/browse/DRILL-4120
> >
> > It would mean a great deal to us if it was.
> > The solution is, as I understood Jason and Jacques, ready and only needs
> to
> > be merged.
> >
> > Regards,
> >  -Stefán
> >
> > On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra 
> wrote:
> >
> > > Hi everyone,
> > >
> > >   Just a note to  update everyone that the QA team is testing out the
> > build
> > > from master.
> > >   There are no further commits expected for the 1.6.0 release.
> > >   The repo is open for commits but try not to break anything :)
> > >
> > >
> > > Parth
> > >
> > > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra 
> wrote:
> > >
> > > > Okay we are down to the final one -
> > > >
> > > > DRILL-4482 - Avro no longer selects data correctly from a
> > > > sub-structure.(Jason)
> > > >
> > > > Note that MapR QA team is going to start testing 1.6 snapshot now
> > before
> > > I
> > > > roll out the release candidate. DRILL-4482 can be merged in later as
> it
> > > is
> > > > not likely to affect the  Hopefully there will be no show stoppers (.
> > > >
> > > > The plan is to roll out the release candidate by Thursday.
> > > >
> > > > Thanks
> > > >
> > > > Parth
> > > >
> > > >
> > > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> > wrote:
> > > >
> > > >> OK, let's leave it out then.
> > > >>
> > > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> > > altekruseja...@gmail.com
> > > >> > wrote:
> > > >>
> > > >>> To be honest I was expecting a longer review cycle so I hadn't run
> > the
> > > >>> unit
> > > >>> tests before posting it for review. There were only very minor
> > > functional
> > > >>> changes, so I wasn't thinking it would be an issue, and I was
> > > >>> anticipating
> > > >>> having to update the patch before merging it. I could update the
> test
> > > >>> that
> > > >>> is failing but I don't see much sense in trying to get it into the
> > > >>> release
> > > >>> because it only introduces new tests and some small core
> refactoring.
> > > >>>
> > > >>> I'm all for getting it merged so everyone can start using it, I
> just
> > > >>> think
> > > >>> it doesn't really matter if it happens on the release branch or
> back
> > on
> > > >>> master once we cut a release branch.
> > > >>>
> > > >>> I would rather try to focus on getting the Avro issues resolved,
> > which
> > > is
> > > >>> what I'm working on right now.
> > > >>>
> > > >>> - Jason
> > > >>>
> > > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> > > wrote:
> > > >>>
> > > >>> > Sounds good Jason. Let's finalize this in the hangout.
> > > >>> > Do you have the expected plans for the failing tests? If so can
> you
> > > >>> update
> > > >>> > those and put in a pull request and we'll merge and run the
> tests.
> > > >>> > Any reason for the operator test framework to be punted? You
> have a
> > > +1
> > > >>> to
> > > >>> > merge it.
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
> > kfar...@maprtech.com
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > We should update the expected results (i.e. the expected query
> > plan
> > > >>> in
> > > >>> > this
> > > >>> > > case) and not mark them as Failing. We do not have a Failing
> test
> > > >>> > directory
> > > >>> > > today.
> > > >>> > >
> > > >>> > > If we are sure the expected query plans for these tests are
> > > incorrect
> > > >>> > > (today), we should go ahead and update the expected results
> files
> > > >>> with
> > > >>> > the
> > > >>> > > correct query plans.
> > > >>> > >
> > > >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> > > >>> > > pruning/hierarchical/plan/count2_MD-185.q
> > > >>> > >

Re: Time for the 1.6 Release

2016-03-10 Thread Stefán Baxter

thank you

On Thu, Mar 10, 2016 at 9:19 PM, Jason Altekruse  wrote:

> I hadn't actually tested out the patch, what I had said was that I could
> add a flag to make avro files behave like parquet and JSON, without schema
> validation. The patch made it so the behavior of directories would be
> different from that of individual files, removing the schema validation. I
> tried applying it just now and it still doesn't appear to make the dirN
> columns work, but I don't understand why. I will try to take a look tonight
> and post a patch. It will be up to Parth if he wants to put it in the
> release once the full fix is merged.
>
> Jason Altekruse
> Software Engineer at Dremio
> Apache Drill Committer
>
> On Thu, Mar 10, 2016 at 1:09 AM, Stefán Baxter 
> wrote:
>
> > Hi,
> >
> > This issue is still unresolved:
> > https://issues.apache.org/jira/browse/DRILL-4120
> >
> > It would mean a great deal to us if it was.
> > The solution is, as I understood Jason and Jacques, ready and only needs
> to
> > be merged.
> >
> > Regards,
> >  -Stefán
> >
> > On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra 
> wrote:
> >
> > > Hi everyone,
> > >
> > >   Just a note to  update everyone that the QA team is testing out the
> > build
> > > from master.
> > >   There are no further commits expected for the 1.6.0 release.
> > >   The repo is open for commits but try not to break anything :)
> > >
> > >
> > > Parth
> > >
> > > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra 
> wrote:
> > >
> > > > Okay we are down to the final one -
> > > >
> > > > DRILL-4482 - Avro no longer selects data correctly from a
> > > > sub-structure.(Jason)
> > > >
> > > > Note that MapR QA team is going to start testing 1.6 snapshot now
> > before
> > > I
> > > > roll out the release candidate. DRILL-4482 can be merged in later as
> it
> > > is
> > > > not likely to affect the  Hopefully there will be no show stoppers (.
> > > >
> > > > The plan is to roll out the release candidate by Thursday.
> > > >
> > > > Thanks
> > > >
> > > > Parth
> > > >
> > > >
> > > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> > wrote:
> > > >
> > > >> OK, let's leave it out then.
> > > >>
> > > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> > > altekruseja...@gmail.com
> > > >> > wrote:
> > > >>
> > > >>> To be honest I was expecting a longer review cycle so I hadn't run
> > the
> > > >>> unit
> > > >>> tests before posting it for review. There were only very minor
> > > functional
> > > >>> changes, so I wasn't thinking it would be an issue, and I was
> > > >>> anticipating
> > > >>> having to update the patch before merging it. I could update the
> test
> > > >>> that
> > > >>> is failing but I don't see much sense in trying to get it into the
> > > >>> release
> > > >>> because it only introduces new tests and some small core
> refactoring.
> > > >>>
> > > >>> I'm all for getting it merged so everyone can start using it, I
> just
> > > >>> think
> > > >>> it doesn't really matter if it happens on the release branch or
> back
> > on
> > > >>> master once we cut a release branch.
> > > >>>
> > > >>> I would rather try to focus on getting the Avro issues resolved,
> > which
> > > is
> > > >>> what I'm working on right now.
> > > >>>
> > > >>> - Jason
> > > >>>
> > > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> > > wrote:
> > > >>>
> > > >>> > Sounds good Jason. Let's finalize this in the hangout.
> > > >>> > Do you have the expected plans for the failing tests? If so can
> you
> > > >>> update
> > > >>> > those and put in a pull request and we'll merge and run the
> tests.
> > > >>> > Any reason for the operator test framework to be punted? You
> have a
> > > +1
> > > >>> to
> > > >>> > merge it.
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
> > kfar...@maprtech.com
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > We should update the expected results (i.e. the expected query
> > plan
> > > >>> in
> > > >>> > this
> > > >>> > > case) and not mark them as Failing. We do not have a Failing
> test
> > > >>> > directory
> > > >>> > > today.
> > > >>> > >
> > > >>> > > If we are sure the expected query plans for these tests are
> > > incorrect
> > > >>> > > (today), we should go ahead and update the expected results
> files
> > > >>> with
> > > >>> > the
> > > >>> > > correct query plans.
> > > >>> > >
> > > >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> > > >>> > > pruning/hierarchical/plan/count2_MD-185.q
> > > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> > > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > Khurram
> > > >>> > >
> > > >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> > > >>> > altekruseja...@gmail.com>
> > > >>> > > wrote:
> > > >>> > >
> >

Re: Time for the 1.6 Release

2016-03-10 Thread Jason Altekruse

I hadn't actually tested out the patch, what I had said was that I could
add a flag to make avro files behave like parquet and JSON, without schema
validation. The patch made it so the behavior of directories would be
different from that of individual files, removing the schema validation. I
tried applying it just now and it still doesn't appear to make the dirN
columns work, but I don't understand why. I will try to take a look tonight
and post a patch. It will be up to Parth if he wants to put it in the
release once the full fix is merged.

Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer

On Thu, Mar 10, 2016 at 1:09 AM, Stefán Baxter 
wrote:

> Hi,
>
> This issue is still unresolved:
> https://issues.apache.org/jira/browse/DRILL-4120
>
> It would mean a great deal to us if it was.
> The solution is, as I understood Jason and Jacques, ready and only needs to
> be merged.
>
> Regards,
>  -Stefán
>
> On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra  wrote:
>
> > Hi everyone,
> >
> >   Just a note to  update everyone that the QA team is testing out the
> build
> > from master.
> >   There are no further commits expected for the 1.6.0 release.
> >   The repo is open for commits but try not to break anything :)
> >
> >
> > Parth
> >
> > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra  wrote:
> >
> > > Okay we are down to the final one -
> > >
> > > DRILL-4482 - Avro no longer selects data correctly from a
> > > sub-structure.(Jason)
> > >
> > > Note that MapR QA team is going to start testing 1.6 snapshot now
> before
> > I
> > > roll out the release candidate. DRILL-4482 can be merged in later as it
> > is
> > > not likely to affect the  Hopefully there will be no show stoppers (.
> > >
> > > The plan is to roll out the release candidate by Thursday.
> > >
> > > Thanks
> > >
> > > Parth
> > >
> > >
> > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> wrote:
> > >
> > >> OK, let's leave it out then.
> > >>
> > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> > altekruseja...@gmail.com
> > >> > wrote:
> > >>
> > >>> To be honest I was expecting a longer review cycle so I hadn't run
> the
> > >>> unit
> > >>> tests before posting it for review. There were only very minor
> > functional
> > >>> changes, so I wasn't thinking it would be an issue, and I was
> > >>> anticipating
> > >>> having to update the patch before merging it. I could update the test
> > >>> that
> > >>> is failing but I don't see much sense in trying to get it into the
> > >>> release
> > >>> because it only introduces new tests and some small core refactoring.
> > >>>
> > >>> I'm all for getting it merged so everyone can start using it, I just
> > >>> think
> > >>> it doesn't really matter if it happens on the release branch or back
> on
> > >>> master once we cut a release branch.
> > >>>
> > >>> I would rather try to focus on getting the Avro issues resolved,
> which
> > is
> > >>> what I'm working on right now.
> > >>>
> > >>> - Jason
> > >>>
> > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> > wrote:
> > >>>
> > >>> > Sounds good Jason. Let's finalize this in the hangout.
> > >>> > Do you have the expected plans for the failing tests? If so can you
> > >>> update
> > >>> > those and put in a pull request and we'll merge and run the tests.
> > >>> > Any reason for the operator test framework to be punted? You have a
> > +1
> > >>> to
> > >>> > merge it.
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
> kfar...@maprtech.com
> > >
> > >>> > wrote:
> > >>> >
> > >>> > > We should update the expected results (i.e. the expected query
> plan
> > >>> in
> > >>> > this
> > >>> > > case) and not mark them as Failing. We do not have a Failing test
> > >>> > directory
> > >>> > > today.
> > >>> > >
> > >>> > > If we are sure the expected query plans for these tests are
> > incorrect
> > >>> > > (today), we should go ahead and update the expected results files
> > >>> with
> > >>> > the
> > >>> > > correct query plans.
> > >>> > >
> > >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> > >>> > > pruning/hierarchical/plan/count2_MD-185.q
> > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> > >>> > >
> > >>> > > Thanks,
> > >>> > > Khurram
> > >>> > >
> > >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> > >>> > altekruseja...@gmail.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > The cutoff time sounds good to me. I don't think I'm too
> > >>> overburdened,
> > >>> > I
> > >>> > > > mostly tried putting together a merge branch today and had to
> > kick
> > >>> a
> > >>> > few
> > >>> > > > things out as I found issues.
> > >>> > > >
> > >>> > > > As we are not running the full regression cluster at Dremio, it
> > >>> would
> > >>> > be
> > >>> > > > helpful

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

2016-03-10 Thread amansinha100

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/407#discussion_r55749381
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
 ---
@@ -162,6 +162,25 @@ private IterOutcome doWork() throws 
ClassTransformationException, IOException, S
 allocationVectors = Lists.newArrayList();
 transfers.clear();
 
+// If both sides of Union-All are empty
+if(unionAllInput.isBothSideEmpty()) {
+  for(int i = 0; i < outputFields.size(); ++i) {
--- End diff --

Ok, I suppose we are not supporting the SELECT * case, so the output fields 
only depends on what is specified in the SELECT list, not from the table.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-4490: Ensure the count generated by Conv...

2016-03-10 Thread hsuanyi

Github user hsuanyi commented on a diff in the pull request:

https://github.com/apache/drill/pull/423#discussion_r55730852
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestAggregateFunctions.java
 ---
@@ -460,4 +466,23 @@ public void testGroupBySystemFuncFileSystemTable() 
throws Exception {
   public void test4443() throws Exception {
 test("SELECT MIN(columns[1]) FROM dfs_test.`%s/agg/4443.csv` GROUP BY 
columns[0]", TEST_RES_PATH);
   }
+
+  @Test
+  public void testCountStarRequired() throws Exception {
+final String query = "select count(*) as col from 
cp.`tpch/region.parquet` limit 0";
--- End diff --

Only schema is being validated in this case.
It might be confusing if the query returns the records but we only inspect 
the schema.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

2016-03-10 Thread adeneche

GitHub user adeneche opened a pull request:

https://github.com/apache/drill/pull/424

Drill 4484: NPE when querying  empty directory

this PR also includes the fix for DRILL-4376. I will rebase once that fix 
has been merged into master

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/adeneche/incubator-drill DRILL-4484

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #424


commit b8fa596d421eca31ff28a40e681c544c10521078
Author: adeneche 
Date:   2016-03-09T12:44:02Z

DRILL-4376: Wrong results when doing a count(*) on part of directories with 
metadata cache

commit e8c5dcb64926d0931c48cb0eba3f17dc2b597822
Author: adeneche 
Date:   2016-03-10T09:40:06Z

DRILL-4484: NPE when querying  empty directory




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Time for the 1.6 Release

2016-03-10 Thread Stefán Baxter

Hi,

This issue is still unresolved:
https://issues.apache.org/jira/browse/DRILL-4120

It would mean a great deal to us if it was.
The solution is, as I understood Jason and Jacques, ready and only needs to
be merged.

Regards,
 -Stefán

On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra  wrote:

> Hi everyone,
>
>   Just a note to  update everyone that the QA team is testing out the build
> from master.
>   There are no further commits expected for the 1.6.0 release.
>   The repo is open for commits but try not to break anything :)
>
>
> Parth
>
> On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra  wrote:
>
> > Okay we are down to the final one -
> >
> > DRILL-4482 - Avro no longer selects data correctly from a
> > sub-structure.(Jason)
> >
> > Note that MapR QA team is going to start testing 1.6 snapshot now before
> I
> > roll out the release candidate. DRILL-4482 can be merged in later as it
> is
> > not likely to affect the  Hopefully there will be no show stoppers (.
> >
> > The plan is to roll out the release candidate by Thursday.
> >
> > Thanks
> >
> > Parth
> >
> >
> > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra  wrote:
> >
> >> OK, let's leave it out then.
> >>
> >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> altekruseja...@gmail.com
> >> > wrote:
> >>
> >>> To be honest I was expecting a longer review cycle so I hadn't run the
> >>> unit
> >>> tests before posting it for review. There were only very minor
> functional
> >>> changes, so I wasn't thinking it would be an issue, and I was
> >>> anticipating
> >>> having to update the patch before merging it. I could update the test
> >>> that
> >>> is failing but I don't see much sense in trying to get it into the
> >>> release
> >>> because it only introduces new tests and some small core refactoring.
> >>>
> >>> I'm all for getting it merged so everyone can start using it, I just
> >>> think
> >>> it doesn't really matter if it happens on the release branch or back on
> >>> master once we cut a release branch.
> >>>
> >>> I would rather try to focus on getting the Avro issues resolved, which
> is
> >>> what I'm working on right now.
> >>>
> >>> - Jason
> >>>
> >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> wrote:
> >>>
> >>> > Sounds good Jason. Let's finalize this in the hangout.
> >>> > Do you have the expected plans for the failing tests? If so can you
> >>> update
> >>> > those and put in a pull request and we'll merge and run the tests.
> >>> > Any reason for the operator test framework to be punted? You have a
> +1
> >>> to
> >>> > merge it.
> >>> >
> >>> >
> >>> >
> >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz  >
> >>> > wrote:
> >>> >
> >>> > > We should update the expected results (i.e. the expected query plan
> >>> in
> >>> > this
> >>> > > case) and not mark them as Failing. We do not have a Failing test
> >>> > directory
> >>> > > today.
> >>> > >
> >>> > > If we are sure the expected query plans for these tests are
> incorrect
> >>> > > (today), we should go ahead and update the expected results files
> >>> with
> >>> > the
> >>> > > correct query plans.
> >>> > >
> >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> >>> > > pruning/hierarchical/plan/count2_MD-185.q
> >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> >>> > >
> >>> > > Thanks,
> >>> > > Khurram
> >>> > >
> >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> >>> > altekruseja...@gmail.com>
> >>> > > wrote:
> >>> > >
> >>> > > > The cutoff time sounds good to me. I don't think I'm too
> >>> overburdened,
> >>> > I
> >>> > > > mostly tried putting together a merge branch today and had to
> kick
> >>> a
> >>> > few
> >>> > > > things out as I found issues.
> >>> > > >
> >>> > > > As we are not running the full regression cluster at Dremio, it
> >>> would
> >>> > be
> >>> > > > helpful if someone could merge Jacques patch (DRILL-4474) after
> >>> running
> >>> > > the
> >>> > > > complete set of tests. Could someone at MapR pick up the test
> >>> updates
> >>> > and
> >>> > > > merging this patch?
> >>> > > >
> >>> > > > These are the failures I was seeing so far, my fix was just going
> >>> to be
> >>> > > to
> >>> > > > mark the tests as failing:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> Functional/ctas/ctas_auto_partition/existing_partition_pruning/hierarchical/plan/count2_MD-185.q
> >>> > > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> >>> > > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> >>> > > >
> >>> > > > I will just go ahead and merge the other two fixes as I had a
> clean
> >>> > unit
> >>> > > > test run and neither of the patches will have any impact on the
> >>> > > regression
> >>> > > > tests. One is a build fix, that only changes behavior when
> running
> >>> the
> >>> > > >

Re: Time for the 1.6 Release

2016-03-10 Thread Stefán Baxter

You are correct, it was, sorry.

On Thu, Mar 10, 2016 at 9:02 AM, Abdel Hakim Deneche 
wrote:

> Hey Stefán, I just left a comment in the JIRA. I think the issue is related
> to the double quotes.
>
> On Thu, Mar 10, 2016 at 9:55 AM, Stefán Baxter 
> wrote:
>
> > Hi,
> >
> > I noticed that using the IN operator with a sub structure has stopped
> > working:
> >
> > select s.client_ip.ip from dfs.asa.`/processed/venuepoint/transactions`
> as
> > s where s.client_ip.ip in ("unknown") limit 2;
> > Error: PARSE ERROR: Encountered "\"" at line 1, column 103.
> > Was expecting one of:
> > ...
> > SQL Query select s.client_ip.ip from
> > dfs.asa.`/processed/venuepoint/transactions` as s where s.client_ip.ip in
> > ("2.69.200.113","unknown") limit 2
> >
> >
> >   ^
> > (the hat should point at the _ip part of the where clause)
> >
> > I will report this in Jira and mark it as a blocker.
> >
> > Regards,
> >  -Stefán
> >
> > On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra 
> wrote:
> >
> > > Hi everyone,
> > >
> > >   Just a note to  update everyone that the QA team is testing out the
> > build
> > > from master.
> > >   There are no further commits expected for the 1.6.0 release.
> > >   The repo is open for commits but try not to break anything :)
> > >
> > >
> > > Parth
> > >
> > > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra 
> wrote:
> > >
> > > > Okay we are down to the final one -
> > > >
> > > > DRILL-4482 - Avro no longer selects data correctly from a
> > > > sub-structure.(Jason)
> > > >
> > > > Note that MapR QA team is going to start testing 1.6 snapshot now
> > before
> > > I
> > > > roll out the release candidate. DRILL-4482 can be merged in later as
> it
> > > is
> > > > not likely to affect the  Hopefully there will be no show stoppers (.
> > > >
> > > > The plan is to roll out the release candidate by Thursday.
> > > >
> > > > Thanks
> > > >
> > > > Parth
> > > >
> > > >
> > > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> > wrote:
> > > >
> > > >> OK, let's leave it out then.
> > > >>
> > > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> > > altekruseja...@gmail.com
> > > >> > wrote:
> > > >>
> > > >>> To be honest I was expecting a longer review cycle so I hadn't run
> > the
> > > >>> unit
> > > >>> tests before posting it for review. There were only very minor
> > > functional
> > > >>> changes, so I wasn't thinking it would be an issue, and I was
> > > >>> anticipating
> > > >>> having to update the patch before merging it. I could update the
> test
> > > >>> that
> > > >>> is failing but I don't see much sense in trying to get it into the
> > > >>> release
> > > >>> because it only introduces new tests and some small core
> refactoring.
> > > >>>
> > > >>> I'm all for getting it merged so everyone can start using it, I
> just
> > > >>> think
> > > >>> it doesn't really matter if it happens on the release branch or
> back
> > on
> > > >>> master once we cut a release branch.
> > > >>>
> > > >>> I would rather try to focus on getting the Avro issues resolved,
> > which
> > > is
> > > >>> what I'm working on right now.
> > > >>>
> > > >>> - Jason
> > > >>>
> > > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> > > wrote:
> > > >>>
> > > >>> > Sounds good Jason. Let's finalize this in the hangout.
> > > >>> > Do you have the expected plans for the failing tests? If so can
> you
> > > >>> update
> > > >>> > those and put in a pull request and we'll merge and run the
> tests.
> > > >>> > Any reason for the operator test framework to be punted? You
> have a
> > > +1
> > > >>> to
> > > >>> > merge it.
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
> > kfar...@maprtech.com
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > We should update the expected results (i.e. the expected query
> > plan
> > > >>> in
> > > >>> > this
> > > >>> > > case) and not mark them as Failing. We do not have a Failing
> test
> > > >>> > directory
> > > >>> > > today.
> > > >>> > >
> > > >>> > > If we are sure the expected query plans for these tests are
> > > incorrect
> > > >>> > > (today), we should go ahead and update the expected results
> files
> > > >>> with
> > > >>> > the
> > > >>> > > correct query plans.
> > > >>> > >
> > > >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> > > >>> > > pruning/hierarchical/plan/count2_MD-185.q
> > > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> > > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > Khurram
> > > >>> > >
> > > >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> > > >>> > altekruseja...@gmail.com>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > The cutoff time sounds good to me. I don't

Re: Time for the 1.6 Release

2016-03-10 Thread Abdel Hakim Deneche

Hey Stefán, I just left a comment in the JIRA. I think the issue is related
to the double quotes.

On Thu, Mar 10, 2016 at 9:55 AM, Stefán Baxter 
wrote:

> Hi,
>
> I noticed that using the IN operator with a sub structure has stopped
> working:
>
> select s.client_ip.ip from dfs.asa.`/processed/venuepoint/transactions` as
> s where s.client_ip.ip in ("unknown") limit 2;
> Error: PARSE ERROR: Encountered "\"" at line 1, column 103.
> Was expecting one of:
> ...
> SQL Query select s.client_ip.ip from
> dfs.asa.`/processed/venuepoint/transactions` as s where s.client_ip.ip in
> ("2.69.200.113","unknown") limit 2
>
>
>   ^
> (the hat should point at the _ip part of the where clause)
>
> I will report this in Jira and mark it as a blocker.
>
> Regards,
>  -Stefán
>
> On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra  wrote:
>
> > Hi everyone,
> >
> >   Just a note to  update everyone that the QA team is testing out the
> build
> > from master.
> >   There are no further commits expected for the 1.6.0 release.
> >   The repo is open for commits but try not to break anything :)
> >
> >
> > Parth
> >
> > On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra  wrote:
> >
> > > Okay we are down to the final one -
> > >
> > > DRILL-4482 - Avro no longer selects data correctly from a
> > > sub-structure.(Jason)
> > >
> > > Note that MapR QA team is going to start testing 1.6 snapshot now
> before
> > I
> > > roll out the release candidate. DRILL-4482 can be merged in later as it
> > is
> > > not likely to affect the  Hopefully there will be no show stoppers (.
> > >
> > > The plan is to roll out the release candidate by Thursday.
> > >
> > > Thanks
> > >
> > > Parth
> > >
> > >
> > > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra 
> wrote:
> > >
> > >> OK, let's leave it out then.
> > >>
> > >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> > altekruseja...@gmail.com
> > >> > wrote:
> > >>
> > >>> To be honest I was expecting a longer review cycle so I hadn't run
> the
> > >>> unit
> > >>> tests before posting it for review. There were only very minor
> > functional
> > >>> changes, so I wasn't thinking it would be an issue, and I was
> > >>> anticipating
> > >>> having to update the patch before merging it. I could update the test
> > >>> that
> > >>> is failing but I don't see much sense in trying to get it into the
> > >>> release
> > >>> because it only introduces new tests and some small core refactoring.
> > >>>
> > >>> I'm all for getting it merged so everyone can start using it, I just
> > >>> think
> > >>> it doesn't really matter if it happens on the release branch or back
> on
> > >>> master once we cut a release branch.
> > >>>
> > >>> I would rather try to focus on getting the Avro issues resolved,
> which
> > is
> > >>> what I'm working on right now.
> > >>>
> > >>> - Jason
> > >>>
> > >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> > wrote:
> > >>>
> > >>> > Sounds good Jason. Let's finalize this in the hangout.
> > >>> > Do you have the expected plans for the failing tests? If so can you
> > >>> update
> > >>> > those and put in a pull request and we'll merge and run the tests.
> > >>> > Any reason for the operator test framework to be punted? You have a
> > +1
> > >>> to
> > >>> > merge it.
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz <
> kfar...@maprtech.com
> > >
> > >>> > wrote:
> > >>> >
> > >>> > > We should update the expected results (i.e. the expected query
> plan
> > >>> in
> > >>> > this
> > >>> > > case) and not mark them as Failing. We do not have a Failing test
> > >>> > directory
> > >>> > > today.
> > >>> > >
> > >>> > > If we are sure the expected query plans for these tests are
> > incorrect
> > >>> > > (today), we should go ahead and update the expected results files
> > >>> with
> > >>> > the
> > >>> > > correct query plans.
> > >>> > >
> > >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> > >>> > > pruning/hierarchical/plan/count2_MD-185.q
> > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> > >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> > >>> > >
> > >>> > > Thanks,
> > >>> > > Khurram
> > >>> > >
> > >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> > >>> > altekruseja...@gmail.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > The cutoff time sounds good to me. I don't think I'm too
> > >>> overburdened,
> > >>> > I
> > >>> > > > mostly tried putting together a merge branch today and had to
> > kick
> > >>> a
> > >>> > few
> > >>> > > > things out as I found issues.
> > >>> > > >
> > >>> > > > As we are not running the full regression cluster at Dremio, it
> > >>> would
> > >>> > be
> > >>> > > > helpful if someone could merge Jacques patch (DRILL-4474) after
> > >>> running
> > >>> > > the
> >

[jira] [Created] (DRILL-4495) IN operator does not work

2016-03-10 Thread JIRA

Stefán Baxter created DRILL-4495:


 Summary: IN operator does not work
 Key: DRILL-4495
 URL: https://issues.apache.org/jira/browse/DRILL-4495
 Project: Apache Drill
  Issue Type: Bug
  Components: SQL Parser
Affects Versions: 1.6.0
Reporter: Stefán Baxter
Priority: Blocker
 Fix For: 1.6.0


I noticed that using the IN operator with a sub structure has stopped working:

select s.client_ip.ip from dfs.asa.`/processed/venuepoint/transactions` as s 
where s.client_ip.ip in ("unknown") limit 2;
Error: PARSE ERROR: Encountered "\"" at line 1, column 103.
Was expecting one of:
...
SQL Query select s.client_ip.ip from 
dfs.asa.`/processed/venuepoint/transactions` as s where s.client_ip.ip in 
("2.69.200.113","unknown") limit 2


  ^
(the hat should point at the _ip part of the where clause)

I will report this in Jira and mark it as a blocker.

The IN works with nothing here using the latest 1.6-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Time for the 1.6 Release

2016-03-10 Thread Stefán Baxter

Hi,

I noticed that using the IN operator with a sub structure has stopped
working:

select s.client_ip.ip from dfs.asa.`/processed/venuepoint/transactions` as
s where s.client_ip.ip in ("unknown") limit 2;
Error: PARSE ERROR: Encountered "\"" at line 1, column 103.
Was expecting one of:
...
SQL Query select s.client_ip.ip from
dfs.asa.`/processed/venuepoint/transactions` as s where s.client_ip.ip in
("2.69.200.113","unknown") limit 2


  ^
(the hat should point at the _ip part of the where clause)

I will report this in Jira and mark it as a blocker.

Regards,
 -Stefán

On Thu, Mar 10, 2016 at 3:50 AM, Parth Chandra  wrote:

> Hi everyone,
>
>   Just a note to  update everyone that the QA team is testing out the build
> from master.
>   There are no further commits expected for the 1.6.0 release.
>   The repo is open for commits but try not to break anything :)
>
>
> Parth
>
> On Tue, Mar 8, 2016 at 5:16 PM, Parth Chandra  wrote:
>
> > Okay we are down to the final one -
> >
> > DRILL-4482 - Avro no longer selects data correctly from a
> > sub-structure.(Jason)
> >
> > Note that MapR QA team is going to start testing 1.6 snapshot now before
> I
> > roll out the release candidate. DRILL-4482 can be merged in later as it
> is
> > not likely to affect the  Hopefully there will be no show stoppers (.
> >
> > The plan is to roll out the release candidate by Thursday.
> >
> > Thanks
> >
> > Parth
> >
> >
> > On Tue, Mar 8, 2016 at 9:31 AM, Parth Chandra  wrote:
> >
> >> OK, let's leave it out then.
> >>
> >> On Tue, Mar 8, 2016 at 9:25 AM, Jason Altekruse <
> altekruseja...@gmail.com
> >> > wrote:
> >>
> >>> To be honest I was expecting a longer review cycle so I hadn't run the
> >>> unit
> >>> tests before posting it for review. There were only very minor
> functional
> >>> changes, so I wasn't thinking it would be an issue, and I was
> >>> anticipating
> >>> having to update the patch before merging it. I could update the test
> >>> that
> >>> is failing but I don't see much sense in trying to get it into the
> >>> release
> >>> because it only introduces new tests and some small core refactoring.
> >>>
> >>> I'm all for getting it merged so everyone can start using it, I just
> >>> think
> >>> it doesn't really matter if it happens on the release branch or back on
> >>> master once we cut a release branch.
> >>>
> >>> I would rather try to focus on getting the Avro issues resolved, which
> is
> >>> what I'm working on right now.
> >>>
> >>> - Jason
> >>>
> >>> On Tue, Mar 8, 2016 at 8:58 AM, Parth Chandra 
> wrote:
> >>>
> >>> > Sounds good Jason. Let's finalize this in the hangout.
> >>> > Do you have the expected plans for the failing tests? If so can you
> >>> update
> >>> > those and put in a pull request and we'll merge and run the tests.
> >>> > Any reason for the operator test framework to be punted? You have a
> +1
> >>> to
> >>> > merge it.
> >>> >
> >>> >
> >>> >
> >>> > On Mon, Mar 7, 2016 at 9:33 PM, Khurram Faraaz  >
> >>> > wrote:
> >>> >
> >>> > > We should update the expected results (i.e. the expected query plan
> >>> in
> >>> > this
> >>> > > case) and not mark them as Failing. We do not have a Failing test
> >>> > directory
> >>> > > today.
> >>> > >
> >>> > > If we are sure the expected query plans for these tests are
> incorrect
> >>> > > (today), we should go ahead and update the expected results files
> >>> with
> >>> > the
> >>> > > correct query plans.
> >>> > >
> >>> > > Functional/ctas/ctas_auto_partition/existing_partition_
> >>> > > pruning/hierarchical/plan/count2_MD-185.q
> >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_2.q
> >>> > > Functional/ctas/ctas_auto_partition/general/plan/drill3947_4.q
> >>> > >
> >>> > > Thanks,
> >>> > > Khurram
> >>> > >
> >>> > > On Tue, Mar 8, 2016 at 8:58 AM, Jason Altekruse <
> >>> > altekruseja...@gmail.com>
> >>> > > wrote:
> >>> > >
> >>> > > > The cutoff time sounds good to me. I don't think I'm too
> >>> overburdened,
> >>> > I
> >>> > > > mostly tried putting together a merge branch today and had to
> kick
> >>> a
> >>> > few
> >>> > > > things out as I found issues.
> >>> > > >
> >>> > > > As we are not running the full regression cluster at Dremio, it
> >>> would
> >>> > be
> >>> > > > helpful if someone could merge Jacques patch (DRILL-4474) after
> >>> running
> >>> > > the
> >>> > > > complete set of tests. Could someone at MapR pick up the test
> >>> updates
> >>> > and
> >>> > > > merging this patch?
> >>> > > >
> >>> > > > These are the failures I was seeing so far, my fix was just going
> >>> to be
> >>> > > to
> >>> > > > mark the tests as failing:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> Functional/ctas/ctas_auto_partition/existing_partition_pruning/hierarchical/plan/count2_MD-185.q
> >>> > > >

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

[GitHub] drill pull request: DRILL-4476: Allow UnionAllRecordBatch to manag...

[GitHub] drill pull request: DRILL-4490: Ensure the count generated by Conv...

[GitHub] drill pull request: Drill 4484: NPE when querying empty directory

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

Re: Time for the 1.6 Release

[jira] [Created] (DRILL-4495) IN operator does not work

Re: Time for the 1.6 Release

18 matches

Site Navigation

Mail list logo

Footer information