[GitHub] drill pull request #824: DRILL-3867: Store relative paths in metadata file

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/824


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #857: DRILL-5599: Notify StatusHandler that batch sending...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/857


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #849: DRILL-5568: Include hadoop-common jars inside drill...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/849


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #855: DRILL-5590: Bugs in CSV field matching, null column...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/855


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #852: DRILL-5587: Validate Parquet blockSize and pageSize...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/852


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #855: DRILL-5590: Bugs in CSV field matching, null columns

[amansinha100]

Github user amansinha100 commented on the issue:

https://github.com/apache/drill/pull/855
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New PMC Chair of Apache Drill

[Julian Hyde]

Congratulations!

On Fri, Jun 23, 2017 at 5:00 PM, Aman Sinha  wrote:
> Thank you all !
>
> -Aman
>
> On Fri, Jun 23, 2017 at 3:59 PM, Paul Rogers  wrote:
>
>> Congratulations Aman!
>>
>> And, thanks much to Parth, our outgoing chair, for his contributions as
>> PMC chair over the last year!
>>
>> - Paul
>>
>> > On Jun 23, 2017, at 12:48 PM, Aman Bawa  wrote:
>> >
>> > Congratulations, Aman – well deserved!
>> >
>> > On 6/23/17, 12:46 PM, "Arina Yelchiyeva" 
>> wrote:
>> >
>> >Congratulations, Aman!
>> >
>> >Kind regards
>> >Arina
>> >
>> >> On Jun 23, 2017, at 10:37 PM, Abhishek Girish 
>> wrote:
>> >>
>> >> Congratulations Aman!
>> >>
>> >>> On Fri, Jun 23, 2017 at 12:06 PM, Parth Chandra 
>> wrote:
>> >>>
>> >>> Hello Drill developers,
>> >>>
>> >>>   I'm very pleased to announce that the Drill PMC has voted to elect
>> Aman
>> >>> Sinha
>> >>> as the new PMC chair of Apache Drill. Please join me in congratulating
>> >>> Aman!
>> >>>
>> >>>   Thanks,
>> >>>
>> >>> Parth
>> >>>
>> >
>> >
>>
>>

Re: FindHardDistributionScans throws a NPE while visiting a TableScan

[Muhammad Gelbana]

With pleasure. I can't successfully run all test cases
 ATM. When I overcome
that, I'll push the fix.

*-*
*Muhammad Gelbana*
http://www.linkedin.com/in/mgelbana

On Wed, Jun 21, 2017 at 1:47 PM, Khurram Faraaz  wrote:

> Muhammad, please create a pull request and someone will review you code,
> ensure that existing unit tests don't fail due to your changes.
>
>
> Thanks,
>
> Khurram
>
> 
> From: Muhammad Gelbana 
> Sent: Wednesday, June 21, 2017 4:11:41 PM
> To: dev@drill.apache.org
> Subject: Re: FindHardDistributionScans throws a NPE while visiting a
> TableScan
>
> This has been bugging me for sometime, and I've only solved it after
> starting this thread !
>
> I solved this by overriding the
> *org.apache.calcite.rel.AbstractRelNode.accept(RelShuttle)* method for the
> relational node(s) containing *JdbcTableScan* to avoid this.
>
> @Override
> > public RelNode accept(RelShuttle shuttle) {
> >
> > if(shuttle.getClass().getName().equals("org.apache.drill.
> exec.planner.sql.handlers.FindHardDistributionScans")){
> > return this;
> > }
> > return super.accept(shuttle);
> > }
>
>
> If someone finds this introducing another bug, please tell me about it.
>
>
> *-*
> *Muhammad Gelbana*
> http://www.linkedin.com/in/mgelbana
>
> On Tue, Jun 20, 2017 at 2:13 AM, Jinfeng Ni  wrote:
>
> > unwrap() essentially is doing a cast.  If it returns null for
> > unwrap(DrillTranslatableTable.class) or unwrap(DrillTable.class), it
> means
> > the table associate with this TableScan does not implement either
> > interface. My suspicion is  JDBC storage plugin returns JdbcTable [1],
> > unlikely other storage plugin which returns an instance implementing
> > DrillTable.
> >
> > This seems to indicate FindHardDistributionScans could not be used to
> > non-DrillTable. I'm not sure if that's the intention of that code,
> though.
> >
> > 1.
> > https://github.com/apache/calcite/blob/master/core/src/
> > main/java/org/apache/calcite/adapter/jdbc/JdbcSchema.java#L233-L234
> >
> > On Mon, Jun 19, 2017 at 2:19 PM, Muhammad Gelbana 
> > wrote:
> >
> > > Everyone,
> > >
> > > I made a copy of the Jdbc plugin and made modifications to it by
> adding a
> > > few rules. None of the modification I made or the rules I wrote should
> > have
> > > anything extra to do with handling the following SQL query
> > >
> > > SELECT * FROM incorta.SALES.SALES SALES WHERE 1 = 2 LIMIT 1
> > >
> > >
> > > I know the query is useless, but I need to to know how to fix the
> > following
> > > error thrown while handling this query. This is the final query plan:
> > >
> > > DEBUG o.a.d.e.p.s.h.DefaultSqlHandler - HEP_BOTTOM_UP:Convert SUM to
> > $SUM0
> > > > (0ms):
> > > > DrillLimitRel(*fetch=[1]*): rowcount = 1.0, cumulative cost = {201.0
> > > > rows, 205.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2653
> > > >   DrillLimitRel(*offset=[0], fetch=[0]*): rowcount = 1.0, cumulative
> > cost
> > > > = {200.0 rows, 201.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2651
> > > > GelbanaJdbcDrel: rowcount = 100.0, cumulative cost = {200.0 rows,
> > > > 201.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2649
> > > >   JdbcTableScan(table=[[gelbana, SALES, SALES]]): rowcount =
> 100.0,
> > > > cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0
> > > memory},
> > > > id = 2572
> > >
> > >
> > > This is the throw error stacktrace
> > >
> > > [Error Id: 83ea094a-db24-4d6d-bf0d-271db26db933 on 640fb7ebbd1a:31010]
> > > at
> > > org.apache.drill.common.exceptions.UserException$
> > > Builder.build(UserException.java:543)
> > > ~[drill-common-1.9.0.jar:1.9.0]
> > > at
> > > org.apache.drill.exec.work.foreman.Foreman$ForemanResult.
> > > close(Foreman.java:825)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.moveToState(
> > > Foreman.java:935)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> > > [drill-java-exec-1.9.0.jar:1.9.0]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1142)
> > > [na:1.8.0_131]
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:617)
> > > [na:1.8.0_131]
> > > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> > > Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> > Unexpected
> > > exception during fragment initialization: null
> > > ... 4 common frames omitted
> > > *Caused by: java.lang.NullPointerException: null*
> > > at
> > > org.apache.drill.exec.planner.sql.handlers.FindHardDistributionScans.
> > > visit(FindHardDistributionScans.java:55)
> > > ~[drill-java-exec-1.9.0.jar:1.9.0]
> > > at org.apache.calcite.rel.core.TableScan.accept(TableScan.java:166)
> > > ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> > > at org.apache.calcite.rel.RelShuttle

Re: Drill test cases failures after a fresh clone

[Muhammad Gelbana]

Thank you all for your replies. Here is the jira issue as requested.
https://issues.apache.org/jira/browse/DRILL-5606

Please tell me right away if you need more information.

*-*
*Muhammad Gelbana*
http://www.linkedin.com/in/mgelbana

On Fri, Jun 23, 2017 at 7:52 PM, Paul Rogers  wrote:

> Hi Muhammad,
>
> The tests have worked for me. That said, we do want them to work “out of
> the box” for everyone, so let’s see if we can track down the issues you are
> having.
>
> TestClassTransformation.testCompilationNoDebug is a timeout issue.
> Sometimes this just means that tests run slowly, perhaps due to other load
> on the system. When that happens, JUnit fails them due to a hard-coded,
> fixed timeout. Was there other load on our machine while running the tests?
>
> The others look like functional issues. We need more details since the
> summary messages are truncated. Did you happen to capture the entire test
> output? If so, you can find the full details for each test so we can see
> what’s happening.
>
> Otherwise, you can run the test individually:
>
> > cd exec/java-exec
> > mvn surefire:test -Dtest=TestName
>
> Try this with the four tests that failed (in one case, there were two
> failures in the same test class.)
>
> You can also run the four tests in our IDE. Now, sometimes I’ve found that
> tests work fine in one of the three cases, but fail in others. (Maven set
> up a bunch of system properties, for example. Maven runs tests in a single
> JVM, which can occasionally cause problems, but your IDE will launch a new
> JVM for each run…)
>
> Once we see what you find, we can start to figure out what is going on.
>
> To make this easier to track, I’d suggest opening a JIRA ticket to report
> the issue, then posting your test results there. We’ll need that JIRA if we
> have to make any fixes to tests.
>
> Thanks,
>
> - Paul
>
> > On Jun 23, 2017, at 4:35 AM, Muhammad Gelbana 
> wrote:
> >
> > I intend to create a pull request, so I forked Drill on github and cloned
> > the repository locally on my machine.
> >
> > I built the project by running
> > *mvn clean install -DskipTests*
> >
> > Then I tried running the tests by running
> > *mvn clean install*
> >
> > Then I started facing test failures. The first time, the following tests
> > failed:
> >
> > Tests in error:
> >>
> >> TestClassTransformation.testJDKClassCompiler:73->
> compilationInnerClass:118
> >> »  ...
> >>  TestClassTransformation.testCompilationNoDebug:88 »  test timed out
> >> after 5000...
> >> *  TestCastFunctions.testToDateForTimeStamp:79 »  at position 0 column
> >> '`col`' mi...*
> >>  TestNewMathFunctions.testTrigoMathFunc:111->runTest:85 »
> ExecutionSetup
> >> Failur...
> >> *  TestNewDateFunctions.testIsDate:61 »  After matching 0 records, did
> not
> >> find e...*
> >>
> >
> > I thought to run the tests again because I found a test case failing due
> to
> > a timeout, so after the second run, the following tests failed *again*:
> >
> > Tests in error:
> >> *  TestCastFunctions.testToDateForTimeStamp:79 »  at position 0 column
> >> '`col`' mi...*
> >> *  TestNewDateFunctions.testIsDate:61 »  After matching 0 records, did
> not
> >> find e...*
> >
> >
> > Running tests take around half an hour and I was wondering if there is
> > something I need to setup or configure before I can run the tests and get
> > 100% success rate since I haven't modified anything yet.
> >
> > *-*
> > *Muhammad Gelbana*
> > http://www.linkedin.com/in/mgelbana
>
>

[jira] [Created] (DRILL-5606) Some tests fail after creating a fresh clone

[Muhammad Gelbana (JIRA)]

Muhammad Gelbana created DRILL-5606:
---

 Summary: Some tests fail after creating a fresh clone
 Key: DRILL-5606
 URL: https://issues.apache.org/jira/browse/DRILL-5606
 Project: Apache Drill
  Issue Type: Bug
  Components: Tools, Build & Test
 Environment: {noformat}
$ uname -a
Linux mg-mate 4.4.0-81-generic #104-Ubuntu SMP Wed Jun 14 08:17:06 UTC 2017 
x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04.2 LTS
Release:16.04
Codename:   xenial

$ java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-0ubuntu1.16.04.2-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
{noformat}

Environment variables JAVA_HOME, JRE_HOME, JDK_HOME aren't configured. Java 
executable is found as the PATH environment variables links to it. I can 
provide more details if needed.
Reporter: Muhammad Gelbana


I cloned Drill from Github using this url: 
[https://github.com/apache/drill.git] and I didn't change the branch 
afterwards, so I'm using *master*.

Afterwards, I ran the following command

{noformat}
mvn clean install
{noformat}

I attached the full log but here is a snippet indicating the failing tests:
{noformat}
Failed tests: 
  TestExtendedTypes.checkReadWriteExtended:60 expected:<...ateDay" : 
"1997-07-1[6"
  },
  "drill_timestamp" : {
"$date" : "2009-02-23T08:00:00.000Z"
  },
  "time" : {
"$time" : "19:20:30.450Z"
  },
  "interval" : {
"$interval" : "PT26.400S"
  },
  "integer" : {
"$numberLong" : 4
  },
  "inner" : {
"bin" : {
  "$binary" : "ZHJpbGw="
},
"drill_date" : {
  "$dateDay" : "1997-07-16]"
},
"drill_...> but was:<...ateDay" : "1997-07-1[5"
  },
  "drill_timestamp" : {
"$date" : "2009-02-23T08:00:00.000Z"
  },
  "time" : {
"$time" : "19:20:30.450Z"
  },
  "interval" : {
"$interval" : "PT26.400S"
  },
  "integer" : {
"$numberLong" : 4
  },
  "inner" : {
"bin" : {
  "$binary" : "ZHJpbGw="
},
"drill_date" : {
  "$dateDay" : "1997-07-15]"
},
"drill_...>

Tests in error: 
  TestCastFunctions.testToDateForTimeStamp:79 »  at position 0 column '`col`' 
mi...
  TestNewDateFunctions.testIsDate:61 »  After matching 0 records, did not find 
e...

Tests run: 2128, Failures: 1, Errors: 2, Skipped: 139

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Drill Root POM .. SUCCESS [ 19.805 s]
[INFO] tools/Parent Pom ... SUCCESS [  0.605 s]
[INFO] tools/freemarker codegen tooling ... SUCCESS [  7.077 s]
[INFO] Drill Protocol . SUCCESS [  7.959 s]
[INFO] Common (Logical Plan, Base expressions)  SUCCESS [  7.734 s]
[INFO] Logical Plan, Base expressions . SUCCESS [  8.099 s]
[INFO] exec/Parent Pom  SUCCESS [  0.575 s]
[INFO] exec/memory/Parent Pom . SUCCESS [  0.513 s]
[INFO] exec/memory/base ... SUCCESS [  4.666 s]
[INFO] exec/rpc ... SUCCESS [  2.684 s]
[INFO] exec/Vectors ... SUCCESS [01:11 min]
[INFO] contrib/Parent Pom . SUCCESS [  0.547 s]
[INFO] contrib/data/Parent Pom  SUCCESS [  0.496 s]
[INFO] contrib/data/tpch-sample-data .. SUCCESS [  2.698 s]
[INFO] exec/Java Execution Engine . FAILURE [19:09 min]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)