[GitHub] drill issue #972: DRILL-5838: Fix MaprDB filter pushdown for the case of nes...

[vvysotskyi]

Github user vvysotskyi commented on the issue:

https://github.com/apache/drill/pull/972
  
@paul-rogers, Thanks for the so informative comments, I completely agree 
with the convention that you have proposed. 
The problem which this PR is addressed, partially connected to the third 
point:

> When calling back out to the external system, the interface (plugin) must 
convert from Drill format back out to the external system format.

MaprDB uses `FieldPath` class to represent the field. Since the string, 
that is received in the method `JsonConditionBuilder.createJsonScanSpec()` from 
`SchemaPath` instance will be parsed using `org.ojai.FieldPath.parseFrom()` 
method, to avoid any ambiguousness, I converted `SchemaPath` to `FieldPath` and 
then called `FieldPath.asPathString()`. 


---

Volunteer Drill Apache

[Prabu T.S.]

Hi team,

I am interested to volunteer for this project.

Thanks
Prabu

[GitHub] drill issue #990: DRILL-5872: Workaround for invalid cost in physical plans

[priteshm]

Github user priteshm commented on the issue:

https://github.com/apache/drill/pull/990
  
@amansinha100  or @arina-ielchiieva  can you please review this?


---

[GitHub] drill pull request #990: DRILL-5872: Workaround for invalid cost in physical...

[arina-ielchiieva]

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/990#discussion_r144603055
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/PhysicalPlan.java 
---
@@ -94,6 +94,14 @@ public double totalCost() {
 for (final PhysicalOperator ops : getSortedOperators()) {
   totalCost += ops.getCost();
 }
+
+// As it turns out, sometimes the total cost can be out of range.
+// This throws off other code, so clamp the cost at the maximum
+// double value.
+
+if (Double.isNaN(totalCost)) {
+  totalCost = Double.MAX_VALUE;
--- End diff --

May be better to replace with 0 or negative number, like -1 (like the 
second option better). Won't be max double value confusing in this case? It's 
just nan is not a number and replace it with double max value.


---

[GitHub] drill issue #990: DRILL-5872: Workaround for invalid cost in physical plans

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/990
  
Is there a Jira for storage plugin fix?


---

[jira] [Created] (DRILL-5876) Remove netty-tcnative inclusion in java-exec/pom.xml

[Parth Chandra (JIRA)]

Parth Chandra created DRILL-5876:


 Summary: Remove netty-tcnative inclusion in java-exec/pom.xml
 Key: DRILL-5876
 URL: https://issues.apache.org/jira/browse/DRILL-5876
 Project: Apache Drill
  Issue Type: Bug
Reporter: Parth Chandra


The inclusion of netty-tcnative is causing all kinds of problems. The os 
specific classifier  required is determined by a maven extension which in turn 
requires an additional eclipse plugin. The eclipse plugin has a problem that 
may corrupt the current workspace.
It is safe to not include the dependency since it is required only at runtime. 
The only case in which this is required is when a developer has to debug 
SSL/OpenSSL issues in the Java client or the server when launching from within 
an IDE. In this case, the dependency can be enabled by uncommenting the 
relevant lines in the pom file.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] drill pull request #991: DRILL-5876: Remove netty-tcnative dependency from j...

[parthchandra]

GitHub user parthchandra opened a pull request:

https://github.com/apache/drill/pull/991

DRILL-5876: Remove netty-tcnative dependency from java-exec/pom.xml

Commented out the netty-tcnative dependency. Tested build on command line, 
intellij. Ran unit tests.
@paul-rogers, @vrozov  Can you guys please review?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/parthchandra/drill DRILL-5876

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/991.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #991


commit 9bf86429958f2e04843dc91322f3f75d9e34fde3
Author: Parth Chandra 
Date:   2017-10-13T17:29:31Z

DRILL-5876: Remove netty-tcnative dependency from java-exec/pom.xml




---

RE: Newbie: Join queries in MySQL.

[Kunal Khatua]

You could try creating simple views (with different column names) on those 
tables. That way, you don’t have to try changing the column names in the table 
itself. 

-Original Message-
From: Charuta Rajopadhye [mailto:charuta...@gmail.com] 
Sent: Thursday, October 12, 2017 11:28 PM
To: dev@drill.apache.org
Subject: Re: Newbie: Join queries in MySQL.

Hi Timothy,

Thank you very much for your reply.

Regards,
Charuta

On Fri, Oct 13, 2017 at 9:53 AM, Timothy Farkas  wrote:

> Hi Charuta,
>
> This is a known issue that other people have been bumping into as 
> well. As of now there is no work around other than changing the column 
> names in your tables. I've filed these tickets regarding the issue but 
> haven't gotten around to working on them yet:
>
> https://issues.apache.org/jira/browse/DRILL-5713
> https://issues.apache.org/jira/browse/DRILL-4211
>
> Maybe someone else has some bandwidth to pick them up?
>
> Thanks,
> Tim
>
> 
> From: Charuta Rajopadhye 
> Sent: Thursday, October 12, 2017 2:16:10 AM
> To: dev@drill.apache.org
> Subject: Newbie: Join queries in MySQL.
>
> Hi Team,
>
>
> I have created a configuration for JDBC storage plugin in Drill, for a 
> MySQL database.
>
> I am trying to fire join query on 2 tables. Both my tables have a 
> column called 'id'/
>
> Table defination:
>
> CREATE TABLE `simpleone` (
>
>  `id` int(11) NOT NULL AUTO_INCREMENT,
>
>  `name` text NOT NULL,
>
>  PRIMARY KEY (`id`)
>
> ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1
>
> CREATE TABLE `simpletwo` (
>
>  `id` int(11) NOT NULL AUTO_INCREMENT,
>
>  `city` text NOT NULL,
>
>  PRIMARY KEY (`id`)
>
> ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1
>
>
> because of the redundant ‘id’ column, i get values from one ‘id’ 
> column as NULL.
>
> I tried using a few permutations to fire my queries:
>
> 0: jdbc:drill:zk=local>* select * from 
> **pluginjdbc**.drillTest.simpleone,
> *
> *pluginjdbc**.drillTest.simpletwo where simpleone.id 
>  = simpletwo.id ;*
>
> +-+---+---++
>
> | id  |   name|  *id0*  |  city  |
>
> +-+---+---++
>
> | 1   | J. D Salinger | *null*  | New York City  |
>
> | 2   | Charlotte Bronte  | *null*  | Thornton   |
>
> +-+---+---++
>
> 2 rows selected (0.224 seconds)
>
>
> 0: jdbc:drill:zk=local> *select tb1.id , tb2.id 
>  from pluginjdbc.drillTest.simpleone as tb1, 
> pluginjdbc.drillTest.simpletwo as tb2 where tb1.id  = 
> tb2.id
> ;*
>
> +-+---+
>
> | id  |  id0  |
>
> +-+---+
>
> | 1   | null  |
>
> | 2   | null  |
>
> +-+---+
>
> 2 rows selected (0.368 seconds)
>
>
> 0: jdbc:drill:zk=local> *select tb1.id  as col1, tb2.id 
>  as col2 from pluginjdbc.drillTest.simpleone as tb1, 
> pluginjdbc.drillTest.simpletwo as tb2 where tb1.id  = 
> tb2.id
> ;*
>
> +---+---+
>
> | col1  | col2  |
>
> +---+---+
>
> | 1 | null  |
>
> | 2 | null  |
>
> +---+---+
>
> 2 rows selected (0.236 seconds)
>
>
> Could somebody please guide me through this? Please let me know if i 
> am missing something.
>
> Thanks,
>
> Charuta
>

[GitHub] drill pull request #991: DRILL-5876: Remove netty-tcnative dependency from j...

[vrozov]

Github user vrozov commented on a diff in the pull request:

https://github.com/apache/drill/pull/991#discussion_r144624216
  
--- Diff: exec/java-exec/pom.xml ---
@@ -701,18 +707,21 @@
   
-  
-kr.motd.maven
-os-maven-plugin
-1.5.0.Final
-  
-
+  
--- End diff --

Can it be in a separate profile (openssl) that is disabled by default?


---

Re: Volunteer Drill Apache

[Saurabh Mahapatra]

Anything we can offer as a getting started guide for developers who want to 
contribute.

Prabhu: can you introduce yourself and tell us a little about your background?

Best,
Saurabh

Sent from my iPhone



> On Oct 13, 2017, at 7:08 AM, Prabu T.S.  wrote:
> 
> Hi team,
> 
> I am interested to volunteer for this project.
> 
> Thanks
> Prabu

Re: Volunteer Drill Apache

[Prabu T.S.]

Hi Saurabh,

I have 9 years of experience in software development and infrastructure. 7
years worked as Production & UAT application and infrastructure support for
Java enterprise banking application and initial two years as Java/j2ee
developer. I would like to enhance my development skills.

Below are my skill set.

*Programming **Languages**: *JAVA,Python.

*Database Query Languages*: Oracle SQL, PL-SQL.

*Scripting Languages*:JavaScript, VB Script,Unix shell scripting,Perl.

*J2EE Technology*: JSP, Servlets, JDBC,JMS

*M**essaging **M**iddleware** : *IBM MQ

*Framework*: Struts, BEA Weblogic Portal, Websphere Portal,Spring,Hibernate.

*Tools*:IntelliJ,Eclipse,RationalRose,Toad,Attivio-Search-tool,QTP,Autosys.
,I-Sprint,Sql Developer,SVN,GitHub,Tivoli,SSH Tectia, putty,SharePoint,NDM,

*Build tool*:Junit*.*

*Performance Monitoring Tool: *HP Diagnostics, RT View, Gomez, TeaLeaf

*Deployment Tool:* Udeploy

*Ticketing Tools*: ServiceNow, JIRA, Resolve-It.

*Web &Application Server**: *IBM HTTP Server (IHS)*,*iPlanet*,*Tomcat,
Jetty,Weblogic ,Websphere.

*Database*: Oracle 9i/10g

*Content Management System*: Documentum, Teamsite.

*Operating system**:* Windows, UNIX (Solaris,AIX,Linux).

Thanks
Prabu







On Fri, Oct 13, 2017 at 2:54 PM, Saurabh Mahapatra <
saurabhmahapatr...@gmail.com> wrote:

> Anything we can offer as a getting started guide for developers who want
> to contribute.
>
> Prabhu: can you introduce yourself and tell us a little about your
> background?
>
> Best,
> Saurabh
>
> Sent from my iPhone
>
>
>
> > On Oct 13, 2017, at 7:08 AM, Prabu T.S.  wrote:
> >
> > Hi team,
> >
> > I am interested to volunteer for this project.
> >
> > Thanks
> > Prabu
>

[GitHub] drill pull request #991: DRILL-5876: Remove netty-tcnative dependency from j...

[parthchandra]

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/991#discussion_r144651132
  
--- Diff: exec/java-exec/pom.xml ---
@@ -701,18 +707,21 @@
   
-  
-kr.motd.maven
-os-maven-plugin
-1.5.0.Final
-  
-
+  
--- End diff --

How would that work in conjunction with the main pom's profiles? I would 
want to have both the (apache | mapr | cdh | hdp)  profile and the openssl 
profile enabled.


---

[GitHub] drill pull request #991: DRILL-5876: Remove netty-tcnative dependency from j...

[vrozov]

Github user vrozov commented on a diff in the pull request:

https://github.com/apache/drill/pull/991#discussion_r144659314
  
--- Diff: exec/java-exec/pom.xml ---
@@ -701,18 +707,21 @@
   
-  
-kr.motd.maven
-os-maven-plugin
-1.5.0.Final
-  
-
+  
--- End diff --

For example `-P apache,openssl'. Another option will be to enable openssl 
profile based on a property.


---

[GitHub] drill pull request #991: DRILL-5876: Remove netty-tcnative dependency from j...

[parthchandra]

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/991#discussion_r144666062
  
--- Diff: exec/java-exec/pom.xml ---
@@ -701,18 +707,21 @@
   
-  
-kr.motd.maven
-os-maven-plugin
-1.5.0.Final
-  
-
+  
--- End diff --

How about we forget the whole os dependent stuff and put in all the 
variants (there are only four) and make the scope for all of them `provided`?
Since netty is internally loading the correct jar based on the os, all we 
need to do is make sure that the jar files are in the classpath.


---

[GitHub] drill pull request #992: DRILL-5873: (C++ Client) Improve SASL error reporti...

[parthchandra]

GitHub user parthchandra opened a pull request:

https://github.com/apache/drill/pull/992

DRILL-5873: (C++ Client) Improve SASL error reporting.

@sohami what do you think? I overrode the SASL_NOMECH error message. The 
message from the SASL library is very unfriendly.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/parthchandra/drill DRILL-5873

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/992.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #992


commit 31e71147c37a7e37348dfae323d894114204c8dd
Author: Parth Chandra 
Date:   2017-10-13T18:00:31Z

DRILL-5873: (C++ Client) Improve SASL error reporting.




---

[GitHub] drill issue #992: DRILL-5873: (C++ Client) Improve SASL error reporting.

[superbstreak]

Github user superbstreak commented on the issue:

https://github.com/apache/drill/pull/992
  
@parthchandra Thanks for adding this!


---

[GitHub] drill pull request #993: DRILL-5874: NPE in AnonWebUserConnection.cleanupSes...

[sohami]

GitHub user sohami opened a pull request:

https://github.com/apache/drill/pull/993

DRILL-5874: NPE in AnonWebUserConnection.cleanupSession()



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sohami/drill DRILL-5874

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/993.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #993


commit f549f07bcc56f8978fa5745c52ffb4e68f67f419
Author: Sorabh Hamirwasia 
Date:   2017-10-11T22:57:21Z

DRILL-5874: NPE in AnonWebUserConnection.cleanupSession()




---

[GitHub] drill pull request #994: Merge from latest

[priteshm]

GitHub user priteshm opened a pull request:

https://github.com/apache/drill/pull/994

Merge from latest



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/priteshm/drill master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/994.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #994


commit c75dc4904d3ecb734f9369db6a9b4011956fb07c
Author: Paul Rogers 
Date:   2017-03-10T23:56:18Z

DRILL-5344: External sort priority queue copier fails with an empty batch

Unit tests showed that the “priority queue copier” does not handle an
empty batch. This has not been an issue because code elsewhere in the
sort specifically works around this issue. This fix resolves the issue
at the source to avoid the need for future work-arounds.

closes #778

commit ee15632df3a869b3cc1063f882356d7eaab5b9f7
Author: Paul Rogers 
Date:   2017-03-14T23:18:24Z

DRILL-5323: Test tools for row sets

Provide test tools to create, populate and compare row sets

To simplify tests, we need a TestRowSet concept that wraps a
VectorContainer and provides easy ways to:

- Define a schema for the row set.
- Create a set of vectors that implement the schema.
- Populate the row set with test data via code.
- Add an SV2 to the row set.
- Pass the row set to operator components (such as generated code
blocks.)
- Examine the contents of a row set
- Compare the results of the operation with an expected result set.
- Dispose of the underling direct memory when work is done.

This code builds on that in DRILL-5324 to provide a complete row set
API. See DRILL-5318 for the spec.

Note: this code can be reviewed as-is, but cannot be committed until
after DRILL-5324 is committed: this code has compile-time dependencies
on that code. This PR will be rebased once DRILL-5324 is pulled into
master.

Handles maps and intervals

The row set schema is refined to provide two forms of schema. A
physical schema shows the nested structure of the data with maps
expanding into their contents.

Updates the row set schema builder to easily build a schema with maps.

An access schema shows the row “flattened” to include just scalar
(non-map) columns, with all columns at a single level, with dotted
names identifying nested fields. This form makes for very simple access.

Then, provides tools for reading and writing batches with maps by
presenting the flattened view to the row reader and writer.

HyperVectors have a very complex structure for maps. The hyper row set
implementation takes a first crack at mapping that structure into the
standardized row set format.

Also provides a handy way to set an INTERVAL column from an int. There
is no good mapping from an int to an interval, so an arbitrary
convention is used. This convention is not generally useful, but is
very handy for quickly generating test data.

As before, this is a partial PR. The code here still depends on
DRILL-5324 to provide the column accessors needed by the row reader and
writer.

All this code is getting rather complex, so this commit includes a unit
test of the schema and row set code.

Revisions to support arrays

Arrays require a somewhat different API. Refactored to allow arrays to
appear as a field type.

While refactoring, moved interfaces to more logical locations.

Added more comments.

Rejiggered the row set schema to provide both a physical and flattened
(access) schema, both driven from the original batch schema.

Pushed some accessor and writer classes into the accessor layer.

Added tests for arrays.

Also added more comments where needed.

Moved tests to DRILL-5318

The test classes previously here depend on the new “operator fixture”.
To provide a non-cyclic checkin order, moved the tests to the PR with
the fixtures so that this PR is clear of dependencies. The tests were
reviewed in the context of DRILL-5318.

Also pulls in batch sizer support for map fields which are required by
the tests.

closes #785

commit 51ce7843dfd5d9979dcd205df4797869c12dc3a2
Author: Paul Rogers 
Date:   2017-03-14T23:18:24Z

DRILL-5318: Sub-operator test fixture

This commit depends on:

* DRILL-5323

This PR cannot be accepted (or built) until the above are pulled and
this PR is rebased on top of them. The PR is issued now so that reviews
can be done in parallel.

Provides the following:

* A new OperatorFixture to set up all the objects needed to test at t

[GitHub] drill pull request #994: Merge from latest

[priteshm]

Github user priteshm closed the pull request at:

https://github.com/apache/drill/pull/994


---