[GitHub] drill pull request: DRILL-4690: CORS in REST API
Github user PythonicNinja commented on the pull request: https://github.com/apache/drill/pull/507#issuecomment-222073117 I have updated PR according to your and laurentgo ideas. @hnfgns: Can you check second round of review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (DRILL-4697) Editing of multi-line queries in sqlline
Matt Keranen created DRILL-4697: --- Summary: Editing of multi-line queries in sqlline Key: DRILL-4697 URL: https://issues.apache.org/jira/browse/DRILL-4697 Project: Apache Drill Issue Type: Wish Components: Client - CLI Affects Versions: 1.6.0 Reporter: Matt Keranen Priority: Minor Being used to the PostgreSQL psql client, editing multiline queries in sqlline seems difficult. Each line of an unterminated (semicolon) query appears to be a separate history entry, thus uneditable. The ability to up-arrow to bring the entire previous query would be very helpful, or the option to send the query text to an external shell editor (Vi). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Manta Object Store Support
It should "just work". Nothing in the logs? -- Jacques Nadeau CTO and Co-Founder, Dremio On Wed, May 25, 2016 at 7:01 PM, Elijah Zupancic wrote: > Hi Tomer, > > Thanks for your advice about creating a Hadoop FileSystem implementation. > I just finished a prototypical implementation of a Hadoop file system for > Manta: https://github.com/dekobon/hadoop-manta < > https://github.com/dekobon/hadoop-manta> > > I see the example for enabling S3 with Apache Drill and I’ve verified that > it works. However, when I attempt to replicate the configuration of S3 for > Manta, I’m unable to get the Hadoop FileSystem driver to load. I’ve > verified that the FileSystem driver works in Hadoop by checking all of the > hdfs dfs -* commands and I’ve got a fair bit of automated testing around it. > > What’s the magic to get it turned on with Drill? Do I need to do something > to make the jar load other than copy it into jars/3rdparty? Right now, I’m > just testing in drill-embedded for what it is worth. > > Thanks, > Elijah Zupancic > > > On May 6, 2016, at 4:11 PM, Tomer Shiran wrote: > > > > Does Manta have a Hadoop FileSystem API implementation? That's what Drill > > uses for S3, HDFS, MapR-FS, Azure Blob Storage, etc. You could > potentially > > write a Drill storage plugin, but you get a lot for free if you already > > have the file system implementation. > > > > On Fri, May 6, 2016 at 9:43 AM, Elijah Zupancic > wrote: > > > >> I'm trying to get started contributing to Apache Drill. I've got the > >> project checked out and it is building to my satisfaction. Right now, > I'm > >> trying to add support for the open source object store Manta ( > >> https://github.com/joyent/manta). I thought that this would be a good > >> learning project. > >> > >> Initially, I want to add support in the same way that S3 has support. > >> However, I can't seem to find a reference to the S3 storage driver in > the > >> code base. Is the s3 storage driver part of a different project? How > would > >> you suggest that I get started? > >> > >> Thank you, > >> Elijah Zupancic > >> > >
Re: are random numbers broken?
There are two issues, one is we use hive's deterministic flag in an opposite way, another one is when drill visit an expression, we take a previous found holder, but that is not right if this function is nondeterministic. A JIRA was created and a pull request of fix was posted: https://issues.apache.org/jira/browse/DRILL-4618 https://github.com/apache/drill/pull/509 On Mon, Apr 11, 2016 at 5:53 PM, Jinfeng Ni wrote: > Sounds like there are two issues. > > 1. random() delivers the same value when used multiple times in each > returned row. This seems to be caused by a bug in common subexpression > (CSE) reduction, which identify the two appearances of random() are > CSE. As a result, the generated code share the only instance of > random(). > > 2. rand(). Looks like Drill does not recognize hive's rand() as > 'isRandom'=true. As such, rand() is regarded as a constant expression > in generated code. > > Both of them are bugs that we should fix. >
Re: Contributing to Drill?
FYI ... Bridget has updated the page to reflect the current review process. It should just be a replacement of the out-dated material with the process that's documented in a Google doc. But if we've mis-represented something, please let us know. Thanks. -- Zelaine On Tue, Apr 12, 2016 at 3:35 PM, Jacques Nadeau wrote: > Can a committer respond to this request? > > thanks, > Jacques > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Thu, Apr 7, 2016 at 9:07 AM, Zelaine Fong wrote: > > > A Drill user is eager to contribute a new XML plugin, and was planning to > > follow the instructions noted at > > https://drill.apache.org/docs/apache-drill-contribution-guidelines/. > > However, my understanding is we're no longer following that process. > > > > Can one of the dev commiters update that Web page with the latest > > guidelines? > > > > Thanks. > > > > -- Zelaine > > >
Re: Issue with Drill installation
Hello!
Forgive me if you've already done so, but as a sanity check: Have you set
up the Windows environment variables as per this article?
http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
It's a little tricky, notice how for the 'Path' variable you need the
trailing 'bin' directory, but you don't need it for the JAVA_HOME one.
Best,
Nathan Griffith
Technical Writer
Dremio
On Wed, May 25, 2016 at 2:36 PM, Doddi, Lakshmi <
lakshmi.do...@cvscaremark.com> wrote:
> Hi,
>
> I am trying to install apache drill 1.6 on windows 7 laptop.
>
> This is how my drill-override.conf looks like :
>
> drill.exec: {
> cluster-id: "drillbits1",
> zk.connect: "localhost:2181",
> sys.store.provider.local.path="C:/data/drill"
> }
>
> When I issue the command I see the below error and I am not able to open
> up the drill console using http://localhost:8047
>
> Command: sqlline.bat -u "jdbc:drill:zk=local"
>
> Output:
> C:\DEV\drill\apache-drill-1.6.0>bin\sqlline.bat -u "jdbc:drill:zk=local"
> DRILL_ARGS - " -u jdbc:drill:zk=local"
> HADOOP_HOME not detected...
> HBASE_HOME not detected...
> Calculating Drill classpath...
> 16:33:42.894 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - Scanning
> classpath for resources with pathname "drill-module.conf".
> 16:33:42.903 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-common-1.6.0.jar!/drill-module.conf.
> 16:33:42.903 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-gis-1.6.0.jar!/drill-module.conf.
> 16:33:42.903 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-hive-exec-shaded-1.6.0.jar!/drill-module.conf.
> 16:33:42.903 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-java-exec-1.6.0.jar!/drill-module.conf.
> 16:33:42.904 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-jdbc-storage-1.6.0.jar!/drill-module.conf.
> 16:33:42.904 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-kudu-storage-1.6.0.jar!/drill-module.conf.
> 16:33:42.904 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-logical-1.6.0.jar!/drill-module.conf.
> 16:33:42.904 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-memory-base-1.6.0.jar!/drill-module.conf.
> 16:33:42.905 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-mongo-storage-1.6.0.jar!/drill-module.conf.
> 16:33:42.905 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-storage-hbase-1.6.0.jar!/drill-module.conf.
> 16:33:42.905 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-storage-hive-core-1.6.0.jar!/drill-module.conf.
> 16:33:42.906 [main] DEBUG o.a.d.c.scanner.ClassPathScanner - - collected
> resource URL
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/3rdparty/drill-memory-base-1.6.0.jar!/drill-module.conf.
> 16:33:42.936 [main] INFO o.a.drill.common.config.DrillConfig -
> Configuration and plugin file(s) identified in 151ms.
> Base Configuration:
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-common-1.6.0.jar!/drill-default.conf
>
> Intermediate Configuration and Plugin files, in order of precedence:
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-mongo-storage-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-logical-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-java-exec-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-gis-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-storage-hive-core-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-common-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-kudu-storage-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-storage-hbase-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/drill-memory-base-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/3rdparty/drill-memory-base-1.6.0.jar!/drill-module.conf
> -
> jar:file:/C:/DEV/drill/apache-drill-1.6.0/jars/d
[jira] [Created] (DRILL-4696) select four table inner join result.Waiting for a long time after drill report error java.lang.OutOfMemoryError: Java heap space
david_hudavy created DRILL-4696: --- Summary: select four table inner join result.Waiting for a long time after drill report error java.lang.OutOfMemoryError: Java heap space Key: DRILL-4696 URL: https://issues.apache.org/jira/browse/DRILL-4696 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Affects Versions: 1.6.0 Environment: Test Environment: SUSE Linux Enterprise Server 11 SP3 (x86_64) cluster MySQL 5.7.11 Enterprise Server - Advanced Edition Drill cluster Reporter: david_hudavy Test Environment: cluster 10-3 MySQL 5.7.11 Enterprise Server - Advanced Edition Drill cluster Test Scope: select performance of huge table(30M records). MySQL table: Eps Eps_EpsImei mscIden EpsStatic inner join (Four table each have 30M records) -- four table inner join: (take time Drill Crash) 0: jdbc:drill:zk=SC-1:6181,SC-2:6181,PL-3:618> select . . . . . . . . . . . . . . . . . . . . . . .> EpsStatic.EpsProfileId, . . . . . . . . . . . . . . . . . . . . . . .> mscIden.mscId, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsMmeAddr, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsMmeRealm, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsLastInsertSent, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsMobilityNotifInfo, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsAaaAddr , . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsAaaRealm , . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsMmeRegServ, . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsHomoImsVoip , . . . . . . . . . . . . . . . . . . . . . . .> Eps.EpsUeSrVccCap, . . . . . . . . . . . . . . . . . . . . . . .> Eps_EpsImei.EpsImeiSv . . . . . . . . . . . . . . . . . . . . . . .> from mysql.user_data.Eps,mysql.user_data.Eps_EpsImei,mysql.user_data.mscIden,mysql.user_data.EpsStatic . . . . . . . . . . . . . . . . . . . . . . .> where mscIden.mscId=Eps.mscId and Eps.mscId =Eps_EpsImei.mscId and Eps_EpsImei.mscId=EpsStatic.mscId . . . . . . . . . . . . . . . . . . . . . . .> and mscIden.mscId='0'; Drill Crash 2016-05-13 09:52:35,131 [28cacd19-0f04-cbb1-b418-73a76dcd6ebe:frag:0:0] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in FragmentExecutor. java.lang.OutOfMemoryError: Java heap space at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2157) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1964) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3316) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3040) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2288) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2681) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2547) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2505) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38] at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) ~[commons-dbcp-1.4.jar:1.4] at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) ~[commons-dbcp-1.4.jar:1.4] at org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) ~[drill-jdbc-storage-1.6.0.jar:1.6.0] at org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:108) ~[drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:136) ~[drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) ~[drill-jdbc-storage-1.6.0.jar:1.6.0] at org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) ~[drill-jdbc-storage-1.6.0.jar:1.6.0] at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:146) ~[drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:
[GitHub] drill pull request: DRILL-4618: Fix hive function loader not corre...
GitHub user chunhui-shi opened a pull request: https://github.com/apache/drill/pull/509 DRILL-4618: Fix hive function loader not correctly take random flag; ⦠â¦and function visitor should not use previous function holder if this function is nondeterministic You can merge this pull request into a Git repository by running: $ git pull https://github.com/chunhui-shi/drill DRILL-4618-random Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/509.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #509 commit 81fe40f9890f1aa7a8a5893bff41b9d8dc0f9399 Author: chunhui-shi Date: 2016-05-26T06:22:01Z DRILL-4618: Fix hive function loader not correctly take random flag; and function visitor should not use previous function holder if this function is nondeterministic --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
