[jira] [Created] (DRILL-7046) Support for loading and parsing new RM config file

2019-02-20 Thread Sorabh Hamirwasia (JIRA) 
Sorabh Hamirwasia created DRILL-7046:


 Summary: Support for loading and parsing new RM config file
 Key: DRILL-7046
 URL: https://issues.apache.org/jira/browse/DRILL-7046
 Project: Apache Drill
  Issue Type: Sub-task
  Components: Execution - Flow
Affects Versions: 1.16.0
Reporter: Sorabh Hamirwasia
Assignee: Sorabh Hamirwasia


This Jira will help to add support for loading new RM specific configuration 
file if needed and also parsing it to create all the required in-memory 
objects. The details of the new configuration is defined in [Function 
Spec|[https://docs.google.com/document/d/1cX0lPLL-QzBGUwcekAPvUBgubdcNlWRdy7IG_IFt-Ok/edit?usp=sharing]|https://docs.google.com/document/d/1cX0lPLL-QzBGUwcekAPvUBgubdcNlWRdy7IG_IFt-Ok/edit?usp=sharing].]
 RM configuration file will also support override, default and distrib specific 
files. It will only be loaded when RM is enabled which will be controlled by 
*drill.exec.rm.enabled* parameter in Drill main configuration file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (DRILL-7025) Running Drill unit tests from within IntelliJ IDEA fails with FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist

2019-02-20 Thread Vladimir Sitnikov (JIRA) 


 [ 
https://issues.apache.org/jira/browse/DRILL-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Sitnikov resolved DRILL-7025.
--
Resolution: Cannot Reproduce

> Running Drill unit tests from within IntelliJ IDEA fails with 
> FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist
> ---
>
> Key: DRILL-7025
> URL: https://issues.apache.org/jira/browse/DRILL-7025
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Tools, Build  Test
>Affects Versions: 1.15.0
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> I start org.apache.drill.TestExampleQueries via regular "run tests" button, 
> and it produces the following exception:
> {noformat}
> java.lang.RuntimeException: This should not happen
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:298)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyResourceToRoot(BaseDirTestWatcher.java:223)
> at 
> org.apache.drill.TestExampleQueries.setupTestFiles(TestExampleQueries.java:42)
> Caused by: java.io.FileNotFoundException: Source 
> './src/test/resources/tpchmulti' does not exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:296)
> ... 2 more{noformat}
> In fact IDEA creates "run configuration" that uses 
> "/Users/vladimirsitnikov/Documents/work/drill" as "Working directory".
> That directory is a root where all drill sources are located (i.e. I have 
> /Users/vladimirsitnikov/Documents/work/drill/exec, 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec and so on).
> It looks like java-exec tests assume working directory is set to 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec, however it is 
> not the case when individual tests are run.
> The workaround is to add "exec/java-exec" to "working directory".
> It would be so much better if tests could be run from both working 
> directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-7025) Running Drill unit tests from within IntelliJ IDEA fails with FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist

2019-02-20 Thread Vladimir Sitnikov (JIRA) 


[ 
https://issues.apache.org/jira/browse/DRILL-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773392#comment-16773392
 ] 

Vladimir Sitnikov commented on DRILL-7025:
--

I've updated the configuration to $MODULE_WORKING_DIR$ (see 
[https://youtrack.jetbrains.com/issue/IDEA-188622#focus=streamItem-27-2764924.0-0]
 ), and it healed the config.

> Running Drill unit tests from within IntelliJ IDEA fails with 
> FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist
> ---
>
> Key: DRILL-7025
> URL: https://issues.apache.org/jira/browse/DRILL-7025
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Tools, Build  Test
>Affects Versions: 1.15.0
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> I start org.apache.drill.TestExampleQueries via regular "run tests" button, 
> and it produces the following exception:
> {noformat}
> java.lang.RuntimeException: This should not happen
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:298)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyResourceToRoot(BaseDirTestWatcher.java:223)
> at 
> org.apache.drill.TestExampleQueries.setupTestFiles(TestExampleQueries.java:42)
> Caused by: java.io.FileNotFoundException: Source 
> './src/test/resources/tpchmulti' does not exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:296)
> ... 2 more{noformat}
> In fact IDEA creates "run configuration" that uses 
> "/Users/vladimirsitnikov/Documents/work/drill" as "Working directory".
> That directory is a root where all drill sources are located (i.e. I have 
> /Users/vladimirsitnikov/Documents/work/drill/exec, 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec and so on).
> It looks like java-exec tests assume working directory is set to 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec, however it is 
> not the case when individual tests are run.
> The workaround is to add "exec/java-exec" to "working directory".
> It would be so much better if tests could be run from both working 
> directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-7025) Running Drill unit tests from within IntelliJ IDEA fails with FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist

2019-02-20 Thread Vladimir Sitnikov (JIRA) 


[ 
https://issues.apache.org/jira/browse/DRILL-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773388#comment-16773388
 ] 

Vladimir Sitnikov commented on DRILL-7025:
--

[~vitalii], [~ben-zvi], here's how I open the project.

1) IntelliJ IDEA 2019.1 EAP
 2) File -> Open
 3) Chose root {{pom.xml}}
 4) Open as project
 5) When the project loads, "template" JUnit run configuration has 
"/Users/vladimirsitnikov/Documents/work/drill" for "working directory"

For instance, I have {{$PROJECT_DIR$}} in 
/Users/vladimirsitnikov/Documents/work/drill/.idea/workspace.xml
{code:xml}

  
  
  

  
  

  
{code}
I did try to delete {{.idea}} folder, and it changes [nothing, absolutely 
nothing|https://youtu.be/riwKuKSbFDs?t=101].

 

I don't really remember where $PROJECT_DIR$ comes from, however it is present 
even in 
/Users/vladimirsitnikov/Library/Preferences/IntelliJIdea13/options/project.default.xml
 (which dates back to 2015)

 

It looks like it should use 
[MODULE_WORKING_DIR|https://github.com/JetBrains/intellij-community/blob/831cc2982f9e9d290337c775a3a9c737920c240d/plugins/junit/src/com/intellij/execution/junit/JUnitConfiguration.java#L610]
 there (see 
[https://blog.jetbrains.com/idea/2018/05/intellij-idea-2018-1-4-is-released/] )

> Running Drill unit tests from within IntelliJ IDEA fails with 
> FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist
> ---
>
> Key: DRILL-7025
> URL: https://issues.apache.org/jira/browse/DRILL-7025
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Tools, Build  Test
>Affects Versions: 1.15.0
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> I start org.apache.drill.TestExampleQueries via regular "run tests" button, 
> and it produces the following exception:
> {noformat}
> java.lang.RuntimeException: This should not happen
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:298)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyResourceToRoot(BaseDirTestWatcher.java:223)
> at 
> org.apache.drill.TestExampleQueries.setupTestFiles(TestExampleQueries.java:42)
> Caused by: java.io.FileNotFoundException: Source 
> './src/test/resources/tpchmulti' does not exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:296)
> ... 2 more{noformat}
> In fact IDEA creates "run configuration" that uses 
> "/Users/vladimirsitnikov/Documents/work/drill" as "Working directory".
> That directory is a root where all drill sources are located (i.e. I have 
> /Users/vladimirsitnikov/Documents/work/drill/exec, 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec and so on).
> It looks like java-exec tests assume working directory is set to 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec, however it is 
> not the case when individual tests are run.
> The workaround is to add "exec/java-exec" to "working directory".
> It would be so much better if tests could be run from both working 
> directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-7025) Running Drill unit tests from within IntelliJ IDEA fails with FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist

2019-02-20 Thread Boaz Ben-Zvi (JIRA) 


[ 
https://issues.apache.org/jira/browse/DRILL-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773371#comment-16773371
 ] 

Boaz Ben-Zvi commented on DRILL-7025:
-

_TestExampleQueries_  works for me as well with no special setting. 
[~vladimirsitnikov] Some setting in your configuration may be "unusual".  For 
me – under  "Run->Edit Configurations" the "Working Directory" is  
{{$MODULE_DIR$}} , and the "Use classpath of module" is selected as 
{{drill-java-exec}} .

> Running Drill unit tests from within IntelliJ IDEA fails with 
> FileNotFoundException: Source './src/test/resources/tpchmulti' does not exist
> ---
>
> Key: DRILL-7025
> URL: https://issues.apache.org/jira/browse/DRILL-7025
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Tools, Build  Test
>Affects Versions: 1.15.0
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> I start org.apache.drill.TestExampleQueries via regular "run tests" button, 
> and it produces the following exception:
> {noformat}
> java.lang.RuntimeException: This should not happen
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:298)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyResourceToRoot(BaseDirTestWatcher.java:223)
> at 
> org.apache.drill.TestExampleQueries.setupTestFiles(TestExampleQueries.java:42)
> Caused by: java.io.FileNotFoundException: Source 
> './src/test/resources/tpchmulti' does not exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
> at 
> org.apache.drill.test.BaseDirTestWatcher.copyTo(BaseDirTestWatcher.java:296)
> ... 2 more{noformat}
> In fact IDEA creates "run configuration" that uses 
> "/Users/vladimirsitnikov/Documents/work/drill" as "Working directory".
> That directory is a root where all drill sources are located (i.e. I have 
> /Users/vladimirsitnikov/Documents/work/drill/exec, 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec and so on).
> It looks like java-exec tests assume working directory is set to 
> /Users/vladimirsitnikov/Documents/work/drill/exec/java-exec, however it is 
> not the case when individual tests are run.
> The workaround is to add "exec/java-exec" to "working directory".
> It would be so much better if tests could be run from both working 
> directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-7045) UDF string_binary java.lang.IndexOutOfBoundsException:

2019-02-20 Thread jean-claude (JIRA) 
jean-claude created DRILL-7045:
--

 Summary: UDF string_binary java.lang.IndexOutOfBoundsException:
 Key: DRILL-7045
 URL: https://issues.apache.org/jira/browse/DRILL-7045
 Project: Apache Drill
  Issue Type: Bug
  Components: Functions - Drill
Affects Versions: 1.15.0
Reporter: jean-claude


Given a large field like

 

cat input.json

{ "col0":

[jira] [Updated] (DRILL-7042) Apache Drill v1.15.0 failed to generate deb/rpm package

2019-02-20 Thread Vitalii Diravka (JIRA) 


 [ 
https://issues.apache.org/jira/browse/DRILL-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitalii Diravka updated DRILL-7042:
---
Summary: Apache Drill v1.15.0 failed to generate deb/rpm package  (was: 
DRILL: Apache drill v1.15.0 failed to generate deb/rpm package)

> Apache Drill v1.15.0 failed to generate deb/rpm package
> ---
>
> Key: DRILL-7042
> URL: https://issues.apache.org/jira/browse/DRILL-7042
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build  Test
>Affects Versions: 1.15.0
> Environment: Ubuntu/Debian/CentOS installed on ARM64 server machine.
>Reporter: Naresh Bhat
>Assignee: Naresh Bhat
>Priority: Major
> Fix For: 1.16.0
>
>
> I tried to create a debian/rpm package on ARM64 machine,  But it failed to 
> generate the debian/rpm package on Ubuntu/CentOS machine. It is required to 
> fix the pom.xml file under distribution folder.
> Error logs while generating DEB package:
> =
> drill$ git branch 
>   master
> * v1.15.0
> drill$ mvn clean -X package -Pdeb -DskipTests
> .
> ..
> [INFO] Creating debian package: target/drill-1.15.0.deb
> [INFO] Building data
> [ERROR] Failed to create debian package target/drill-1.15.0.deb
> org.vafer.jdeb.PackagingException: Failed to create debian package 
> target/drill-1.15.0.deb
> at org.vafer.jdeb.maven.DebMaker.makeDeb (DebMaker.java:247)
> at org.vafer.jdeb.maven.DebMojo.execute (DebMojo.java:416)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
> (DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:210)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:148)
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
> (LifecycleModuleBuilder.java:117)
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
> (LifecycleModuleBuilder.java:81)
> at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
>  (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute 
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced 
> (Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch 
> (Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode 
> (Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main 
> (Launcher.java:356)
> Caused by: org.vafer.jdeb.PackagingException: Could not create deb package
> at org.vafer.jdeb.Processor.createDeb (Processor.java:172)
> at org.vafer.jdeb.maven.DebMaker.makeDeb (DebMaker.java:244)
> at org.vafer.jdeb.maven.DebMojo.execute (DebMojo.java:416)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
> (DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:210)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:148)
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
> (LifecycleModuleBuilder.java:117)
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
> (LifecycleModuleBuilder.java:81)
> at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
>  (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute 
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at