[
https://issues.apache.org/jira/browse/SLIDER-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847369#comment-15847369
]
Sergey Shelukhin edited comment on SLIDER-1192 at 1/31/17 7:32 PM:
-------------------------------------------------------------------
[~steve_l] is there documentation? I can only find this
https://slider.incubator.apache.org/docs/configuration/internal.html that
doesn't mention the above.
Some examples e.g. here https://github.com/prestodb/presto-yarn/issues/33 seem
to have a large file with lots of configs.
Currently we don't explicitly generate that file. What's the minimum we should
generate to use that setting; also where does it go in create args?
was (Author: sershe):
[~steve_l] is there documentation? I can only find this
https://slider.incubator.apache.org/docs/configuration/internal.html that
doesn't mention the above.
Some examples e.g. here https://github.com/prestodb/presto-yarn/issues/33 seem
to have a large file with lots of configs.
Currently we don't seem to generate that file. What's the minimum we should
generate to use that setting; also where does it go in create args?
> Slider Agent tarball should be uploaded to HDFS to avoid client side
> dependency
> -------------------------------------------------------------------------------
>
> Key: SLIDER-1192
> URL: https://issues.apache.org/jira/browse/SLIDER-1192
> Project: Slider
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Gour Saha
> Priority: Blocker
> Fix For: Slider 1.0.0
>
>
> Trying to use Slider client from a Hive process that is shipped with slider
> client JAR
> {noformat}
> Caused by: java.io.FileNotFoundException: File
> /grid/5/sershe/tez-autobuild/dist/hive/lib/slider-agent.tar.gz does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
> at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1965)
> at
> org.apache.slider.providers.ProviderUtils.addAgentTar(ProviderUtils.java:128)
> at
> org.apache.slider.providers.agent.AgentClientProvider.prepareAMAndConfigForLaunch(AgentClientProvider.java:270){noformat}
> I can see the gz file in the slider lib directory. However, using slider
> client one should not even be required to have slider on the box (Hive
> already has the jars); nor is there a good way to find where it is, if
> installed (given that jar is shipped and we are not aware of the slider
> installation).
> The gz file should be included with client jars (perhaps as a resource?)
> Update: setting slider.libdir and/or SLIDER_HOME actually doesn't work at all.
> The code in addAgentTar basically looks for it in the directory of the client
> jar, no option to override it at all.
> That makes slider client absolutely unusable
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
