[ https://issues.apache.org/jira/browse/MESOS-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bernd Mathiske updated MESOS-4304: ---------------------------------- Shepherd: Jie Yu > hdfs operations fail due to prepended / on path for non-hdfs hadoop clients. > ---------------------------------------------------------------------------- > > Key: MESOS-4304 > URL: https://issues.apache.org/jira/browse/MESOS-4304 > Project: Mesos > Issue Type: Bug > Components: fetcher > Affects Versions: 0.26.0 > Reporter: James Cunningham > Assignee: Bernd Mathiske > > This bug was resolved for the hdfs protocol for MESOS-3602 but since the > process checks for the "hdfs" protocol at the beginning of the URI, the fix > does not extend itself to non-hdfs hadoop clients. > {code} > I0107 01:22:01.259490 17678 logging.cpp:172] INFO level logging started! > I0107 01:22:01.259856 17678 fetcher.cpp:422] Fetcher Info: > {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/530dda5a-481a-4117-8154-3aee637d3b38-S3\/root","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"maprfs:\/\/\/mesos\/storm-mesos-0.9.3.tgz"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"http:\/\/s0121.stag.urbanairship.com:36373\/conf\/storm.yaml"}}],"sandbox_directory":"\/mnt\/data\/mesos\/slaves\/530dda5a-481a-4117-8154-3aee637d3b38-S3\/frameworks\/530dda5a-481a-4117-8154-3aee637d3b38-0000\/executors\/word-count-1-1452129714\/runs\/4443d5ac-d034-49b3-bf12-08fb9b0d92d0","user":"root"} > I0107 01:22:01.262171 17678 fetcher.cpp:377] Fetching URI > 'maprfs:///mesos/storm-mesos-0.9.3.tgz' > I0107 01:22:01.262212 17678 fetcher.cpp:248] Fetching directly into the > sandbox directory > I0107 01:22:01.262243 17678 fetcher.cpp:185] Fetching URI > 'maprfs:///mesos/storm-mesos-0.9.3.tgz' > I0107 01:22:01.671777 17678 fetcher.cpp:110] Downloading resource with Hadoop > client from 'maprfs:///mesos/storm-mesos-0.9.3.tgz' to > '/mnt/data/mesos/slaves/530dda5a-481a-4117-8154-3aee637d3b38-S3/frameworks/530dda5a-481a-4117-8154-3aee637d3b38-0000/executors/word-count-1-1452129714/runs/4443d5ac-d034-49b3-bf12-08fb9b0d92d0/storm-mesos-0.9.3.tgz' > copyToLocal: java.net.URISyntaxException: Expected scheme-specific part at > index 7: maprfs: > Usage: java FsShell [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>] > E0107 01:22:02.435556 17678 shell.hpp:90] Command 'hadoop fs -copyToLocal > '/maprfs:///mesos/storm-mesos-0.9.3.tgz' > '/mnt/data/mesos/slaves/530dda5a-481a-4117-8154-3aee637d3b38-S3/frameworks/530dda5a-481a-4117-8154-3aee637d3b38-0000/executors/word-count-1-1452129714/runs/4443d5ac-d034-49b3-bf12-08fb9b0d92d0/storm-mesos-0.9.3.tgz'' > failed; this is the output: > Failed to fetch 'maprfs:///mesos/storm-mesos-0.9.3.tgz': HDFS copyToLocal > failed: Failed to execute 'hadoop fs -copyToLocal > '/maprfs:///mesos/storm-mesos-0.9.3.tgz' > '/mnt/data/mesos/slaves/530dda5a-481a-4117-8154-3aee637d3b38-S3/frameworks/530dda5a-481a-4117-8154-3aee637d3b38-0000/executors/word-count-1-1452129714/runs/4443d5ac-d034-49b3-bf12-08fb9b0d92d0/storm-mesos-0.9.3.tgz''; > the command was either not found or exited with a non-zero exit status: 255 > Failed to synchronize with slave (it's probably exited) > {code} > After a brief chat with [~jieyu], it was recommended to fix the current hdfs > client code because the new hadoop fetcher plugin is slated to use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)