James Peach created MESOS-9348:
----------------------------------
Summary: URL-encoded HDFS artifacts can't be fetched through the
cache.
Key: MESOS-9348
URL: https://issues.apache.org/jira/browse/MESOS-9348
Project: Mesos
Issue Type: Bug
Components: fetcher
Reporter: James Peach
The {{hdfs dfs}} command always does a URI decode on the target output file.
This means that the output file gets stored in the fetcher cache under the
wrong filename and we can never retrieve it.
Here's an example of how the command behaves:
{noformat}
[/tmp]# hdfs dfs -copyToLocal
hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar
$(pwd)/%5B-jpeach-%5D.jar
[/tmp]# ls -l *jpeach*
-rw-r--r-- 1 root root 7285799 Oct 22 23:29 [-jpeach-].jar
{noformat}
Here's how this plays out in the fetcher:
{noformat}
W1022 23:22:13.649587 3186459 fetcher.cpp:395] Copying instead of extracting
resource from URI with 'extract' flag, because it does not seem to be an
archive:
hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar
cp: cannot stat `/srv/mesos/fetch/jarvis/c67-connector-_ASE%5D.jar': No such
file or directory
E1022 23:22:13.652987 3186459 fetcher.cpp:613] EXIT with status 1: Failed to
fetch
'hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar':
cp failed with status: 256
...
# ls -latr /srv/mesos/fetch
...
-rw-r--r-- 1 jarvis jarvis 7285799 Oct 22 23:22 c67-connector-_ASE].jar
{noformat}
The fetcher has downloaded the artifact into the cache, but can't copy it into
the sandbox because it was downloaded to the wrong filename.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
