[
https://issues.apache.org/jira/browse/YARN-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788543#comment-13788543
]
Omkar Vinit Joshi commented on YARN-1283:
-----------------------------------------
Thanks [~yeshavora]. Today as a part of application report we don't return the
scheme and on client side we generate scheme using
HttpConfig.getSchemePrefix(). Ideally server should have returned the url and
client should have used it as it is. Uploading a patch which fixes this
behavior.
> Invalid 'url of job' mentioned in Job output with yarn.http.policy=HTTPS_ONLY
> -----------------------------------------------------------------------------
>
> Key: YARN-1283
> URL: https://issues.apache.org/jira/browse/YARN-1283
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.1.1-beta
> Reporter: Yesha Vora
> Assignee: Omkar Vinit Joshi
>
> After setting yarn.http.policy=HTTPS_ONLY, the job output shows incorrect
> "The url to track the job".
> Currently, its printing
> http://RM:<httpsport>/proxy/application_1381162886563_0001/ instead
> https://RM:<httpsport>/proxy/application_1381162886563_0001/
> http://hostname:8088/proxy/application_1381162886563_0001/ is invalid
> hadoop jar hadoop-mapreduce-client-jobclient-tests.jar sleep -m 1 -r 1
> 13/10/07 18:39:39 INFO client.RMProxy: Connecting to ResourceManager at
> hostname/100.00.00.000:8032
> 13/10/07 18:39:40 INFO mapreduce.JobSubmitter: number of splits:1
> 13/10/07 18:39:40 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/10/07 18:39:40 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.partitioner.class
> is deprecated. Instead, use mapreduce.job.partitioner.class
> 13/10/07 18:39:40 INFO Configuration.deprecation:
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 13/10/07 18:39:40 INFO Configuration.deprecation:
> mapred.mapoutput.value.class is deprecated. Instead, use
> mapreduce.map.output.value.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/10/07 18:39:40 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.mapoutput.key.class
> is deprecated. Instead, use mapreduce.map.output.key.class
> 13/10/07 18:39:40 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/10/07 18:39:40 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1381162886563_0001
> 13/10/07 18:39:40 INFO impl.YarnClientImpl: Submitted application
> application_1381162886563_0001 to ResourceManager at
> hostname/100.00.00.000:8032
> 13/10/07 18:39:40 INFO mapreduce.Job: The url to track the job:
> http://hostname:8088/proxy/application_1381162886563_0001/
> 13/10/07 18:39:40 INFO mapreduce.Job: Running job: job_1381162886563_0001
> 13/10/07 18:39:46 INFO mapreduce.Job: Job job_1381162886563_0001 running in
> uber mode : false
> 13/10/07 18:39:46 INFO mapreduce.Job: map 0% reduce 0%
> 13/10/07 18:39:53 INFO mapreduce.Job: map 100% reduce 0%
> 13/10/07 18:39:58 INFO mapreduce.Job: map 100% reduce 100%
> 13/10/07 18:39:58 INFO mapreduce.Job: Job job_1381162886563_0001 completed
> successfully
> 13/10/07 18:39:58 INFO mapreduce.Job: Counters: 43
> File System Counters
> FILE: Number of bytes read=26
> FILE: Number of bytes written=177279
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> HDFS: Number of bytes read=48
> HDFS: Number of bytes written=0
> HDFS: Number of read operations=1
> HDFS: Number of large read operations=0
> HDFS: Number of write operations=0
> Job Counters
> Launched map tasks=1
> Launched reduce tasks=1
> Other local map tasks=1
> Total time spent by all maps in occupied slots (ms)=7136
> Total time spent by all reduces in occupied slots (ms)=6062
> Map-Reduce Framework
> Map input records=1
> Map output records=1
> Map output bytes=4
> Map output materialized bytes=22
> Input split bytes=48
> Combine input records=0
> Combine output records=0
> Reduce input groups=1
> Reduce shuffle bytes=22
> Reduce input records=1
> Reduce output records=0
> Spilled Records=2
> Shuffled Maps =1
> Failed Shuffles=0
> Merged Map outputs=1
> GC time elapsed (ms)=60
> CPU time spent (ms)=1700
> Physical memory (bytes) snapshot=567582720
> Virtual memory (bytes) snapshot=4292997120
> Total committed heap usage (bytes)=846594048
> Shuffle Errors
> BAD_ID=0
> CONNECTION=0
> IO_ERROR=0
> WRONG_LENGTH=0
> WRONG_MAP=0
> WRONG_REDUCE=0
> File Input Format Counters
> Bytes Read=0
> File Output Format Counters
> Bytes Written=0
--
This message was sent by Atlassian JIRA
(v6.1#6144)
