[jira] [Resolved] (MAPREDUCE-7371) DistributedCache alternative APIs should not use DistributedCache APIs internally

2022-01-08 Thread Takanobu Asanuma (Jira) 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takanobu Asanuma resolved MAPREDUCE-7371.
-
Fix Version/s: 3.4.0
   Resolution: Fixed

> DistributedCache alternative APIs should not use DistributedCache APIs 
> internally
> -
>
> Key: MAPREDUCE-7371
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7371
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> DistributedCache has been deprecated long back and it's still being used as 
> of today. However, all the alternative Job APIs (for deprecated 
> DistributedCache APIs) still internally use DistributedCache APIs only, this 
> is a deadlock and it leaves no room for removal of DistributedCache APIs in 
> future. We should move core logic to Job or JobContext as required and let 
> deprecated DistributedCache APIs point to the right alternatives internally.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Resolved] (MAPREDUCE-7354) Use empty array constants present in TaskCompletionEvent to avoid creating redundant objects

2021-06-21 Thread Takanobu Asanuma (Jira) 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takanobu Asanuma resolved MAPREDUCE-7354.
-
Fix Version/s: 3.3.2
   3.4.0
   Resolution: Fixed

> Use empty array constants present in TaskCompletionEvent to avoid creating 
> redundant objects
> 
>
> Key: MAPREDUCE-7354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7354
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> As per one of the suggestions over PR#3115 (HDFS-16075), we should use empty 
> array constant present in TaskCompletionEvent to avoid creating redundant 
> objects.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Resolved] (MAPREDUCE-7350) Replace Guava Lists usage by Hadoop's own Lists in hadoop-mapreduce-project

2021-06-06 Thread Takanobu Asanuma (Jira) 


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takanobu Asanuma resolved MAPREDUCE-7350.
-
Fix Version/s: 3.4.0
   Resolution: Fixed

> Replace Guava Lists usage by Hadoop's own Lists in hadoop-mapreduce-project
> ---
>
> Key: MAPREDUCE-7350
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7350
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7099) Daily build fails in MapReduce JobClient though there isn't any error

2018-05-31 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created MAPREDUCE-7099:
---

 Summary: Daily build fails in MapReduce JobClient though there 
isn't any error
 Key: MAPREDUCE-7099
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7099
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: build
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


Looks like the build always fails lately. Please see the results of 
hadoop-qbt-trunk-java8-linux-x86:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86//artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt

{noformat}
[INFO] Results:
[INFO] 
[WARNING] Tests run: 565, Failures: 0, Errors: 0, Skipped: 10
[INFO] 
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 02:06 h
[INFO] Finished at: 2018-05-30T12:32:39+00:00
[INFO] Final Memory: 25M/645M
[INFO] 
[WARNING] The requested profile "parallel-tests" could not be activated because 
it does not exist.
[WARNING] The requested profile "shelltest" could not be activated because it 
does not exist.
[WARNING] The requested profile "native" could not be activated because it does 
not exist.
[WARNING] The requested profile "yarn-ui" could not be activated because it 
does not exist.
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on 
project hadoop-mapreduce-client-jobclient: There was a timeout or other error 
in the fork -> [Help 1]
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7098) Upgrade common-langs version to 3.7 in hadoop-mapreduce-project

2018-05-25 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created MAPREDUCE-7098:
---

 Summary: Upgrade common-langs version to 3.7 in 
hadoop-mapreduce-project
 Key: MAPREDUCE-7098
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7098
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


commons-lang 2.6 is widely used. Let's upgrade to 3.6.

This jira is separated from HADOOP-10783.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed

2018-01-24 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created MAPREDUCE-7041:
---

 Summary: Several unit tests in hadoop-mapreduce-client-jobclient 
are failed
 Key: MAPREDUCE-7041
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041
 Project: Hadoop Map/Reduce
  Issue Type: Bug
 Environment: These tests fails in trunk now. MAPREDUCE-6984 may be 
related.
{noformat}
hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed
hadoop.mapred.TestJobCleanup.testCustomAbort
hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort
{noformat}
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-7034) Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce

2018-01-11 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created MAPREDUCE-7034:
---

 Summary: Moving logging APIs over to slf4j the rest of all in 
hadoop-mapreduce
 Key: MAPREDUCE-7034
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7034
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Reporter: Takanobu Asanuma


It seems there are some left.
{noformat}
find hadoop-mapreduce-project -name "*.java" | xargs grep "import 
org.apache.commons.logging.Log" | wc -l
   7
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6985) MapReduce native optimization does not work properly due to a shuffle error

2017-10-18 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created MAPREDUCE-6985:
---

 Summary: MapReduce native optimization does not work properly due 
to a shuffle error
 Key: MAPREDUCE-6985
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6985
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 3.0.0-beta1
Reporter: Takanobu Asanuma
Priority: Critical


I confirmed MapReduce native optimization (MAPREDUCE-2841) of 
Hadoop-3.0.0-beta1 does not work properly due to a shffle error.

{noformat}
2017-10-19 11:59:40,513 WARN mapred.Task: Could not find output size
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find 
output/file.out in any of the configured local directories
at 
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:489)
at 
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:166)
at 
org.apache.hadoop.mapred.MROutputFiles.getOutputFile(MROutputFiles.java:57)
at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:1248)
at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:1228)
at org.apache.hadoop.mapred.Task.done(Task.java:1174)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:345)
at 
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
{noformat}

My build and run environment is below:
CentOS: 7.3
cmake: 3.6.3
Maven: 3.5.0
Java: 1.8.0_131
Hadoop: 3.0.0-beta1



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org