[jira] Commented: (MAPREDUCE-1465) archive partSize should be configurable

2010-02-08 Thread Mahadev konar (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831078#action_12831078
 ] 

Mahadev konar commented on MAPREDUCE-1465:
--

no allen, its just a configuration parameter thats hard coded.

> archive partSize should be configurable
> ---
>
> Key: MAPREDUCE-1465
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1465
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: harchive
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Mahadev konar
>
> The archive part size is current set to 2GB.  For archiving 10^5 small files, 
> it took 52 minutes since there is only 1 mapper.
> {noformat}
> -bash-3.1$ time $H archive ${Q} -archiveName ${DIR}.3.har -p ${PARENT} ${DIR} 
> ${PARENT}
> 10/02/06 01:55:14 INFO mapred.JobClient: Running job: job_201002042035_5737
> ...
> 10/02/06 02:47:18 INFO mapred.JobClient:  map 100% reduce 100%
> 10/02/06 02:47:19 INFO mapred.JobClient: Job complete: job_201002042035_5737
> ...
> 10/02/06 02:47:19 INFO mapred.JobClient: Reduce input records=12
> real52m27.188s
> user0m29.314s
> sys 0m1.276s
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (MAPREDUCE-1465) archive partSize should be configurable

2010-02-08 Thread Allen Wittenauer (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831056#action_12831056
 ] 

Allen Wittenauer commented on MAPREDUCE-1465:
-

Is it 2gb because of the limits of jar?

> archive partSize should be configurable
> ---
>
> Key: MAPREDUCE-1465
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1465
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: harchive
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Mahadev konar
>
> The archive part size is current set to 2GB.  For archiving 10^5 small files, 
> it took 52 minutes since there is only 1 mapper.
> {noformat}
> -bash-3.1$ time $H archive ${Q} -archiveName ${DIR}.3.har -p ${PARENT} ${DIR} 
> ${PARENT}
> 10/02/06 01:55:14 INFO mapred.JobClient: Running job: job_201002042035_5737
> ...
> 10/02/06 02:47:18 INFO mapred.JobClient:  map 100% reduce 100%
> 10/02/06 02:47:19 INFO mapred.JobClient: Job complete: job_201002042035_5737
> ...
> 10/02/06 02:47:19 INFO mapred.JobClient: Reduce input records=12
> real52m27.188s
> user0m29.314s
> sys 0m1.276s
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.