[ https://issues.apache.org/jira/browse/NUTCH-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
kaveh minooie updated NUTCH-1634: --------------------------------- Description: right now this is the ouput: WebTable statistics start Statistics for WebTable: status 2 (status_fetched): 1115 min score: 0.0 retry 2: 2 retry 0: 11369 jobs: {db_stats-job_local1037462743_0001={jobID=job_local1037462743_0001, jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1181220864, CPU_MILLISECONDS=0, SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166199}, File Output Format Counters ={BYTES_WRITTEN=373}}}} retry 1: 5 status 5 (status_redir_perm): 69 max score: 1.0 TOTAL urls: 11376 status 3 (status_gone): 3 status 4 (status_redir_temp): 3 status 1 (status_unfetched): 10186 avg score: 0.00342827 WebTable statistics: done status 2 (status_fetched): 1115 min score: 0.0 retry 2: 2 retry 0: 11369 jobs: {db_stats-job_local1037462743_0001={jobID=job_local1037462743_0001, jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1181220864, CPU_MILLISECONDS=0, SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166199}, File Output Format Counters ={BYTES_WRITTEN=373}}}} retry 1: 5 status 5 (status_redir_perm): 69 max score: 1.0 TOTAL urls: 11376 status 3 (status_gone): 3 status 4 (status_redir_temp): 3 status 1 (status_unfetched): 10186 avg score: 0.00342827 imho, it should be this: WebTable statistics start Statistics for WebTable: status 2 (status_fetched): 1115 min score: 0.0 retry 2: 2 retry 0: 11369 jobs: {db_stats-job_local801282144_0001={jobID=job_local801282144_0001, jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1122631680, CPU_MILLISECONDS=0, SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166191}, File Output Format Counters ={BYTES_WRITTEN=373}}}} retry 1: 5 status 5 (status_redir_perm): 69 max score: 1.0 TOTAL urls: 11376 status 3 (status_gone): 3 status 4 (status_redir_temp): 3 status 1 (status_unfetched): 10186 avg score: 0.00342827 WebTable statistics: done was://TODO > readdb -stats show the result twice > ----------------------------------- > > Key: NUTCH-1634 > URL: https://issues.apache.org/jira/browse/NUTCH-1634 > Project: Nutch > Issue Type: Bug > Components: crawldb > Affects Versions: 2.2.1 > Reporter: kaveh minooie > Priority: Minor > > right now this is the ouput: > WebTable statistics start > Statistics for WebTable: > status 2 (status_fetched): 1115 > min score: 0.0 > retry 2: 2 > retry 0: 11369 > jobs: {db_stats-job_local1037462743_0001={jobID=job_local1037462743_0001, > jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, > Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, > MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, > MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1181220864, CPU_MILLISECONDS=0, > SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, > REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, > REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, > FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166199}, File > Output Format Counters ={BYTES_WRITTEN=373}}}} > retry 1: 5 > status 5 (status_redir_perm): 69 > max score: 1.0 > TOTAL urls: 11376 > status 3 (status_gone): 3 > status 4 (status_redir_temp): 3 > status 1 (status_unfetched): 10186 > avg score: 0.00342827 > WebTable statistics: done > status 2 (status_fetched): 1115 > min score: 0.0 > retry 2: 2 > retry 0: 11369 > jobs: {db_stats-job_local1037462743_0001={jobID=job_local1037462743_0001, > jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, > Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, > MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, > MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1181220864, CPU_MILLISECONDS=0, > SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, > REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, > REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, > FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166199}, File > Output Format Counters ={BYTES_WRITTEN=373}}}} > retry 1: 5 > status 5 (status_redir_perm): 69 > max score: 1.0 > TOTAL urls: 11376 > status 3 (status_gone): 3 > status 4 (status_redir_temp): 3 > status 1 (status_unfetched): 10186 > avg score: 0.00342827 > imho, it should be this: > WebTable statistics start > Statistics for WebTable: > status 2 (status_fetched): 1115 > min score: 0.0 > retry 2: 2 > retry 0: 11369 > jobs: {db_stats-job_local801282144_0001={jobID=job_local801282144_0001, > jobName=db_stats, counters={File Input Format Counters ={BYTES_READ=0}, > Map-Reduce Framework={MAP_OUTPUT_MATERIALIZED_BYTES=209, > MAP_INPUT_RECORDS=11376, REDUCE_SHUFFLE_BYTES=0, SPILLED_RECORDS=24, > MAP_OUTPUT_BYTES=602928, COMMITTED_HEAP_BYTES=1122631680, CPU_MILLISECONDS=0, > SPLIT_RAW_BYTES=757, COMBINE_INPUT_RECORDS=45504, REDUCE_INPUT_RECORDS=12, > REDUCE_INPUT_GROUPS=12, COMBINE_OUTPUT_RECORDS=12, PHYSICAL_MEMORY_BYTES=0, > REDUCE_OUTPUT_RECORDS=12, VIRTUAL_MEMORY_BYTES=0, MAP_OUTPUT_RECORDS=45504}, > FileSystemCounters={FILE_BYTES_READ=1819, FILE_BYTES_WRITTEN=166191}, File > Output Format Counters ={BYTES_WRITTEN=373}}}} > retry 1: 5 > status 5 (status_redir_perm): 69 > max score: 1.0 > TOTAL urls: 11376 > status 3 (status_gone): 3 > status 4 (status_redir_temp): 3 > status 1 (status_unfetched): 10186 > avg score: 0.00342827 > WebTable statistics: done -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira