[
https://issues.apache.org/jira/browse/CASSANDRA-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203433#comment-13203433
]
Samarth Gahire commented on CASSANDRA-3859:
-------------------------------------------
I have just checked the patches and it seems that you have added the progress
reporting while the generation of sstables (When the write method of the
BulkRecorWriter is executed).
But in our case the timeout issue is because of the time taken for the
streaming the sstables to the Cassandra (When the close() method of the
BulkRecorWriter is executed).
When the SSTableLoader comes into the picture and start loading the sstables if
the size of the sstables generated is big and it is taking more than 10 minutes
to load(stream),I dont see any progress reporting there, and the task will fail
of timed out.
> Add Progress Reporting to Cassandra OutputFormats
> -------------------------------------------------
>
> Key: CASSANDRA-3859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3859
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop, Tools
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Priority: Minor
> Labels: bulkloader, hadoop, mapreduce, sstableloader
> Fix For: 1.1
>
> Attachments: 0001-add-progress-reporting-to-BOF.txt,
> 0002-Add-progress-to-CFOF.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When we are using the BulkOutputFormat to load the data to cassandra. We
> should use the progress reporting to Hadoop Job within Sstable loader because
> while loading the data for particular task if streaming is taking more time
> and progress is not reported to Job it may kill the task with timeout
> exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
