[
https://issues.apache.org/jira/browse/GIRAPH-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maja Kabiljo updated GIRAPH-648:
--------------------------------
Attachment: GIRAPH-648.patch
Unfortunately this is not as straightforward as it sounds, because methods from
io formats, readers/writers/OutputCommitter have JobContext or
TaskAttemptContext as an argument, and in some cases those hold the copy of
Configuration, not the original. So I added a way to track which parameters
where added to GiraphConfiguration, and wrapped all io related calls to append
those parameters to JobContext/TaskAttemptContext before passing control to
actual io formats.
Cleaned up HiveGiraphRunner and moved all control to its io formats, I can do
similar for HCatalog in a separate patch.
This will also help us do GIRAPH-639 in a cleaner way, and it will actually be
possible to mix different kind of input formats (hcat, hive, hbase, or
whatever).
> Allow IO formats to add parameters to Configuration
> ---------------------------------------------------
>
> Key: GIRAPH-648
> URL: https://issues.apache.org/jira/browse/GIRAPH-648
> Project: Giraph
> Issue Type: Improvement
> Reporter: Maja Kabiljo
> Assignee: Maja Kabiljo
> Attachments: GIRAPH-648.patch
>
>
> Currently we heavily rely on some runners (HCatGiraphRunner and
> HiveGiraphRunner) to prepare Configuration before application starts, and we
> have no way of using hcat/hive io without these runners. It would be better
> and more flexible if io formats would add what's needed for underlying io to
> Configuration themselves.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
