[ https://issues.apache.org/jira/browse/CASSANDRA-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803168#comment-14803168 ]
Jim Witschey commented on CASSANDRA-8367: ----------------------------------------- [~zvo] I'm closing this for now -- if this hasn't been resolved to your satisfaction, could you follow the contribution instructions Philip linked to and reopen? Thanks. > Clash between Cassandra and Crunch mapreduce config > --------------------------------------------------- > > Key: CASSANDRA-8367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8367 > Project: Cassandra > Issue Type: Bug > Components: Hadoop > Reporter: Radovan Zvoncek > Priority: Minor > > We would like to use Cassandra's (Cql)BulkOutputFormats to implement Resource > IOs for Crunch. We want to do this to allow Crunch users write results of > their jobs directly to Cassandra (thus skipping writing them to file system). > In the process of doing this, we found out there is a clash in the mapreduce > job config. The affected config key is 'mapreduce.output.basename'. Cassandra > is using it [1] for something different than Crunch [2]. This is resulting in > some obscure behavior I personally don't understand, but it causes the jobs > to fail. > We went ahead and re-implemented the output format classes to use different > config key, but we'd very much like to stop using them. > [1] > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L54 > [2] > https://github.com/apache/crunch/blob/3f13ee65c9debcf6bd7366607f58beae6c73ffe2/crunch-core/src/main/java/org/apache/crunch/io/CrunchOutputs.java#L99 -- This message was sent by Atlassian JIRA (v6.3.4#6332)