[jira] [Commented] (CASSANDRA-8367) Clash between Cassandra and Crunch mapreduce config
[ https://issues.apache.org/jira/browse/CASSANDRA-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803168#comment-14803168 ] Jim Witschey commented on CASSANDRA-8367: - [~zvo] I'm closing this for now -- if this hasn't been resolved to your satisfaction, could you follow the contribution instructions Philip linked to and reopen? Thanks. > Clash between Cassandra and Crunch mapreduce config > --- > > Key: CASSANDRA-8367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8367 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Radovan Zvoncek >Priority: Minor > > We would like to use Cassandra's (Cql)BulkOutputFormats to implement Resource > IOs for Crunch. We want to do this to allow Crunch users write results of > their jobs directly to Cassandra (thus skipping writing them to file system). > In the process of doing this, we found out there is a clash in the mapreduce > job config. The affected config key is 'mapreduce.output.basename'. Cassandra > is using it [1] for something different than Crunch [2]. This is resulting in > some obscure behavior I personally don't understand, but it causes the jobs > to fail. > We went ahead and re-implemented the output format classes to use different > config key, but we'd very much like to stop using them. > [1] > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L54 > [2] > https://github.com/apache/crunch/blob/3f13ee65c9debcf6bd7366607f58beae6c73ffe2/crunch-core/src/main/java/org/apache/crunch/io/CrunchOutputs.java#L99 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8367) Clash between Cassandra and Crunch mapreduce config
[ https://issues.apache.org/jira/browse/CASSANDRA-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385332#comment-14385332 ] Philip Thompson commented on CASSANDRA-8367: [~zvo], can you please submit this as a patch as detailed by http://wiki.apache.org/cassandra/HowToContribute ? Or we will be closing this ticket. > Clash between Cassandra and Crunch mapreduce config > --- > > Key: CASSANDRA-8367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8367 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Radovan Zvoncek >Priority: Minor > > We would like to use Cassandra's (Cql)BulkOutputFormats to implement Resource > IOs for Crunch. We want to do this to allow Crunch users write results of > their jobs directly to Cassandra (thus skipping writing them to file system). > In the process of doing this, we found out there is a clash in the mapreduce > job config. The affected config key is 'mapreduce.output.basename'. Cassandra > is using it [1] for something different than Crunch [2]. This is resulting in > some obscure behavior I personally don't understand, but it causes the jobs > to fail. > We went ahead and re-implemented the output format classes to use different > config key, but we'd very much like to stop using them. > [1] > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L54 > [2] > https://github.com/apache/crunch/blob/3f13ee65c9debcf6bd7366607f58beae6c73ffe2/crunch-core/src/main/java/org/apache/crunch/io/CrunchOutputs.java#L99 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8367) Clash between Cassandra and Crunch mapreduce config
[ https://issues.apache.org/jira/browse/CASSANDRA-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230101#comment-14230101 ] Michael Shuler commented on CASSANDRA-8367: --- [~zvo] do you have a patch that implements the changeover to a different config key? Someone could look at that :) > Clash between Cassandra and Crunch mapreduce config > --- > > Key: CASSANDRA-8367 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8367 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Radovan Zvoncek >Priority: Minor > > We would like to use Cassandra's (Cql)BulkOutputFormats to implement Resource > IOs for Crunch. We want to do this to allow Crunch users write results of > their jobs directly to Cassandra (thus skipping writing them to file system). > In the process of doing this, we found out there is a clash in the mapreduce > job config. The affected config key is 'mapreduce.output.basename'. Cassandra > is using it [1] for something different than Crunch [2]. This is resulting in > some obscure behavior I personally don't understand, but it causes the jobs > to fail. > We went ahead and re-implemented the output format classes to use different > config key, but we'd very much like to stop using them. > [1] > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java#L54 > [2] > https://github.com/apache/crunch/blob/3f13ee65c9debcf6bd7366607f58beae6c73ffe2/crunch-core/src/main/java/org/apache/crunch/io/CrunchOutputs.java#L99 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
