[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-07-04 Thread Amareshwari Sriramadasu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1888:
---

Status: Resolved  (was: Patch Available)
Resolution: Fixed

I just committed this.
Thanks Ravi!


 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch, 
 1888.v4.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-07-01 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Attachment: 1888.v4.patch

Attaching patch for trunk covering the case of reducer=NONE and mapper is a 
command.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch, 
 1888.v4.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-07-01 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


  Status: Patch Available  (was: Reopened)
Hadoop Flags:   (was: [Reviewed])

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch, 
 1888.v4.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-29 Thread Amareshwari Sriramadasu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1888:
---

Status: Open  (was: Patch Available)

Canceling the patch to fix test failures.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-29 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Status: Patch Available  (was: Open)

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-29 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Attachment: 1888.v3.patch

Attaching new patch fixing TrApp.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-29 Thread Amareshwari Sriramadasu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1888:
---

  Status: Resolved  (was: Patch Available)
Hadoop Flags: [Reviewed]
  Resolution: Fixed

I just committed this. Thanks Ravi !

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch, 1888.v3.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-28 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Attachment: 1888.v1.patch

 Please don't change genArgs() in TestStreaming. You can override it in 
 TestStreamingJavaTasks as done in other tests.

Hmm. That will be duplication of code. I want to avoid that. Modified all other 
testcases also that are derived from TestStreaming so that reuse of code is 
done(in my new patch).

 Is the Change in TestFileOutputFormat required?

This is just a duplication of code of those 2 lines. Though unrelated to this 
JIRA, removed those 2 lines.


Attaching new patch for trunk with the above mentioned changes.


 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-28 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Attachment: 1888.v2.patch

Attaching patch by fixing the Reducer NONE case.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-28 Thread Amareshwari Sriramadasu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1888:
---

Status: Patch Available  (was: Open)

+1 Patch looks fine.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch, 1888.v1.patch, 1888.v2.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (MAPREDUCE-1888) Streaming overrides user given output key and value types.

2010-06-25 Thread Ravi Gummadi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1888:


Attachment: 1888.patch

Attaching patch fixing the issue. Added testcase that validates the user set 
config properties mapreduce.map.output.key.class and mapreduce.output.key.class 
are honored in streaming jobs with java classes as mapper and reducer.

 Streaming overrides user given output key and value types.
 --

 Key: MAPREDUCE-1888
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1888
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Amareshwari Sriramadasu
Assignee: Ravi Gummadi
 Fix For: 0.22.0

 Attachments: 1888.patch


 The following code in StreamJob.java overrides user given output key and 
 value types.
 {code}
 idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
 job.setMapOutputValueClass(idResolver.getOutputValueClass());
 
 idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
 IdentifierResolver.TEXT_ID));
 conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
   idResolver.getOutputReaderClass(), OutputReader.class);
 job.setOutputKeyClass(idResolver.getOutputKeyClass());
 job.setOutputValueClass(idResolver.getOutputValueClass());
 {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.