Make streaming to handle non-utf8 byte array
--------------------------------------------
Key: HADOOP-1722
URL: https://issues.apache.org/jira/browse/HADOOP-1722
Project: Hadoop
Issue Type: Improvement
Components: contrib/streaming
Reporter: Runping Qi
Right now, the streaming framework expects the output sof the steam process
(mapper or reducer) are line
oriented UTF-8 text. This limit makes it impossible to use those programs whose
outputs may be non-UTF-8
(international encoding, or maybe even binary data). Streaming can overcome
this limit by introducing a simple
encoding protocol. For example, it can allow the mapper/reducer to hexencode
its keys/values,
the framework decodes them in the Java side.
This way, as long as the mapper/reducer executables follow this encoding
protocol,
they can output arabitary bytearray and the streaming framework can handle them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
