python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread tgbaggio
Hi, In HBaseConverter.scala https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/pythonconverters/HBaseConverters.scala , the python converter HBaseResultToStringConverter return only the value of first column in the result. In my opinion, it limits

RE: python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread Yan Zhou.sc
converter in HBaseConverter.scala(spark/examples) In my opinion this would be useful - there was another thread where returning only the value of first column in the result was mentioned. Please create a SPARK JIRA and a pull request. Cheers On Mon, Jan 5, 2015 at 6:42 AM, tgbaggio gen.tan

Re: python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread Ted Yu
HBaseConverter is in Spark source tree. Therefore I think it makes sense for this improvement to be accepted so that the example is more useful. Cheers On Mon, Jan 5, 2015 at 7:54 AM, Nick Pentreath nick.pentre...@gmail.com wrote: Hey These converters are actually just intended to be

Re: python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread Nick Pentreath
Absolutely; as I mentioned by all means submit a PR - I just wanted to point out that any specific converter is not officially supported, although the interface is of course. I'm happy to review a PR just ping me when ready. — Sent from Mailbox On Mon, Jan 5, 2015 at 7:06 PM, Ted Yu

Re: python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread Nick Pentreath
Hey  These converters are actually just intended to be examples of how to set up a custom converter for a specific input format. The converter interface is there to provide flexibility where needed, although with the new SparkSQL data store interface the intention is that most common use

Re: python converter in HBaseConverter.scala(spark/examples)

2015-01-05 Thread Ted Yu
In my opinion this would be useful - there was another thread where returning only the value of first column in the result was mentioned. Please create a SPARK JIRA and a pull request. Cheers On Mon, Jan 5, 2015 at 6:42 AM, tgbaggio gen.tan...@gmail.com wrote: Hi, In HBaseConverter.scala