Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

UnaryTransformer’s scaladoc says "Abstract class for transformers that take one input column, apply transformation, and output the result as a new column.” If you want to allow specification of more than one input column, or if your output column already exists, or you want multiple output

Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

What is the difference between UnaryTransformer and Transformer classes. In which scenarios should we use one or the other ? On Sun, Jul 31, 2016 at 8:27 PM, janardhan shetty wrote: > Developing in scala but any help with difference between UnaryTransformer > (Is this

Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

Developing in scala but any help with difference between UnaryTransformer (Is this experimental still ?)and Transformer class is appreciated. Right now encountering error for the code which extends UnaryTransformer override protected def outputDataType: DataType = new StringType Error:(26, 53)

Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

Hi, I've developed a simple ML estimator (in Java) that implements conditional Markov model for sequence labelling in Vitk toolkit. You can check it out here: https://github.com/phuonglh/vn.vitk/blob/master/src/main/java/vn/vitk/tag/CMM.java Phuong Le-Hong On Fri, Jul 29, 2016 at 9:01 AM,

Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

Thanks Steve. Any pointers to custom estimators development as well ? On Wed, Jul 27, 2016 at 11:35 AM, Steve Rowe wrote: > You can see the source for my transformer configurable bridge to Lucene > analysis components here, in my company Lucidworks’ spark-solr project: < >

Re: Writing custom Transformers and Estimators like Tokenizer in spark ML

You can see the source for my transformer configurable bridge to Lucene analysis components here, in my company Lucidworks’ spark-solr project: . Here’s a

Writing custom Transformers and Estimators like Tokenizer in spark ML

1. Any links or blogs to develop *custom* transformers ? ex: Tokenizer 2. Any links or blogs to develop *custom* estimators ? ex: any ml algorithm