Re: [ML] Migrating transformers from mllib to ml

2017-11-07 Thread Joseph Bradley
Hi, we do still want to do this migration; it's just been a bit stalled due to low bandwidth. There are still a few feature parity items which need to be completed, so the deprecation will likely not happen until after 2.3. Joseph On Tue, Nov 7, 2017 at 12:38 AM, 颜发才(Yan Facai)

Re: [ML] Migrating transformers from mllib to ml

2017-11-07 Thread Yan Facai
Hi, I have migrated HashingTF from mllib to ml, and wait for review. see: [SPARK-21748][ML] Migrate the implementation of HashingTF from MLlib to ML #18998 https://github.com/apache/spark/pull/18998 On Mon, Nov 6, 2017 at 10:58 PM, Marco Gaido wrote: > Hello, > > I

[ML] Migrating transformers from mllib to ml

2017-11-06 Thread Marco Gaido
Hello, I saw that there are several TODOs to migrate some transformers (like HashingTF and IDF) to use only ml.Vector in order to avoid the overhead of converting them to the mllib ones and back. Is there any reason why this has not been done so far? Is it to avoid code duplication? If so, is it