Re: Thoughts on extedning ML exporting in Spark?

2017-11-19 Thread Timur Shenkao
Hello guys, Have you considered PFA? http://dmg.org/pfa/docs/document_structure/ As Sean noticed, "there are already 1.5 supported formats " + PMML is quite rigid. There are, at least, 2 implementations of PFA. *Scala* Hadrian: https://github.com/opendatagroup/hadrian. *Python* Titus: https//gi

Re: Handling nulls in vector columns is non-trivial

2017-06-24 Thread Timur Shenkao
Hi Franklyn, I had the same problem like yours with vectors & Maps. I tried: 1) UDF --> cumbersome and difficult to maintain. One has to re-write / re-implement UDFs + extensive docs should be provided for colleagues + something weird may happen when you migrate to new Spark version 2) RDD / DataF

Re: Should we consider a Spark 2.1.1 release?

2017-03-20 Thread Timur Shenkao
Hello guys, Spark benefits from stable versions not frequent ones. A lot of people still have 1.6.x in production. Those who wants the freshest (like me) can always deploy night builts. My question is: how long version 1.6 will be supported? On Sunday, March 19, 2017, Holden Karau wrote: > This

Re: Java 9

2017-02-07 Thread Timur Shenkao
If I'm not wrong, they got fid of *sun.misc.Unsafe *in Java 9. This class is till used by several libraries & frameworks. http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ On Tue, Feb 7, 2017 at 12:51 PM, Pete Robbins wrote: > Yes, I agree but it may be worthwhile starti

Java 8

2016-08-20 Thread Timur Shenkao
Hello, guys! Are there any plans / tickets / branches in repository on Java 8? I ask because ML library will gain in performance. I'd like to take part in refactoring.