Re: Why VectorUDT private?

sorry meant to say: we know when we upgrade that we might run into minor inconveniences that are completely our own doing/fault. also, with yarn it has become really easy to run against an exact spark version of our choosing, since there is no longer such a thing as a centrally managed spark

Re: Why VectorUDT private?

i agree with that. we work within that assumption. we compile and run against a single exact spark version. we know when we upgrade that we might run into minor inconveniences that our completely our own doing/fault. the trade off has been totally worth it to me. On Thu, Mar 30, 2017 at 1:20

Re: Why VectorUDT private?

I think really the right way to think about things that are marked private is, "this may disappear or change in a future minor release". If you are okay with that, working about the visibility restrictions is reasonable. On Thu, Mar 30, 2017 at 5:52 AM, Koert Kuipers wrote:

Re: Why VectorUDT private?

I stopped asking long time ago why things are private in spark... I mean... The conversion between ml and mllib vectors is private... the conversion between spark vector and breeze used to be (or still is?) private. it just goes on. Lots of useful stuff is private[SQL]. Luckily there are simple

Re: Why VectorUDT private?

spark version 2.1.0, vector is from ml package. the Vector in mllib has a public VectorUDT type On Thu, Mar 30, 2017 at 10:57 AM, Ryan wrote: > I'm writing a transformer and the input column is vector type(which is the > output column from other transformer). But as the

Why VectorUDT private?

I'm writing a transformer and the input column is vector type(which is the output column from other transformer). But as the VectorUDT is private, how could I check/transform schema for the vector column?