i agree with that. we work within that assumption. we compile and run against a single exact spark version. we know when we upgrade that we might run into minor inconveniences that our completely our own doing/fault. the trade off has been totally worth it to me.
On Thu, Mar 30, 2017 at 1:20 PM, Michael Armbrust <mich...@databricks.com> wrote: > I think really the right way to think about things that are marked private > is, "this may disappear or change in a future minor release". If you are > okay with that, working about the visibility restrictions is reasonable. > > On Thu, Mar 30, 2017 at 5:52 AM, Koert Kuipers <ko...@tresata.com> wrote: > >> I stopped asking long time ago why things are private in spark... I >> mean... The conversion between ml and mllib vectors is private... the >> conversion between spark vector and breeze used to be (or still is?) >> private. it just goes on. Lots of useful stuff is private[SQL]. >> >> Luckily there are simple ways to get around these visibility restrictions >> >> On Mar 29, 2017 22:57, "Ryan" <ryan.hd....@gmail.com> wrote: >> >>> I'm writing a transformer and the input column is vector type(which is >>> the output column from other transformer). But as the VectorUDT is private, >>> how could I check/transform schema for the vector column? >>> >> >
