Tests can just be changed to accept either output too :p On Thu, Jun 7, 2018, 5:19 PM Dean Wampler <deanwamp...@gmail.com> wrote:
> Do the tests expect a particular console output order? That would annoy > them. ;) You could sort the expected and output lines, then diff... > > > *Dean Wampler, Ph.D.* > > *VP, Fast Data Engineering at Lightbend* > Author: Programming Scala, 2nd Edition > <http://shop.oreilly.com/product/0636920033073.do>, Fast Data > Architectures for Streaming Applications > <http://www.oreilly.com/data/free/fast-data-architectures-for-streaming-applications.csp>, > and other content from O'Reilly > @deanwampler <http://twitter.com/deanwampler> > http://polyglotprogramming.com > https://github.com/deanwampler > > On Thu, Jun 7, 2018 at 5:09 PM, Holden Karau <hol...@pigscanfly.ca> wrote: > >> If the difference is the order of the welcome message I think that should >> be fine. >> >> On Thu, Jun 7, 2018, 4:43 PM Dean Wampler <deanwamp...@gmail.com> wrote: >> >>> I'll point the Scala team to this issue, but it's unlikely to get fixed >>> any time soon. >>> >>> dean >>> >>> >>> *Dean Wampler, Ph.D.* >>> >>> *VP, Fast Data Engineering at Lightbend* >>> Author: Programming Scala, 2nd Edition >>> <http://shop.oreilly.com/product/0636920033073.do>, Fast Data >>> Architectures for Streaming Applications >>> <http://www.oreilly.com/data/free/fast-data-architectures-for-streaming-applications.csp>, >>> and other content from O'Reilly >>> @deanwampler <http://twitter.com/deanwampler> >>> http://polyglotprogramming.com >>> https://github.com/deanwampler >>> >>> On Thu, Jun 7, 2018 at 4:27 PM, DB Tsai <d_t...@apple.com> wrote: >>> >>>> Thanks Felix for bringing this up. >>>> >>>> Currently, in Scala 2.11.8, we initialize the Spark by overriding >>>> loadFIles() before REPL sees any file since there is no good hook in Scala >>>> to load our initialization code. >>>> >>>> In Scala 2.11.12 and newer version of the Scala 2.12.x, loadFIles() >>>> method was removed. >>>> >>>> Alternatively, one way we can do in the newer version of Scala is by >>>> overriding initializeSynchronous() suggested by Som Snytt; I have a working >>>> PR with this approach, >>>> https://github.com/apache/spark/pull/21495 , and this approach should >>>> work for older version of Scala too. >>>> >>>> However, in the newer version of Scala, the first thing that the REPL >>>> calls is printWelcome, so in the newer version of Scala, welcome message >>>> will be shown and then the URL of the SparkUI in this approach. This will >>>> cause UI inconsistencies between different versions of Scala. >>>> >>>> We can also initialize the Spark in the printWelcome which I feel more >>>> hacky. It will only work for newer version of Scala since in order version >>>> of Scala, printWelcome is called in the end of the initialization process. >>>> If we decide to go this route, basically users can not use Scala older than >>>> 2.11.9. >>>> >>>> I think this is also a blocker for us to move to newer version of Scala >>>> 2.12.x since the newer version of Scala 2.12.x has the same issue. >>>> >>>> In my opinion, Scala should fix the root cause and provide a stable >>>> hook for 3rd party developers to initialize their custom code. >>>> >>>> DB Tsai | Siri Open Source Technologies [not a contribution] | >>>> Apple, Inc >>>> >>>> > On Jun 7, 2018, at 6:43 AM, Felix Cheung <felixcheun...@hotmail.com> >>>> wrote: >>>> > >>>> > +1 >>>> > >>>> > Spoke to Dean as well and mentioned the problem with 2.11.12 >>>> https://github.com/scala/bug/issues/10913 >>>> > >>>> > _____________________________ >>>> > From: Sean Owen <sro...@gmail.com> >>>> > Sent: Wednesday, June 6, 2018 12:23 PM >>>> > Subject: Re: Scala 2.12 support >>>> > To: Holden Karau <hol...@pigscanfly.ca> >>>> > Cc: Dean Wampler <deanwamp...@gmail.com>, Reynold Xin < >>>> r...@databricks.com>, dev <dev@spark.apache.org> >>>> > >>>> > >>>> > If it means no change to 2.11 support, seems OK to me for Spark >>>> 2.4.0. The 2.12 support is separate and has never been mutually compatible >>>> with 2.11 builds anyway. (I also hope, suspect that the changes are >>>> minimal; tests are already almost entirely passing with no change to the >>>> closure cleaner when built for 2.12) >>>> > >>>> > On Wed, Jun 6, 2018 at 1:33 PM Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> > Just chatted with Dean @ the summit and it sounds like from Adriaan >>>> there is a fix in 2.13 for the API change issue that could be back ported >>>> to 2.12 so how about we try and get this ball rolling? >>>> > >>>> > It sounds like it would also need a closure cleaner change, which >>>> could be backwards compatible but since it’s such a core component and we >>>> might want to be cautious with it, we could when building for 2.11 use the >>>> old cleaner code and for 2.12 use the new code so we don’t break anyone. >>>> > >>>> > How do folks feel about this? >>>> > >>>> > >>>> > >>>> >>>> >>> >
