Do the tests expect a particular console output order? That would annoy them. ;) You could sort the expected and output lines, then diff...
*Dean Wampler, Ph.D.* *VP, Fast Data Engineering at Lightbend* Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do>, Fast Data Architectures for Streaming Applications <http://www.oreilly.com/data/free/fast-data-architectures-for-streaming-applications.csp>, and other content from O'Reilly @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com https://github.com/deanwampler On Thu, Jun 7, 2018 at 5:09 PM, Holden Karau <hol...@pigscanfly.ca> wrote: > If the difference is the order of the welcome message I think that should > be fine. > > On Thu, Jun 7, 2018, 4:43 PM Dean Wampler <deanwamp...@gmail.com> wrote: > >> I'll point the Scala team to this issue, but it's unlikely to get fixed >> any time soon. >> >> dean >> >> >> *Dean Wampler, Ph.D.* >> >> *VP, Fast Data Engineering at Lightbend* >> Author: Programming Scala, 2nd Edition >> <http://shop.oreilly.com/product/0636920033073.do>, Fast Data >> Architectures for Streaming Applications >> <http://www.oreilly.com/data/free/fast-data-architectures-for-streaming-applications.csp>, >> and other content from O'Reilly >> @deanwampler <http://twitter.com/deanwampler> >> http://polyglotprogramming.com >> https://github.com/deanwampler >> >> On Thu, Jun 7, 2018 at 4:27 PM, DB Tsai <d_t...@apple.com> wrote: >> >>> Thanks Felix for bringing this up. >>> >>> Currently, in Scala 2.11.8, we initialize the Spark by overriding >>> loadFIles() before REPL sees any file since there is no good hook in Scala >>> to load our initialization code. >>> >>> In Scala 2.11.12 and newer version of the Scala 2.12.x, loadFIles() >>> method was removed. >>> >>> Alternatively, one way we can do in the newer version of Scala is by >>> overriding initializeSynchronous() suggested by Som Snytt; I have a working >>> PR with this approach, >>> https://github.com/apache/spark/pull/21495 , and this approach should >>> work for older version of Scala too. >>> >>> However, in the newer version of Scala, the first thing that the REPL >>> calls is printWelcome, so in the newer version of Scala, welcome message >>> will be shown and then the URL of the SparkUI in this approach. This will >>> cause UI inconsistencies between different versions of Scala. >>> >>> We can also initialize the Spark in the printWelcome which I feel more >>> hacky. It will only work for newer version of Scala since in order version >>> of Scala, printWelcome is called in the end of the initialization process. >>> If we decide to go this route, basically users can not use Scala older than >>> 2.11.9. >>> >>> I think this is also a blocker for us to move to newer version of Scala >>> 2.12.x since the newer version of Scala 2.12.x has the same issue. >>> >>> In my opinion, Scala should fix the root cause and provide a stable hook >>> for 3rd party developers to initialize their custom code. >>> >>> DB Tsai | Siri Open Source Technologies [not a contribution] | >>> Apple, Inc >>> >>> > On Jun 7, 2018, at 6:43 AM, Felix Cheung <felixcheun...@hotmail.com> >>> wrote: >>> > >>> > +1 >>> > >>> > Spoke to Dean as well and mentioned the problem with 2.11.12 >>> https://github.com/scala/bug/issues/10913 >>> > >>> > _____________________________ >>> > From: Sean Owen <sro...@gmail.com> >>> > Sent: Wednesday, June 6, 2018 12:23 PM >>> > Subject: Re: Scala 2.12 support >>> > To: Holden Karau <hol...@pigscanfly.ca> >>> > Cc: Dean Wampler <deanwamp...@gmail.com>, Reynold Xin < >>> r...@databricks.com>, dev <dev@spark.apache.org> >>> > >>> > >>> > If it means no change to 2.11 support, seems OK to me for Spark 2.4.0. >>> The 2.12 support is separate and has never been mutually compatible with >>> 2.11 builds anyway. (I also hope, suspect that the changes are minimal; >>> tests are already almost entirely passing with no change to the closure >>> cleaner when built for 2.12) >>> > >>> > On Wed, Jun 6, 2018 at 1:33 PM Holden Karau <hol...@pigscanfly.ca> >>> wrote: >>> > Just chatted with Dean @ the summit and it sounds like from Adriaan >>> there is a fix in 2.13 for the API change issue that could be back ported >>> to 2.12 so how about we try and get this ball rolling? >>> > >>> > It sounds like it would also need a closure cleaner change, which >>> could be backwards compatible but since it’s such a core component and we >>> might want to be cautious with it, we could when building for 2.11 use the >>> old cleaner code and for 2.12 use the new code so we don’t break anyone. >>> > >>> > How do folks feel about this? >>> > >>> > >>> > >>> >>> >>
