Maybe it would be quite useful to have tools that help developers spot the possibly subtle bugs that might be introduced by this change. I tried one possible path in this POC package: https://github.com/meggart/ReadOnlySubArrays.jl It introduces ReadOnlySubarrays which are mostly a copy-paste replicate of subarrays.jl, the only difference is that they have a custom setindex! method that throws a warning/error each time you are trying to modify it. When you load the package and do slicing using [], a ReadOnlySubarray will be created and you get notified whenever it is modified, which might introduce unwanted behavior, because the parent array would be modified, too. One could also automatically detect if a singleton dimension would be dropped by the creation (not implemented yet), and warn/error on subsequent array accesses. However, this will not spot unintended behavior when you modifiy the parent array which would also cause a modification of the SubArray. To do this, one needs a modified array implementation (which is unfortunately not Julia), to tag any array that has Subarrays and error if you setindex! them. I started experimenting here: https://github.com/meggart/julia/commit/224188c3e7c4053883d2445bf4e03315cb2ef71e and it seems to work for simple cases. All this is still experimental and will not detect all possible bugs, but might still be a good way to help with checking if your code is transition-ready.
Fabian On Tuesday, November 3, 2015 at 9:57:43 AM UTC+1, Tomas Lycken wrote: > > My point is that what we *really* want here is the ability to test the > new behavior while the old behavior is still the released version, or to > opt-out of the change altogether - all the suggestions in this thread about > how to handle this transition are trying to solve that problem. And > solutions already exists - to pre-test, just test the package against the > Julia master as soon as the change merges (and this will be done by Pkg > evaluator, semi-automatically filing issues in case errors occur), and to > opt-out, just pin your code to depend on julia 0.4. > > The way I see it, the need for soft transitioning is not as high in Julia > as it is in Python - Julia is a fast-moving target, and currently in > pre-release (think of all the 0.x releases as alpha releases of the real > language, if you will). IMO, it’s reasonable to expect breaking changes now > and then, even major ones, even ones that may lead to subtle and > hard-to-find bugs, and it’s reasonable to expect users and package > developers to either put in some extra effort to keep code up-to-date with > the latest and greatest, or pin down dependencies to a version one knows > works (no-one will force you to upgrade to 0.5, if you think 0.4 works > better for you…). Having to put that effort in is the price you pay for > using a system that hasn't reached 1.0 status yet. At least to me, Julia is > awesome enough to make it more than worth it :) > > There were a couple of big breakages during the 0.4 cycle (this slide in > Iain Dunning’s talk from JuliaCon 2015 > <https://youtu.be/pYYmnBma9qk?list=PLP8iPy9hna6Sdx4soiGrSefrmOPdUWixM&t=417> > gives a good overview of the big ones) and the community were quite quick > to fix the major problems that occurred then, so that now when 0.4 was > released, the actual breakage from those changes was relatively minor. As I > see it, this transitioning model is good enough for Julia at the moment - > if/when we want to make breaking changes between 1.x and 2.0, that’s a > different story. > > // T > > On Tuesday, November 3, 2015 at 9:28:43 AM UTC+1, Kristoffer Carlsson > wrote: > > I don't see how this enables "extreme performance gains". As have been >> said, you can already now use slice/sub in base so the only difference is >> if a slice is default or not for the [:] syntax. I do realize that syntax >> matters and no one is going to write slice(A, :, 1) everywhere instead of >> A[:,1]. >> >> However, I am a bit wary about it because the vast majority of the hard >> to track down bugs I've had has been due to some unexpected shared state. >> Wouldn't it be best if you could opt in to use the [:] syntax as slices >> somehow and have the default stay as it is? >> >> >> On Tuesday, November 3, 2015 at 8:30:04 AM UTC+1, Tomas Lycken wrote: >>> >>> There’s no way this transition is going to be smooth without work from >>> package devs to ensure compatibility anyway, but recent history (e.g. the >>> Tuplocalypse) shows that packages recover quite rapidly due to the >>> automated testing done by PkgEval.jl. Thus, packages with decent test >>> suites will probably get a heads-up that this change is breaking for them >>> long before 0.5.0 is released. >>> >>> Given that these changes potentially allow quite extreme performance >>> gains in many applications, I find it unlikely that package devs won’t want >>> to spend some time fixing any bugs this behavior change introduces, but for >>> those that just want to pin their package down before fixing it, they can >>> say julia 0.4 0.4 or similar in their REQUIRE.jl, and not even allow >>> installation on 0.5 (until they fix the problems, change the REQUIRE.jl and >>> tag a new version). Note that many (most?) packages won’t even need this - >>> package testing on the master branch will reveal to devs whether a package >>> even needs touch-up or not. >>> >>> I used from __future__ import print at the top of my python scripts for >>> a long time, so I’m entirely in favor of something like that. However, I >>> guess the behavior needs to be implemented and done in the master branch of >>> Julia before we know exactly how to mimic it in a package (if it’s even >>> possible). >>> >>> // T >>> >>> On Tuesday, November 3, 2015 at 7:41:17 AM UTC+1, Christoph Ortner wrote: >>> >>> Why not do this via Compat.jl ? >>>> C >>>> >>>> On Tuesday, 3 November 2015 04:28:44 UTC, David Anthoff wrote: >>>>> >>>>> No, Compat generall ports the new syntax back to old versions. What >>>>> I'm suggesting is quite different, namely that in julia 0.5 if you just >>>>> use >>>>> `[]` for slicing without anything else, you generate a warning, and then >>>>> you have to opt-in to the NEW syntax. >>>>> >>>>> > -----Original Message----- >>>>> > From: julia...@googlegroups.com [mailto:julia- >>>>> > us...@googlegroups.com] On Behalf Of Eric Forgy >>>>> > Sent: Monday, November 2, 2015 3:30 PM >>>>> > To: julia-users <julia...@googlegroups.com> >>>>> > Subject: RE: [julia-users] Re: Re: are array slices views in 0.4? >>>>> > >>>>> > Hi David, >>>>> > >>>>> > I'm not an expert, i.e. I've never used it, but your idea "using >>>>> OldArrays" >>>>> > sounds exactly like the purpose of Compat.jl, which I believe is the >>>>> > compatibility module referred to. >>>>> > >>>>> > Have a look and good luck: >>>>> > >>>>> > https://github.com/JuliaLang/Compat.jl >>>>> >>>> >>> >> >
