On Tuesday, February 10, 2015 at 1:10:33 PM UTC+11, Jameson wrote: > > > > On Mon Feb 09 2015 at 7:59:20 PM <[email protected] <javascript:>> wrote: > >> >> >> On Tuesday, February 10, 2015 at 1:25:15 AM UTC+11, Simon Danisch wrote: >>> >>> Someone else has to answer this. I suppose it's not that bad. The >>> immutables are getting copied into the heap allocated memory from the >>> array, which should be fairly okay to mutate?! >>> But I seriously don't know the details on this. From a semantic >>> viewpoint it's definitely not okay! >>> >> >> WARNING: if you use unsafe practices to mutate an immutable object you >> enter the realms of undefined behaviour where the compiler/runtime can do >> anything it likes from sending your details to Scammers-R-Us to eating your >> shorts. Seriously if anything in the compiler optimisations, the runtime >> or garbage collector is changed to depend on immutables being, well, >> immutable, then it will cause unknown issues. And there is no guarantee >> that this hasn't happened already, just your specific code hasn't hit it >> ... yet. >> > Semantically, modifying a field of the immutable in the array is the same > as reading the whole thing, creating a new immutable with one field > modified, and writing it back to the array. The only thing you are likely > missing by hacking unsafe_store in this manner is a missing TBAA > annotation. That's fairly unlikely to matter here though (more about this > below). >
This depends on what is the immutable object. I am assuming that its the memory inside the array, not some heap object pointed to by the array. If its the latter ignore most of my rant, but I understand (from your excellent documentation referenced below) its the former for at least some immutable structs. If its the memory inside the array then copying another immutable with different field values over it changes the immutable, its only existence is in the array slot, so effectively arrays of immutables are immutable, except for push pop etc. So a user who makes a local copy of a field of an immutable is stuffed (well, its immutable so the field can't change right?), so is the compiler which hoists x = immutable.field out of a loop, as is LLVM if it understands immutable and keeps field values in registers. As you note AA is supposed to fix this, but if its missing then the problem remains, and the user copy isn't subject to AA correction anyway. > >> Things that may be an issue include: >> >> 1. Julia now has a generational GC, does it assume immutable >> intergenerational pointers cannot change and doesn't rescan for them? >> > If your type contains pointers to other non-isbits types, you are probably > just as well off using a type. They are stored the same. Only `isbits` > types get the special inline-storage behavior. > And then you don't mutate an immutable, but it would be good to document this restriction. > > >> >> 2. Does the compiler generate the correct GC write barriers for unsafe >> operations and does it ignore immutables since they can't change? >> > No, it doesn't generate write barriers during calls to unsafe_store. > Although if you are writing bits data inside the immutable, it hardly > matters, since it doesn't need a write barrier to do that. > Yes, I was thinking of pointers, but your exclusion above cures that. > > >> >> 3. Do optimisations use copies of immutables as implied in the manual, so >> you don't mutate what you think you mutate? >> > Quite frequently. But if you mutate the array data, you mutate the array > data, no questions asked. What you definitely shouldn't do is grab a > pointer to your immutable (via pointer_from_objref) and try to mutate > that (but trying to grab a pointer to an immutable should become an error > in a future version of Julia anyways). > Again cached copies should be invalidated somehow. > > >> >> 4. Does the generated code *not* use copies where it normally might, >> since it doesn't matter if the object is immutable, so again you don't >> mutate what you expected to? >> > This is possible. But it would require LLVM to decide that the array slot > itself couldn't have been re-assigned. Any intervening function call is > typically enough to block optimization on assuming the array data hasn't > changed. But TBAA (type-based alias analysis) is probably more likely to > bite you here, since LLVM won't consider the offset store to affect the > immutable type it lives inside. > Yes, which leads to the above problems. > >> 1. and 2. require the GC to run during the lifetime of the mutated >> immutable, maybe you have been lucky and that hasn't happened ... yet. >> >> 3. and 4. the behaviour depends on what other use is made of the >> immutable, again maybe you have been lucky. >> >> As for passing immutables to C, well was always going to be risky, C just >> doesn't understand immutable. If as suggested by Jameson mutables are >> layout compatible with C then the faster the habit of using immutables for >> that dies the better. >> > There's nothing risky about it. Types have the same layout and the same > behavior when getting passed to C regardless of whether they are declared > mutable or immutable. The ccall code hardly even checks the `.mutable` > field of the type (and that code hopefully will be deprecated soon, since > it doesn't really benefit the user). > Its risky in that a mutation inside the C library won't be detected by AA either. > > Often, however, `immutable` provides a better representation of a C-struct > and is therefore necessary for compatibility (unless the only usages of the > type are by-pointer). > Thats probably the problematic design decision. Hindsight (that wonderful thing) might suggest a C compatible definition for types that are passed to C (somewhat like the nebulous "standard layout" structs in C++11). > > The updated documentation I mentioned above is part of my > ccall-enhancement pull request (which makes calling c functions that > require or return by-value structs actually work correctly). It can be read > at: > > https://github.com/JuliaLang/julia/blob/jn/ccall3/doc/manual/calling-c-and-fortran-code.rst > . > Nice docs. Cheers Lex > > Regards. > > >> >> Train is approaching my stop, so thats enough for this rant. >> >> Cheers >> Lex >> >> >>> >>> 2015-02-09 15:15 GMT+01:00 Mauro <[email protected]>: >>> >> So, this actually modifies an immutable, right? This doesn't seem >>>> right, aren't they supposed to be immutable? >>>> >>>> On Mon, 2015-02-09 at 14:29, Simon Danisch <[email protected]> wrote: >>>> > You can actually modify immutables in an array directly. I actually >>>> > implemented this in GLAbstraction for my own usage. >>>> > I'm implementing FixedSizeArrays.jl >>>> > <https://github.com/SimonDanisch/FixedSizeArrays.jl> at the moment, >>>> which >>>> > will include this feature (In the feature list I call >>>> > it "setindex!/getindex for arrays of FSA (e.g. easy access to single >>>> > fields)") >>>> > This only works for immutables as it needs the tightly packed memory >>>> layout >>>> > of the array... Sadly, immutability doesn't occur in the type >>>> hierarchy, so >>>> > the setindex! that I'm talking about can't be written to work for any >>>> > immutable. >>>> > So this means, you either need to inherit from FixedSizeArrays to use >>>> this, >>>> > or copy the code (when it's transfered to fixedsizearrays). >>>> > Maybe we can open an issue at Julia, to see if we can get something >>>> like >>>> > this into base. >>>> > >>>> > Am Sonntag, 8. Februar 2015 21:35:20 UTC+1 schrieb >>>> [email protected]: >>>> >> >>>> >> I would like to request the following language feature: a function or >>>> >> macro to modify a field of an immutable inside a container. >>>> Consider: >>>> >> >>>> >> immutable T >>>> >> fielda::Int >>>> >> fieldb::Int >>>> >> fieldc::Int >>>> >> end >>>> >> >>>> >> function modify_fieldc!(x::Array{T,1}, sub::Int, newc::Int) >>>> >> x[sub] = T(x[sub].fielda, x[sub].fieldb, newc) >>>> >> end >>>> >> >>>> >> This function modifies one field of an immutable object that sits >>>> inside a >>>> >> container. The above construct, namely: >>>> >> x[sub] = T(x[sub].field1, x[sub].field2, ... , newval, ... >>>> >> x[sub].fieldn) >>>> >> occurs rather frequently in my code. It is not very readable and is >>>> also >>>> >> fragile in the case that I modify my code and put more fields in T >>>> later. >>>> >> It would be much nicer if there were a universal function like this: >>>> >> modifyField!(x, sub, fieldc, newc) >>>> >> >>>> >> Note that I declared T to be 'immutable' rather than 'type' for >>>> >> performance reasons-- I prefer the data in the array x to be packed >>>> in >>>> >> memory rather than accessed with pointers. If T were a 'type' then >>>> >> obviously the problem would go away. >>>> >> >>>> >> Maybe it is already possible to write a function or macro for this >>>> purpose >>>> >> in the existing language? >>>> >> >>>> >> Thanks, >>>> >> Steve Vavasis >>>> >> >>>> >> >>>> >>>>
