I would love to figure out a way to bring the kind of automatic resource
and memory release that Rust has to Julia, but the cost is some fairly
finicky static compiler analysis that is ok for Rust's target demographic
but fairly clearly unacceptable for Julia general audience. What we'd need
is a more dynamic version of something like that. One idea I've had is to
indicate ownership and enforce it at run-time rather than compile time –
and eliminate run-time checks when we can prove that they aren't needed.
This could look something like having "owned references" to values versus
"borrowed references" and check that the object still exists if the
borrowed reference is used and raise an error if it doesn't. When an owned
reference to the thing goes out of scope, immediately finalize it. I'm not
sure how to indicate this, but it would be great to be able to write:
function foo(...)
fh = open(file)
# do stuff with fh
end # <= fh is automatically before the function returns
# it's a runtime error to access the fh object after this
Similarly, you could have something like this:
function bar(...)
a = @local Array(Int,10)
# do stuff with a
end # <= a is automatically freed before the function returns
# it's a runtime error to access the a object after this
Given these semantics, it would be relatively easy to alloca the actual
memory of the array, and only heap allocate the object itself, which could
then reference the stack allocated memory. This is tough to implement,
especially efficiently, but I have a bit of a suspicion that in Julia
mutable objects – and this only makes sense for mutable objects that are
inherently associated with a particular place in memory – are rarely
performance critical in Julia.
On Mon, Dec 15, 2014 at 11:15 PM, John Myles White <johnmyleswh...@gmail.com
> wrote:
> This is taking the thread off-topic, but conceptually such things are
> possible. But Rust has a very different set of semantics for memory
> ownership than Julia has and is doing a lot more analysis at compile-time
> than Julia is doing. So Julia would need to change a lot to be more like
> Rust. I've come to really adore Rust, so I'd like to see us borrow some
> ideas, but my personal sense is that Julia and Rust simply serve different
> niches and shouldn't really move towards one another too much lest each
> language wind up forsaking what makes it useful.
>
> -- John
>
>
> On Dec 15, 2014, at 8:43 PM, Eric Forgy <eric.fo...@gmail.com> wrote:
>
> Hi,
>
> I'm new to Julia and mentioned it to a friend who is more into systems
> than mathematical models and he mentioned his current "crush" is Rust,
> which is also built on LVVM. I may have totally missed the point, but IF I
> understand, Rust does away with garbage collection by "borrow blocking" at
> compile time. The question popped into my head whether we could turn off GC
> in Julia and check for problems at compile time. A google later, brought me
> to this thread. Is that a totally misguided idea?
>
> Best regards,
> Eric
>
> PS: You can tell I'm coming in with almost no background knowledge about
> compilers (or real languages for that matter), but am having fun learning.
> LVVM was developed at my alma mater (PhD in ECE - Computational
> Electromagnetics - from UIUC 2002). Go Illini! :)
>
> On Friday, February 22, 2013 7:11:32 PM UTC+8, Tim Holy wrote:
>>
>> Have you played with SProfile in the Profile package? It's rather good at
>> highlighting which lines, in your code and in base/, trigger the gc. Note
>> that
>> in my experience the gc does not seem to be triggered necessarily on big
>> allocations; for example, even allocating an array as
>> Array(Int, (3,5))
>> rather than
>> Array(Int, 3, 5)
>> can trigger the gc (I see lots of gc() calls coming from our Lapack code
>> for
>> this reason).
>>
>> Because I don't really know how the gc works, I'm not certain that kind
>> of
>> thing actually reflects a problem; perhaps it was just going to have to
>> call gc
>> on the next heap-allocation event, and (3,5) just happened to be the
>> lucky
>> candidate. But there's an open issue about this:
>> https://github.com/JuliaLang/julia/issues/1976
>>
>> Docs are here: https://github.com/timholy/Profile.jl
>> I think they're slightly out of date, but only in very minor ways.
>>
>> --Tim
>>
>>
>>
>> On Thursday, February 21, 2013 03:17:59 PM nathan hodas wrote:
>> > Here's the code that benefits from @nogc:
>> >
>> > Notice the iteration over a Dict and a Set. iscorrect() checks a field
>> of
>> > the Attempt type. I can tell by running this particular that the
>> garbage
>> > collection is running during the for loop.
>> > function meantime(userdata::Dict{Int,Set{Attempt}})
>> > usertimes = Dict{Int,Float64}()
>> > sizehint(usertimes,length(userdata))
>> > for (uid,attempts) in collect(userdata)
>> > s = 0.0;
>> > c = 0.0;
>> > ic = 0.0;
>> > for a in attempts
>> > ic = iscorrect(a)
>> > s += (a.tend - a.tstart)*ic;
>> > c += ic;
>> > end
>> > usertimes[uid] = s/c;
>> > end
>> > usertimes
>> > end
>> >
>> > This code has no benefit from @nogc, regardless of the kernel function
>> k1:
>> >
>> > function dostuff(input1,input2)
>> > output = similar(input1)
>> > len = length(input1)
>> > for i = 1:len
>> > x = input1[i]
>> > for j = 1:len
>> > y = input2[j]
>> > output[i] += k1(x,y)
>> > end
>> > end
>> > output
>> > end
>> >
>> > On Thursday, February 21, 2013 10:37:09 AM UTC-8, Stefan Karpinski
>> wrote:
>> > > Can you post some example code? Are you just iterating the Dict
>> object
>> > > with a for loop?
>> > >
>> > >
>> > > On Thu, Feb 21, 2013 at 1:35 PM, Stefan Karpinski
>> > > <ste...@karpinski.org<javascript:>>
>> > > > wrote:
>> > >> That's good information to have. I'll look into it.
>> > >>
>> > >>
>> > >> On Thu, Feb 21, 2013 at 1:13 PM, nathan hodas
>> > >> <nho...@gmail.com<javascript:>>>
>> > >> > wrote:
>> > >>> It's true that @nogc is not a panacea. For my particular function,
>> it
>> > >>> produces a robust 20x speed up, even after subsequent collection.
>> For
>> > >>> other seemingly similar functions, it has no effect at all. I don't
>> use
>> > >>> any
>> > >>> temporary arrays *that I'm aware of*, but it seems the iterators of
>> a
>> > >>> Dict
>> > >>> are doing something in the background.
>> > >>>
>> > >>> On Wednesday, February 20, 2013 3:44:24 PM UTC-8, Tim Holy wrote:
>> > >>>> The other thing you should check is whether you're allocating more
>> than
>> > >>>> you
>> > >>>> need to. I find that I can often reuse bits of memory, and that
>> can
>> > >>>> dramatically decrease the need for gc. In the long run that may
>> help
>> > >>>> you a
>> > >>>> _lot_ more than temporarily disabling gc, because at some point
>> you'll
>> > >>>> need to
>> > >>>> turn it on again.
>> > >>>>
>> > >>>> There are examples of this kind of thing scattered all over the
>> Julia
>> > >>>> code
>> > >>>> tree. Just because it was rather easy for me to find :-), here's
>> the
>> > >>>> patch I
>> > >>>> pushed to Zip in iterator.jl today:
>> > >>>> https://github.com/JuliaLang/**julia/commit/**
>> > >>>> 89ece095e8ea1fa166074306927c6c**c5f90060f7<https://github.
>> com/JuliaLang
>> > >>>> /julia/commit/89ece095e8ea1fa166074306927c6cc5f90060f7> It got
>> rid of
>> > >>>> two array allocations per iteration, by reusing one array and
>> > >>>> pre-allocating "scratch space" inside the type for the other.
>> > >>>>
>> > >>>> You can also reuse bits of memory by writing functions with a
>> syntax
>> > >>>> like
>> > >>>>
>> > >>>> this:
>> > >>>> function myfunc(out::Array, arg1, arg2)
>> > >>>>
>> > >>>> where out is pre-allocated storage for the output. This helps if
>> you'll
>> > >>>> be
>> > >>>> calling the same function many times and always need an array of
>> the
>> > >>>> same type
>> > >>>> and dimensions. Our matrix multiplication routines can be used in
>> this
>> > >>>> way.
>> > >>>>
>> > >>>> --Tim
>>
>
>
