On Monday, April 20, 2015 11:23:48 AM Peter Brady wrote:
Can you explain why this fixes the issue? My understanding (which is
apparently wrong) is that this kind of parameterization wasn't necessary
since the compiler determines the types when the function is called and
compiles it for that
That sounds like a good way to familiarize myself with the code base.
Thanks again for helping me with your debug-fu. I have to admit that I was
starting to lose faith in julia but I'm glad it turned out to be user error.
As a final note here, on 0.4, if I make my UnsafeSlice immutable the
I wonder if your color printing isn't working? Because for me @code_warntype
raised some big red flags about setk_Unsafe(a). This fixed the problems:
function UnsafeSlice{T,N}(a::Array{T,N}, slicedim::Int, start=1)
p = pointer(a)
str = stride(a, slicedim)
UnsafeSlice{T, N,
Thank you!
julia include(test.jl)
test_all (generic function with 1 method)
julia test_all(5);
test_stride
elapsed time: 1.835237919 seconds (0 bytes allocated)
test_view
elapsed time: 9.453440318 seconds (42 MB allocated, 0.10% gc time in 2
pauses with 1 full sweep)
test_unsafe
elapsed time:
Here's the results of running --track-allocation=user --inline=no on 0.4.
Note that I also deleted all the macros which were affecting the reported
line numbers.
I have three questions base on the data below:
1. Why is the call to size which indexes into a tuple so expensive
2. Why is
First, you need to run it twice, see
http://docs.julialang.org/en/latest/manual/profile/#memory-allocation-analysis
and the part about clear_malloc_data.
Second, I think you have a bug:
size(s::UnsafeSlice) = size(s.size)
should presumably be
size(s::UnsafeSlice) = s.size
--Tim
On
Sorry, I didn't notice you'd included the test function.
What happens if you make
UnsafeSlice{T,N, P:Ptr} : AbstractArray{T,N}
rather than
UnsafeSlice{T,N, P:Ptr} : AbstractArray
?
Also try @code_warntype on those functions.
If these don't work, can you paste in a version of your code that
What about something like unsafe_updateview!(view, indices...) ?
It could be used like this (pseudocode):
view = unsafe_view(data, 1, 1, :) # to construct / allocate
for i in ..., j in ...
unsafe_updateview!(view, i, j, :)
# use view
end
In the trivial case of unsafe_view(data,
Sorry to be slow to chime in here, but the tuple overhaul has landed and they
are still not zero-cost:
function sumcols(A)
s = 0.0
for j = 1:size(A,2)
Aj = slice(A, :, j)
for i = 1:length(Aj)
s += Aj[i]
The latest version of ArrayViews (v0.6.0) now provides unsafe views (that
maintain raw pointers instead of the parent array). See
https://github.com/JuliaLang/ArrayViews.jl#view-types
You may see whether this makes your code more performant. Be careful, you
should make sure that unsafe views
—track-allocation still requires guesswork, as optimizations can move the
allocation to a different place than you would expect.
On April 19, 2015 at 4:36:19 PM, Peter Brady (petertbr...@gmail.com) wrote:
So I discovered the --track-allocation option and now I am really confused:
Here's my
My benchmark shows that element indexing has been as fast as it can be for
array views (or subarrays in Julia 0.4).
Now the problem is actually the construction of views/subarrays. To
optimize the overhead of this part, the compiler may need to introduce
additional optimization.
Dahua
On
It's not just escape analysis, as this (new) issue demonstrates:
https://github.com/JuliaLang/julia/issues/10899
--Tim
On Sunday, April 19, 2015 12:33:51 PM Sebastian Good wrote:
Their size seems much decreased. I’d imagine to totally avoid allocation in
this benchmark requires an optimization
@Dahua, thanks for adding an unsafeview! I appreciate how quickly this
community responds.
I've added the following function to my test.jl script
function setk_unsafeview{T}(a::Array{T,3})
for j=1:size(a,2),i=1:size(a,1)
off = sub2ind(size(a), i, j, 1)
update(unsafe_view(a,
So I discovered the --track-allocation option and now I am really confused:
Here's my session:
$ julia --track-allocation=all
_
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_)| Documentation: http://docs.julialang.org
_ _ _| |_ __ _
Optimizing the creation of many small structures during execution typically
comes down to either cleverly eliminating the need to allocate them in the
first place (via escape analysis, and the like) or making the first generation
of the garbage collector wickedly fast. I understand both of
Their size seems much decreased. I’d imagine to totally avoid allocation in
this benchmark requires an optimization that really has nothing to do with
subarrays per se. You’d have to do an escape analysis and see that Aj never
left sumcols. Not easy in practice, since it’s passed to slice and
So I've done some experimenting I think my use case easily lends itself to
a custom datatype (basically a Ptr with an offset and stride) named
'UnSafeSlice' below. The timings are pretty close to simply passing the
array and offset/stride but the memory allocation is way off.
I've posted my
Constructing a view would cause memory allocation. The compiler has not
been able to completely optimize this out.
I have been considering introducing something like a ``local_view`` that
caches the base pointer. The concern is that such views are dangerous is
passed around -- it does not
As far as I have measured it sub in 0.4 is still not cheap, as it provides the
flexibility to deal with all kinds of strides and offsets, and the view object
itself thus has a certain size. See
https://github.com/rened/FunctionalData.jl#efficiency for a simple analysis,
where the speed is
Inorder to write some differencing algorithms in a semi-dimensional
agnostic manner the code I've written makes heavy use of subarrays which
turn out to be rather costly. I've noticed some posts on the cost of
subarrays here and that things will be better in 0.4. Can someone comment
on how
Thanks for the links. I'll check out ArrayViews as it looks like what I
was going to do manually without wrapping it in a type.
By semi-dim agnostic I meant that the differencing algorithm itself only
cares about one dimension but that dimension is different for different
directions. Only a
This was discussed a few weeks ago
https://groups.google.com/d/msg/julia-users/IxrvV8ABZoQ/uWZu5-IB3McJ
I think the bottom line is that the current implementation *should* be
'zero-cost' once a set of planned improvements and optimizations take
place. One of the key ones is a tuple overhaul.
23 matches
Mail list logo
