Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
On Sunday, April 10, 2016 09:03:06 AM Fred wrote: > A huge size difference ! I have to read my array from data file so I > suppose it is like Y and X is only for simulations ? There turn out to be many situations in which you can take shortcuts if you know the values are sorted in increasing order with no skips in them. For example, a[1:5] has better performance than a[[1:5;]], even though they yield the same answer. One of julia's strengths is that you can leverage this knowledge very effectively. So don't convert to an array unless you have some reason that you want/need to. But yes, any data you read from a data file will likely be an array. Best, --Tim > > Le dimanche 10 avril 2016 17:50:02 UTC+2, Tim Holy a écrit : > > Just FYI: > > > > julia> x = 1:0.1:100 > > 1.0:0.1:1.0e6 > > > > julia> y = collect(x); # this is the same as y = [x;] > > > > julia> sizeof(x) > > 32 > > > > julia> sizeof(y) > > 7928 > > > > --Tim
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
A huge size difference ! I have to read my array from data file so I suppose it is like Y and X is only for simulations ? Le dimanche 10 avril 2016 17:50:02 UTC+2, Tim Holy a écrit : > > Just FYI: > > julia> x = 1:0.1:100 > 1.0:0.1:1.0e6 > > julia> y = collect(x); # this is the same as y = [x;] > > julia> sizeof(x) > 32 > > julia> sizeof(y) > 7928 > > --Tim >
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Just FYI: julia> x = 1:0.1:100 1.0:0.1:1.0e6 julia> y = collect(x); # this is the same as y = [x;] julia> sizeof(x) 32 julia> sizeof(y) 7928 --Tim On Sunday, April 10, 2016 07:26:06 AM Fred wrote: > Maybe my array is too small to see a difference, but if I increase the size > I will lack of RAM ;) > > julia> x = 1:0.1:100 > 1.0:0.1:1.0e6 > > julia> @time searchsorted(x, 8.22) > 0.045590 seconds (33.21 k allocations: 1.535 MB) > 74:73 > > julia> @time searchsorted(x, 8.22) > 0.05 seconds (8 allocations: 288 bytes) > 74:73 > > julia> @time searchsorted(x, 8.22) > 0.05 seconds (8 allocations: 288 bytes) > 74:73 > > julia> @time closest_index(x,8.22) > 0.103219 seconds (4.37 k allocations: 222.884 KB) > 73 > > julia> @time closest_index(x,8.22) > 0.095684 seconds (4 allocations: 160 bytes) > 73 > > julia> @time dicotomy(x, 8.22) > 0.009142 seconds (3.45 k allocations: 173.973 KB) > (73,74) > > julia> @time dicotomy(x, 8.22) > 0.05 seconds (5 allocations: 192 bytes) > (73,74) > > julia> @time dicotomy(x, 8.22) > 0.04 seconds (5 allocations: 192 bytes) > (73,74) > > > > Even better: get rid of the brackets around 1:0.1:10, and you'll be > > > that > > much more impressed. > > > > --Tim
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Maybe my array is too small to see a difference, but if I increase the size I will lack of RAM ;) julia> x = 1:0.1:100 1.0:0.1:1.0e6 julia> @time searchsorted(x, 8.22) 0.045590 seconds (33.21 k allocations: 1.535 MB) 74:73 julia> @time searchsorted(x, 8.22) 0.05 seconds (8 allocations: 288 bytes) 74:73 julia> @time searchsorted(x, 8.22) 0.05 seconds (8 allocations: 288 bytes) 74:73 julia> @time closest_index(x,8.22) 0.103219 seconds (4.37 k allocations: 222.884 KB) 73 julia> @time closest_index(x,8.22) 0.095684 seconds (4 allocations: 160 bytes) 73 julia> @time dicotomy(x, 8.22) 0.009142 seconds (3.45 k allocations: 173.973 KB) (73,74) julia> @time dicotomy(x, 8.22) 0.05 seconds (5 allocations: 192 bytes) (73,74) julia> @time dicotomy(x, 8.22) 0.04 seconds (5 allocations: 192 bytes) (73,74) Even better: get rid of the brackets around 1:0.1:10, and you'll be > that > much more impressed. > > --Tim > >
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Even better: get rid of the brackets around 1:0.1:10, and you'll be that much more impressed. --Tim On Sunday, April 10, 2016 07:16:16 AM Fred wrote: > Hi, > > I post here my best solution taking advantage that the array is sorted. I > expected to be a lot much faster than other solutions, but not really. > I am very impressed by the speed of searchsorted > > > x = [1:0.1:100] > val = 8.22 > > function dicotomy(x, val) > a = start(eachindex(x)) > b = length(x) > j = length(x) > dxbest = abs(x[a]-val) > dx = dxbest > > while true > dx < dxbest ? dxbest = dx : 1 > j = round(Int,(a+b)/2) > dx = x[j]-val > x[j]-val < 0 ? a = j : b = j > abs(a-b) < 2 && break > end > return a,b > end > > @time dicotomy(x, 8.22) > 0.04 seconds (5 allocations: 192 bytes) > (73,74) > > > @time searchsorted(x, 8.22) > 0.05 seconds (7 allocations: 240 bytes) > > @time closest_index(x,8.22) > 0.027618 seconds (4 allocations: 160 bytes) > 73
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Hi, I post here my best solution taking advantage that the array is sorted. I expected that solution to be a lot much faster than other solutions, but not really. Indeed I am very impressed by the speed of searchsorted : x = [1:0.1:100] val = 8.22 function dicotomy(x, val) a = start(eachindex(x)) b = length(x) j = length(x) dxbest = abs(x[a]-val) dx = dxbest while true dx < dxbest ? dxbest = dx : 1 j = round(Int,(a+b)/2) dx = x[j]-val x[j]-val < 0 ? a = j : b = j abs(a-b) < 2 && break end return a,b end @time dicotomy(x, 8.22) 0.04 seconds (5 allocations: 192 bytes) (73,74) @time searchsorted(x, 8.22) 0.05 seconds (7 allocations: 240 bytes) @time closest_index(x,8.22) 0.027618 seconds (4 allocations: 160 bytes) 73
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Hi, I post here my best solution taking advantage that the array is sorted. I expected to be a lot much faster than other solutions, but not really. I am very impressed by the speed of searchsorted : x = [1:0.1:100] val = 8.22 function dicotomy(x, val) a = start(eachindex(x)) b = length(x) j = length(x) dxbest = abs(x[a]-val) dx = dxbest while true dx < dxbest ? dxbest = dx : 1 j = round(Int,(a+b)/2) dx = x[j]-val x[j]-val < 0 ? a = j : b = j abs(a-b) < 2 && break end return a,b end @time dicotomy(x, 8.22) 0.04 seconds (5 allocations: 192 bytes) (73,74) @time searchsorted(x, 8.22) 0.05 seconds (7 allocations: 240 bytes) @time closest_index(x,8.22) 0.027618 seconds (4 allocations: 160 bytes) 73
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
On Sun, 2016-04-10 at 15:24, Fredwrote: > That's true ! But why a loop is faster in a function ? :) Check out: http://docs.julialang.org/en/release-0.4/manual/performance-tips/#avoid-global-variables >> >> I seem to recall that your example loop was not in a function(?) If so, >> that makes it lots slower. >> >>
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
That's true ! But why a loop is faster in a function ? :) > > I seem to recall that your example loop was not in a function(?) If so, > that makes it lots slower. > >
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
> I tested my loop monre than 2 times as it is written in my post and I have > always the same results. The function Tim Holy posted is much faster, I > posted the results above :) I seem to recall that your example loop was not in a function(?) If so, that makes it lots slower. >> Probably you are doing this wrong; it shouldn't be allocating so much >> memory. Is your loop using global variables? Did you remember to time it >> twice (since the first time you call it there is compilation overhead.) >> Did you try the function Tim Holy posted? >>
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
I tested my loop monre than 2 times as it is written in my post and I have always the same results. The function Tim Holy posted is much faster, I posted the results above :) > Probably you are doing this wrong; it shouldn't be allocating so much > memory. Is your loop using global variables? Did you remember to time it > twice (since the first time you call it there is compilation overhead.) > Did you try the function Tim Holy posted? >
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
On Sunday, April 10, 2016 at 8:30:48 AM UTC-4, Fred wrote: > > my loop solution : > 0.000419 seconds (547 allocations: 708.797 KB) > 0.02135 -> 73 > Probably you are doing this wrong; it shouldn't be allocating so much memory. Is your loop using global variables? Did you remember to time it twice (since the first time you call it there is compilation overhead.) Did you try the function Tim Holy posted?
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Thank you very much Mauro ! searchsorted is the simplest solution and one of the fastest but it gives two indices so another comparison is needed to find the closest value : @time searchsorted(x, 8.22) 0.04 seconds (7 allocations: 240 bytes) 74:73 abs(x[73] - 8.22) > abs(x[74] - 8.22) ? i = 74 : i = 73 73
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Thank you very much ! I give you the results : my loop solution : 0.000419 seconds (547 allocations: 708.797 KB) 0.02135 -> 73 @time closest_index(x,8.22) 0.03 seconds (4 allocations: 160 bytes) 73 @time for (i,x) in enumerate(array)... 0.000181 seconds (821 allocations: 19.953 KB) 738.20.02135 @time reduce((x,y)->x[2](x,abs(y-8.22)),1:length(x),x)) 0.005890 seconds (892 allocations: 24.617 KB) Of course in my particular situation the array is sorted, so in that case I although think about using dichotomy
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
If your array is sorted, as your example suggests, there maybe faster methods, binary search comes to mind (implemented in searchsorted). Also, if the array is unsorted but you need to look up many values, it might be worth sorting it first. Mauro On Sun, 2016-04-10 at 13:40, Fredwrote: > Hi, > > I am looking for the most efficient (fastest) way to find the indice of the > element with the nearest value of a float in an array. > > x = [1:0.1:10] > > julia> x > 91-element Array{Float64,1}: > 1.0 > 1.1 > 1.2 > 1.3 > 1.4 > ⋮ > 9.4 > 9.5 > 9.6 > 9.7 > 9.8 > 9.9 > 10.0 > > It is very easy to find the indice of an exact value of x, for example 8.2 > > julia> find(x .== 8.2) > 1-element Array{Int64,1}: > 73 > > But if I want the indice of the closest value of 8.22 > > julia> minimum(abs(x-8.22)) > 0.02135 > > julia> find(x .== minimum(abs(x-8.22))) > 0-element Array{Int64,1} > > > Of course it is easy to do that with a loop but is it the fastest solution ? > > min_i = 0 > min_x = 1.0 > > for i=[1:length(x)] >e = abs(collect(x)[i] - 8.22) >if e < min_x > min_x = e > min_i = i >end > end > > println(min_x, " -> ", min_i) > 0.02135 -> 73 > > > Thanks for your comments !
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
Fred Yes, the fastest way is a loop. (It might be possible to extend e.g. `minimum` by a "predicate" argument, but that would require a change to the base library.) You would write the loop slightly differently, though: ```Julia mini = 0 minx = 0.0 mindx = typemax(Float64) for (i,x) in enumerate(array) dx = abs(x - x0) if dx < mindx mini, minx, mindx = i, x, dx end end mini, minx, mindx ``` This will return `imin=0` for empty inputs. -erik On Sun, Apr 10, 2016 at 7:40 AM, Fredwrote: > Hi, > > I am looking for the most efficient (fastest) way to find the indice of the > element with the nearest value of a float in an array. > > x = [1:0.1:10] > > julia> x > 91-element Array{Float64,1}: > 1.0 > 1.1 > 1.2 > 1.3 > 1.4 > ⋮ > 9.4 > 9.5 > 9.6 > 9.7 > 9.8 > 9.9 > 10.0 > > It is very easy to find the indice of an exact value of x, for example 8.2 > > julia> find(x .== 8.2) > 1-element Array{Int64,1}: > 73 > > But if I want the indice of the closest value of 8.22 > > julia> minimum(abs(x-8.22)) > 0.02135 > > julia> find(x .== minimum(abs(x-8.22))) > 0-element Array{Int64,1} > > > Of course it is easy to do that with a loop but is it the fastest solution ? > > min_i = 0 > min_x = 1.0 > > for i=[1:length(x)] >e = abs(collect(x)[i] - 8.22) >if e < min_x >min_x = e >min_i = i >end > end > > println(min_x, " -> ", min_i) > 0.02135 -> 73 > > > Thanks for your comments ! > -- Erik Schnetter http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Find the indice of the element with the nearest value of a float in an array
indmin(abs(x-val)) is easy and pretty good, but it does create two temporaries. Faster would be function closest_index(x, val) ibest = start(eachindex(x)) dxbest = abs(x[ibest]-val) for I in eachindex(x) dx = abs(x[I]-val) if dx < dxbest dxbest = dx ibest = I end end ibest end This should not allocate any memory and is likely the fastest. (It might be slightly faster with @inbounds, of course...) It would be possible to create an indmin(f, x) that applies f to each element of x and returns the index of the minimum; this would be efficient in the development version of julia but not julia-0.4. Best, --Tim On Sunday, April 10, 2016 04:40:07 AM Fred wrote: > Hi, > > I am looking for the most efficient (fastest) way to find the indice of the > element with the nearest value of a float in an array. > > x = [1:0.1:10] > > julia> x > 91-element Array{Float64,1}: > 1.0 > 1.1 > 1.2 > 1.3 > 1.4 > ⋮ > 9.4 > 9.5 > 9.6 > 9.7 > 9.8 > 9.9 > 10.0 > > It is very easy to find the indice of an exact value of x, for example 8.2 > > julia> find(x .== 8.2) > 1-element Array{Int64,1}: > 73 > > But if I want the indice of the closest value of 8.22 > > julia> minimum(abs(x-8.22)) > 0.02135 > > julia> find(x .== minimum(abs(x-8.22))) > 0-element Array{Int64,1} > > > Of course it is easy to do that with a loop but is it the fastest solution ? > > min_i = 0 > min_x = 1.0 > > for i=[1:length(x)] >e = abs(collect(x)[i] - 8.22) >if e < min_x >min_x = e >min_i = i >end > end > > println(min_x, " -> ", min_i) > 0.02135 -> 73 > > > Thanks for your comments !
[julia-users] Find the indice of the element with the nearest value of a float in an array
Hi, I am looking for the most efficient (fastest) way to find the indice of the element with the nearest value of a float in an array. x = [1:0.1:10] julia> x 91-element Array{Float64,1}: 1.0 1.1 1.2 1.3 1.4 ⋮ 9.4 9.5 9.6 9.7 9.8 9.9 10.0 It is very easy to find the indice of an exact value of x, for example 8.2 julia> find(x .== 8.2) 1-element Array{Int64,1}: 73 But if I want the indice of the closest value of 8.22 julia> minimum(abs(x-8.22)) 0.02135 julia> find(x .== minimum(abs(x-8.22))) 0-element Array{Int64,1} Of course it is easy to do that with a loop but is it the fastest solution ? min_i = 0 min_x = 1.0 for i=[1:length(x)] e = abs(collect(x)[i] - 8.22) if e < min_x min_x = e min_i = i end end println(min_x, " -> ", min_i) 0.02135 -> 73 Thanks for your comments !
[julia-users] find source location for type definition
Is there an equivalent of @edit function(arg1, ...) for type definitions? @edit is so nice in Emacs. Currently I am using M-x find-grep-dired RET \(type\|immutable\) Foo but something built-in for Julia would be very convenient. Best, Tamas
[julia-users] Find
`x .> 3` is a bit-array. The function form, which is probably what fits, is `x->x.>3` (the back-ticks are for quotation, not part of the code).
[julia-users] Find
when trying x=[0,1,3,4,6,1] find(x.>3,x) I always get the error: ERROR: MethodError: `find` has no method matching find(::BitArray{1}, ::Array{Int64,1}) Closest candidates are: find(::Function, ::AbstractArray{T,N}) find(::BitArray{N}) find(::Union{DenseArray{T,N},SubArray{T,N,A<:DenseArray{T,N},I<:Tuple{Vararg{Union{Colon,Int64,Range{Int64,LD}}) Am I doing sth plain wrong here?
[julia-users] find element in sorted list
What is the best way of telling whether an element `x` is present in a sorted list `a`? Using `searchsortedlist`, I also need to check that the element is present in the list. This is what I do function isinsortedlist(a, x) n = searchsortedfirst(a, x) n <= length(a) && a[n] == x end Shouldn't `searchsortedlist` return also a bool to tell whether the element is already in the list? Thank you, michele
Re: [julia-users] find element in sorted list
On Sat, Nov 7, 2015 at 4:34 PM, Milan Bouchet-Valatwrote: > Le samedi 07 novembre 2015 à 06:37 -0800, Michele Zaffalon a écrit : > > What is the best way of telling whether an element `x` is present in > > a sorted list `a`? > > > > Using `searchsortedlist`, I also need to check that the element is > > present in the list. This is what I do > > > > function isinsortedlist(a, x) > > n = searchsortedfirst(a, x) > > n <= length(a) && a[n] == x > > end > > > > Shouldn't `searchsortedlist` return also a bool to tell whether the > > element is already in the list? > Just use searchsorted(), which (as the docs say) "Returns an empty > range located at the insertion point if a does not contain values equal > to x". > Now that you mention it, it is obvious. Thank you. > > > Regards > > > Thank you, > > michele >
[julia-users] Find linearly independent subset within a set of vectors without constructing a matrix
Given a set of vectors v1, , vn, I'd like to construct a matrix where columns correspond to a linearly independent subset of these vectors. Currenly, I form the matrix hcat(v1, ..., vn), use qrfact!() on it, check the diagonal of the :R matrix and construct a new matrix as hcat(vj,...) I'm looking for a way that would use less memory. Is there a more memory efficient way ? (for instance a way to avoid the construction of an intermediary matrix when finding the subset of vectors, or a way to delete columns in place in a matrix)
Re: [julia-users] Find sequence in array?
Le vendredi 11 septembre 2015 à 16:03 -0700, cormull...@mac.com a écrit : > Is there a Julia function/method to find the location(s) of a > sequence of elements in a 1-D array? > > With strings, you can do: > > search("longstring", "str") > > 5:7 > > so with arrays it would hopefully be something like: > > searcharray( [1, 3, 5, 7, 9, 11, 13], [5, 7, 9]) > > 3:5 It doesn't exist at the moment, though an implementation would certainly be accepted. It could be a method for search(), or a new function called findseq(): https://github.com/JuliaLang/julia/issues/10593#issuecomment-90249829 If you want to give it a try, you could generalize a bit the existing method for search() which only works on UInt8 arrays: julia> search(UInt8[1, 3, 5, 7, 9, 11, 13], UInt8[5, 7, 9], 1) 3:5 Regards
Re: [julia-users] Find sequence in array?
Hi Milan - thanks for the clues! I found `Base._searchindex`, which work for integers: julia> a = rand(1:10, 100); julia> Base._searchindex(a, [19272, 52257], 1) 86 It's pretty quick, too.
[julia-users] Find sequence in array?
Is there a Julia function/method to find the location(s) of a sequence of elements in a 1-D array? With strings, you can do: search("longstring", "str") 5:7 so with arrays it would hopefully be something like: searcharray( [1, 3, 5, 7, 9, 11, 13], [5, 7, 9]) 3:5
Re: [julia-users] find function in sparse matrix
Le jeudi 11 septembre 2014 à 20:12 -0700, i.pallikarakis...@alumni.lboro.ac.uk a écrit : Hi everyone, I am new to Julia and just upgraded from 0.2.1 to 0.3.0 and found the following issue : find function is no longer working on sparse matrices. For example A=speye(Bool,10) find(x-x==true,A) -0-element Array{Int64,1} find(x-x==true,full(A)) -10-element Array{Int64,1}: 1 12 23 34 45 56 67 78 89 100 Is this normal ? Thanks for the reproducible report. I don't think this is normal indeed. I've made a pull request to fix this here: https://github.com/JuliaLang/julia/pull/8323 But the code you're calling above is equivalent to find(A), which works fine, and which is specialized for sparse matrices, meaning it's more efficient. Regards
[julia-users] find function in sparse matrix
Hi everyone, I am new to Julia and just upgraded from 0.2.1 to 0.3.0 and found the following issue : find function is no longer working on sparse matrices. For example A=speye(Bool,10) find(x-x==true,A) -0-element Array{Int64,1} find(x-x==true,full(A)) -10-element Array{Int64,1}: 1 12 23 34 45 56 67 78 89 100 Is this normal ? Thank you for your help, Ilias
[julia-users] find nearest non-zero element in matrix
Hi julia-users, I am looking for a function that takes a two-dimensional matrix and a set of matrix indices as inputs and spits out the nearest non-zero element in taxicab distance over the matrix index. I am asking for your advice because this function needs to be very fast as I have to do this search many times. Any ideas or advice would be greatly appreciated. Thanks!
Re: [julia-users] find if a pointer is NULL
testing against C_NULL is probably the easiest way: p == C_NULL On Sat, Jun 14, 2014 at 10:39 PM, J Luis jmfl...@gmail.com wrote: How do I test if a pointer is NULL? (did search the docs but couldn't find and answer) and BTW, I can't get this one either docs: pointer(*type*, *int*) Convert an integer to a pointer of the specified element type. julia pointer(Uint8,0) ERROR: no method pointer(Type{Uint8}, Int64) julia pointer(Uint8,uint8(0)) ERROR: no method pointer(Type{Uint8}, Uint8) Thanks