Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Sunday, 25 October 2015 07:05:03 UTC-7, Christoph Ortner wrote: > > > very nice example - thank you - but I notice that you use linrange :). > > Thanks, Christoph > Man I wish they had called it linrange ;) with type LinRange. Still I am at peace with the new way it works. Hopefully it will get faster in the future, and with the upcoming printing changes I think it will become a minor headache in corner cases at worst (in which case using collect will be no problem). I don't think the python range/xrange is a good comparison as that is a function that was almost always used for looping in that language so of course the iterator is the best idea. linspace is often used for array creation in languages like Matlab. Gabriel
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
very nice example - thank you - but I notice that you use linrange :). Thanks, Christoph
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Memory usage can be an issue if you're simply iterating over a long range, e.g. julia> function pi_half_montecarlo(n) mysum = 0 for x in linrange(0,1,n) y = rand() if x*x + y*y < 1 mysum += 1 end end return 4.0*mysum/n end pi_half_montecarlo (generic function with 1 method) julia> pi_half_montecarlo(10); @time pi_half_montecarlo(1000_000_000)/pi elapsed time: 17.980499945 seconds (112 bytes allocated) 1.009913475585 This would use 8 GB of RAM if one were to write linspace in Julia 0.3 (or collect(linspace(...)) in Julia 0.4). Iterating over FloatRanges/LinSpaces may be less common than iterating over UnitRanges, but it's still not that far fetched. The iterator vs array issue was considered important enough to break compatibility for "range" between Python 2 and Python 3. On Thursday, October 22, 2015 at 5:15:17 PM UTC+2, Christoph Ortner wrote: > > > > On Thursday, 22 October 2015 10:24:50 UTC+1, Andras Niedermayer wrote: >> >> You're making a good point about an Array being sometimes faster than a >> LinSpace. But a LinSpace gets you a factor N improvement in terms of memory >> efficiency for a size N range, an Array only gets you a constant factor >> improvement in speed (the factor 15 being admittedly relatively large in >> this example). >> >> Memory efficiency typically matters more for usability in an exploratory >> interactive session: if my Julia session needs 5 GB RAM, a factor 3 >> increase of memory will crash my computer. If my code runs for 10 seconds >> in an interactive session, 30 seconds is mildly annoying, but not a deal >> breaker. (Obviously, you can construct different examples with memory/time >> where this is different. But my point is that inconvenience changes >> discontinuously in memory usage.) >> > > In my domain (PDEs and related), memory usage is hardly ever an issue for > one-dimensional arrays (as in I cannot for the life of me think of a case > where it is). I don't know about other domains of course. > > Christoph >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Hi Tim, Are you saying that if one removed bounds-checking from the LinSpace iterator, my code below would be much faster? (a) Why would there even have to be a bounds check in such an iterator? (b) That still won't let me use AppleAccelerate, which I suspect requires a plain old array. Christoph On Thursday, 22 October 2015 15:18:12 UTC+1, Tim Holy wrote: > > I may have said this earlier, but we just need > https://github.com/JuliaLang/julia/issues/7799. > > --Tim > > On Thursday, October 22, 2015 12:11:05 AM Christoph Ortner wrote: > > So here is a reason for keeping the linespace a vector: > > > > julia> t = linspace(0, 1, 1_000_000); > > julia> s = collect(t); > > julia> @time for n = 1:10; exp(t); end > > 0.209307 seconds (20 allocations: 76.295 MB, 3.42% gc time) > > julia> @time for n = 1:10; exp(s); end > > 0.054603 seconds (20 allocations: 76.295 MB, 17.66% gc time) > > julia> @time for n = 1:10; AppleAccelerate.exp(s); end > > 0.016640 seconds (40 allocations: 76.295 MB, 31.64% gc time) > > julia> @time for n = 1:10; AppleAccelerate.exp!(s,s); end > > 0.005702 seconds > > > > Now the natural response will be to say that most of the time I won't > care > > so much about performance, and when I do, then I can go optimise. But in > > truth the same can be said about keeping linspace abstract just because > it > > saves memory. (obviously it is not faster!) > > > > I think the argument for abstract types is very strong, but (in my > personal > > view) not at the expense of expected behaviour. > > > > Christoph > >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Thursday, 22 October 2015 10:24:50 UTC+1, Andras Niedermayer wrote: > > You're making a good point about an Array being sometimes faster than a > LinSpace. But a LinSpace gets you a factor N improvement in terms of memory > efficiency for a size N range, an Array only gets you a constant factor > improvement in speed (the factor 15 being admittedly relatively large in > this example). > > Memory efficiency typically matters more for usability in an exploratory > interactive session: if my Julia session needs 5 GB RAM, a factor 3 > increase of memory will crash my computer. If my code runs for 10 seconds > in an interactive session, 30 seconds is mildly annoying, but not a deal > breaker. (Obviously, you can construct different examples with memory/time > where this is different. But my point is that inconvenience changes > discontinuously in memory usage.) > In my domain (PDEs and related), memory usage is hardly ever an issue for one-dimensional arrays (as in I cannot for the life of me think of a case where it is). I don't know about other domains of course. Christoph
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
I may have said this earlier, but we just need https://github.com/JuliaLang/julia/issues/7799. --Tim On Thursday, October 22, 2015 12:11:05 AM Christoph Ortner wrote: > So here is a reason for keeping the linespace a vector: > > julia> t = linspace(0, 1, 1_000_000); > julia> s = collect(t); > julia> @time for n = 1:10; exp(t); end > 0.209307 seconds (20 allocations: 76.295 MB, 3.42% gc time) > julia> @time for n = 1:10; exp(s); end > 0.054603 seconds (20 allocations: 76.295 MB, 17.66% gc time) > julia> @time for n = 1:10; AppleAccelerate.exp(s); end > 0.016640 seconds (40 allocations: 76.295 MB, 31.64% gc time) > julia> @time for n = 1:10; AppleAccelerate.exp!(s,s); end > 0.005702 seconds > > Now the natural response will be to say that most of the time I won't care > so much about performance, and when I do, then I can go optimise. But in > truth the same can be said about keeping linspace abstract just because it > saves memory. (obviously it is not faster!) > > I think the argument for abstract types is very strong, but (in my personal > view) not at the expense of expected behaviour. > > Christoph
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Thursday, October 22, 2015 at 9:44:28 AM UTC-4, Gabriel Gellner wrote: > > A related discussion is about a special Ones type representing an array >> of 1, which would allow efficient generic implementations of >> (non-)weighted statistical functions: >> https://github.com/JuliaStats/StatsBase.jl/issues/135 >> >> But regarding zeros(), there might not be any compelling use case to >> return a special type. Anyway, if arrays are changed to initialize to >> zero [1], that function go could away entirely > > > lol never thought of this kind of special case. You could simply have a > "SameVector" object that just stores the value and the length. * + - ^ > would be easy to define and the space/(# of operations) savings could be > massive ;). With this you would have no need to special case zeros(n) > either just have it return a SameVector with value 0. We could even > generalize it to "BlockVector" for sequences of same values storing start > and stop locations. >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
> > A related discussion is about a special Ones type representing an array > of 1, which would allow efficient generic implementations of > (non-)weighted statistical functions: > https://github.com/JuliaStats/StatsBase.jl/issues/135 > > But regarding zeros(), there might not be any compelling use case to > return a special type. Anyway, if arrays are changed to initialize to > zero [1], that function go could away entirely lol never thought of this kind of special case. You could simply have a "SameVector" object that just stores the value and the length. * + - ^ would be easy to define and the space/(# of operations) savings could be massive ;). With this you would have no need to special case zeros(n) either just have it return a SameVector with value 0. We could even generalize it to "BlockVector" for sequences of same values storing start and stop locations.
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Le mercredi 21 octobre 2015 à 16:22 -0400, Spencer Russell a écrit :
> On Wed, Oct 21, 2015, at 04:07 PM, Gabriel Gellner wrote:
> > That doesn't feel like a reason that they can't be iterators,
> > rather that they might be slow ;) a la python. My point is not
> > about speed but the consistency of the language. Are there many
> > cases in Julia where there is a special type like this because it
> > is convenient/elegant to implement? This feels like a recipe for
> > madness, my guess is that this would be crazy rare.
> >
> > People wondered why people might mind that we get a LinSpace object
> > vs an Array. For me it is this strange feeling that I am getting a
> > special case that doesn't feel well motivated other than there is a
> > nice way to implement it (and that people, again, assumed that it
> > would largely be used for looping). If not all things can be made
> > consistently iterators when they are vector-like then why not have
> > a special function that returns this special type (like your
> > aforementioned linrange)? The fact that I lose my iterator when I
> > use certain functions but not others is a way that this
> > polymorphism that everyone is describing doesn't feel as nice to
> > me, since it will not compose in cases where it likely should,
> > outside of implementation details.
>
> Can you clarify with an example of when you lose the iterator? IMO
> that would be an example of breaking the AbstractArray contract and
> worthy of fixing.
>
> I see your point that there are other functions that return array
> -like objects that could also be implemented without fully
> constructing the array, e.g. `zeros(20)` could return a `Zeros{20}`
> object that acts like a length-20 array full of zeros. As far as I
> know there's no compelling design reason that's a bad idea, it's just
> that nobody has done it.
A related discussion is about a special Ones type representing an array
of 1, which would allow efficient generic implementations of
(non-)weighted statistical functions:
https://github.com/JuliaStats/StatsBase.jl/issues/135
But regarding zeros(), there might not be any compelling use case to
return a special type. Anyway, if arrays are changed to initialize to
zero [1], that function go could away entirely.
Regards
1: https://github.com/JuliaLang/julia/issues/9147
> The nice thing is that as long as people don't over-specify their
> type annotations that change can be done under-the-hood and it
> shouldn't break code. From a pedagogical standpoint I see how these
> sorts of things can add confusion and cause new folks to question
> their understanding, and more consistency reduces that conceptual
> friction.
>
> -s
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
You're making a good point about an Array being sometimes faster than a LinSpace. But a LinSpace gets you a factor N improvement in terms of memory efficiency for a size N range, an Array only gets you a constant factor improvement in speed (the factor 15 being admittedly relatively large in this example). Memory efficiency typically matters more for usability in an exploratory interactive session: if my Julia session needs 5 GB RAM, a factor 3 increase of memory will crash my computer. If my code runs for 10 seconds in an interactive session, 30 seconds is mildly annoying, but not a deal breaker. (Obviously, you can construct different examples with memory/time where this is different. But my point is that inconvenience changes discontinuously in memory usage.) On Thursday, October 22, 2015 at 9:12:22 AM UTC+2, Christoph Ortner wrote: > > P.S.: It occurred to me that I should also have tried this: > > @time for n = 1:10; AppleAccelerate.exp(collect(s)); end > 0.041035 seconds (60 allocations: 152.589 MB, 22.94% gc time) >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
P.S.: It occurred to me that I should also have tried this: @time for n = 1:10; AppleAccelerate.exp(collect(s)); end 0.041035 seconds (60 allocations: 152.589 MB, 22.94% gc time)
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
So here is a reason for keeping the linespace a vector: julia> t = linspace(0, 1, 1_000_000); julia> s = collect(t); julia> @time for n = 1:10; exp(t); end 0.209307 seconds (20 allocations: 76.295 MB, 3.42% gc time) julia> @time for n = 1:10; exp(s); end 0.054603 seconds (20 allocations: 76.295 MB, 17.66% gc time) julia> @time for n = 1:10; AppleAccelerate.exp(s); end 0.016640 seconds (40 allocations: 76.295 MB, 31.64% gc time) julia> @time for n = 1:10; AppleAccelerate.exp!(s,s); end 0.005702 seconds Now the natural response will be to say that most of the time I won't care so much about performance, and when I do, then I can go optimise. But in truth the same can be said about keeping linspace abstract just because it saves memory. (obviously it is not faster!) I think the argument for abstract types is very strong, but (in my personal view) not at the expense of expected behaviour. Christoph
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Sweetness. Thank you. AbstractArray it is for me!
On Wednesday, 21 October 2015 16:29:28 UTC-7, Tim Holy wrote:
>
> On Wednesday, October 21, 2015 03:32:04 PM Gabriel Gellner wrote:
> > * I can't use LinSpace in matrix mult A * b::LinSpace, is this simply a
> > Bug/Missing Feature?
>
> Yes
>
> > * LinSpace objects seem much slower when used in things like element
> > multiplication A .* b::LinSpace is much much Slower than A .* b::Array,
> is
> > this to be expected (the cost of the extra abstraction lack of required
> > denseness) or simply a side effect of lack of optimization?
>
> It's because we can't yet eliminate bounds-checks with @inbounds on
> anything
> except Arrays.
>
> > * In general if I make my code accept AbstractArray{Float64} in general
> > should I expect a performance penalty when calling the function with
> > Array{Float64} parameters?
>
> None whatsoever. The only place that's a bad idea is in declarations of
> type-
> fields, see
> http://docs.julialang.org/en/latest/manual/performance-tips/#avoid-fields-with-abstract-type.
>
>
>
> --Tim
>
>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Isn't the situation actually similar to functions like `eye` which returns a dense array (which doesn't feel intuitive for what an eye or diagonal matrix actually is) whereas we have to call the special version seye to get a sparse array. For most users linspace is like `eye` by default suggesting the dense version (ie from reasoning about other languages), whereas the other names I have heard would be like seye, ie linrange. What we currently lack is top level functions that let us choose which version we want like what we have for the sparse vs dense arrays. The current situation is actually exposing less of the type diversity you mention. It feels like following the dense vs sparse array stuff we would want a similar symmetry for LinSpace vs Array creation. On Wednesday, 21 October 2015 14:20:25 UTC-7, Stefan Karpinski wrote: > > The whole notion of always using a single dense array type is simply a > non-starter. Do we just scrap the whole sparse array thing? There goes half > of the stuff you might want to do in linear algebra or machine learning. > Special matrix types like Diagonal or UpperTriangular, etc.? Toss those out > too. Various types of ranges are just a different form of compactly > representing arrays with special structure. How come no one is complaining > that 1:n is not a dense vector? > > On Wed, Oct 21, 2015 at 4:23 PM, Jonathan Malmaud > wrote: > >> You're making good points for sure - logspace and linspace are >> inconsistent wrt return types. >> >> But I just having trouble seeing how it impacts you as a user of the >> language; it's essentially an implementation detail that allows for some >> optimizations when performing arithmetic on array-like values known to have >> a certain exploitable structure (eg, uniform element spacing). >> >> Your own code will virtually never make explicit references to Vector or >> LinSpace types (except perhaps in specifying the types of fields in new >> types you define). If tomorrow logspace was changed to return a special >> type or linspace to return a plain array, your code would remain identical >> if function arguments were typed as AbstractVector instead of Vector. >> >> So I can see how it could bother someone's aesthetic knowing that under >> the hood these inconsistencies exist (it bothers me a bit). And I can see >> how if you're counting on certain optimizations happening, like scalar >> multiplication on spaces being O(1), it would be frustrating that it will >> depend on if the space was generated by linspace or logspace. But is that >> really leading to 'madness'? >> >> There are other examples of these light-weight wrapper types being used, >> especially in the linear algebra routines. The matrix factorization >> functions, like qrfact, return special types, for one thing. >> >> On Wed, Oct 21, 2015 at 4:07 PM, Gabriel Gellner > > wrote: >> >>> That doesn't feel like a reason that they can't be iterators, rather >>> that they might be slow ;) a la python. My point is not about speed but the >>> consistency of the language. Are there many cases in Julia where there is a >>> special type like this because it is convenient/elegant to implement? This >>> feels like a recipe for madness, my guess is that this would be crazy rare. >>> >>> People wondered why people might mind that we get a LinSpace object vs >>> an Array. For me it is this strange feeling that I am getting a special >>> case that doesn't feel well motivated other than there is a nice way to >>> implement it (and that people, again, assumed that it would largely be used >>> for looping). If not all things can be made consistently iterators when >>> they are vector-like then why not have a special function that returns this >>> special type (like your aforementioned linrange)? The fact that I lose my >>> iterator when I use certain functions but not others is a way that this >>> polymorphism that everyone is describing doesn't feel as nice to me, since >>> it will not compose in cases where it likely should, outside of >>> implementation details. >>> >>> >>> On Wednesday, 21 October 2015 12:55:35 UTC-7, DNF wrote: The reason why not all arrays can be iterators is that in general arrays can not be 'compressed' like that. A linear range can be compressed to: a start value, an increment, and a length, making it incredibly lightweight. Doing this for sin() is not that easy. Doing it for rand() is simply Impossible. On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner wrote: > > I have no issue with the LinSpace object, I simply do not see why it > is the special case for this kind of object (I imagine the choice was > made > since it was seen to be used mainly for looping vs being used for array > creation like similar functions logspace, zeros, ones, etc). If the > iterator version is so good I don't see why all Vectors are not re
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wednesday, October 21, 2015 03:32:04 PM Gabriel Gellner wrote:
> * I can't use LinSpace in matrix mult A * b::LinSpace, is this simply a
> Bug/Missing Feature?
Yes
> * LinSpace objects seem much slower when used in things like element
> multiplication A .* b::LinSpace is much much Slower than A .* b::Array, is
> this to be expected (the cost of the extra abstraction lack of required
> denseness) or simply a side effect of lack of optimization?
It's because we can't yet eliminate bounds-checks with @inbounds on anything
except Arrays.
> * In general if I make my code accept AbstractArray{Float64} in general
> should I expect a performance penalty when calling the function with
> Array{Float64} parameters?
None whatsoever. The only place that's a bad idea is in declarations of type-
fields, see
http://docs.julialang.org/en/latest/manual/performance-tips/#avoid-fields-with-abstract-type.
--Tim
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Okay so I am starting to see the light. I see that if LinSpace becomes
fully replacable for an Array it is fine.
Final Questions:
* I can't use LinSpace in matrix mult A * b::LinSpace, is this simply a
Bug/Missing Feature? Or intentional? In general if basic builtin functions
that operate on Array{Float64} types don't accept LinSpace objects is this
something to be fixed? Or do we make special assumptions about the
meaningfulness of such operations?
* LinSpace objects seem much slower when used in things like element
multiplication A .* b::LinSpace is much much Slower than A .* b::Array, is
this to be expected (the cost of the extra abstraction lack of required
denseness) or simply a side effect of lack of optimization?
* In general if I make my code accept AbstractArray{Float64} in general
should I expect a performance penalty when calling the function with
Array{Float64} parameters?
thanks
On Wednesday, 21 October 2015 14:25:17 UTC-7, Stefan Karpinski wrote:
>
> On Wed, Oct 21, 2015 at 5:13 PM, Gabriel Gellner > wrote:
>
>> Maybe all this is just transitional that soon LinSpace objects will
>> always work like Arrays in all code I might use as an end user. Currently
>> as a new user I have not had this experience. I have noticed that LinSpaces
>> where returned, and had to learn what they were and at times run `collect`
>> to make them into what I wanted. I have not felt this abstraction bleed yet
>> in other areas of Julia.
>
>
> I think this is exactly what's happening. The LinSpace type is hitting
> code that should be abstracted in a lot of places but hasn't yet been.
> You'd get the exact same issues if you'd passed a FloatRange object into
> those functions. So the LinSpace type is really doing us all a favor by
> forcing more Julia libraries to the right level of abstraction.
>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wed, Oct 21, 2015 at 5:13 PM, Gabriel Gellner wrote: > Maybe all this is just transitional that soon LinSpace objects will always > work like Arrays in all code I might use as an end user. Currently as a new > user I have not had this experience. I have noticed that LinSpaces where > returned, and had to learn what they were and at times run `collect` to > make them into what I wanted. I have not felt this abstraction bleed yet in > other areas of Julia. I think this is exactly what's happening. The LinSpace type is hitting code that should be abstracted in a lot of places but hasn't yet been. You'd get the exact same issues if you'd passed a FloatRange object into those functions. So the LinSpace type is really doing us all a favor by forcing more Julia libraries to the right level of abstraction.
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
The whole notion of always using a single dense array type is simply a non-starter. Do we just scrap the whole sparse array thing? There goes half of the stuff you might want to do in linear algebra or machine learning. Special matrix types like Diagonal or UpperTriangular, etc.? Toss those out too. Various types of ranges are just a different form of compactly representing arrays with special structure. How come no one is complaining that 1:n is not a dense vector? On Wed, Oct 21, 2015 at 4:23 PM, Jonathan Malmaud wrote: > You're making good points for sure - logspace and linspace are > inconsistent wrt return types. > > But I just having trouble seeing how it impacts you as a user of the > language; it's essentially an implementation detail that allows for some > optimizations when performing arithmetic on array-like values known to have > a certain exploitable structure (eg, uniform element spacing). > > Your own code will virtually never make explicit references to Vector or > LinSpace types (except perhaps in specifying the types of fields in new > types you define). If tomorrow logspace was changed to return a special > type or linspace to return a plain array, your code would remain identical > if function arguments were typed as AbstractVector instead of Vector. > > So I can see how it could bother someone's aesthetic knowing that under > the hood these inconsistencies exist (it bothers me a bit). And I can see > how if you're counting on certain optimizations happening, like scalar > multiplication on spaces being O(1), it would be frustrating that it will > depend on if the space was generated by linspace or logspace. But is that > really leading to 'madness'? > > There are other examples of these light-weight wrapper types being used, > especially in the linear algebra routines. The matrix factorization > functions, like qrfact, return special types, for one thing. > > On Wed, Oct 21, 2015 at 4:07 PM, Gabriel Gellner > wrote: > >> That doesn't feel like a reason that they can't be iterators, rather that >> they might be slow ;) a la python. My point is not about speed but the >> consistency of the language. Are there many cases in Julia where there is a >> special type like this because it is convenient/elegant to implement? This >> feels like a recipe for madness, my guess is that this would be crazy rare. >> >> People wondered why people might mind that we get a LinSpace object vs an >> Array. For me it is this strange feeling that I am getting a special case >> that doesn't feel well motivated other than there is a nice way to >> implement it (and that people, again, assumed that it would largely be used >> for looping). If not all things can be made consistently iterators when >> they are vector-like then why not have a special function that returns this >> special type (like your aforementioned linrange)? The fact that I lose my >> iterator when I use certain functions but not others is a way that this >> polymorphism that everyone is describing doesn't feel as nice to me, since >> it will not compose in cases where it likely should, outside of >> implementation details. >> >> >> On Wednesday, 21 October 2015 12:55:35 UTC-7, DNF wrote: >>> >>> The reason why not all arrays can be iterators is that in general arrays >>> can not be 'compressed' like that. A linear range can be compressed to: a >>> start value, an increment, and a length, making it incredibly lightweight. >>> Doing this for sin() is not that easy. Doing it for rand() is simply >>> Impossible. >>> >>> On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner >>> wrote: I have no issue with the LinSpace object, I simply do not see why it is the special case for this kind of object (I imagine the choice was made since it was seen to be used mainly for looping vs being used for array creation like similar functions logspace, zeros, ones, etc). If the iterator version is so good I don't see why all Vectors are not returned as this type for all the reasons you mention. In the current state where only linspace returns this kind of special polymorphic type it simply breaks any feeling of consistency in the language. I so something like x = zeros(10) and I get an array great. the next line I do y = linspace(0, 5, 10) I get a new fangled iterator object. They work the same but how do I get an iterator version of zeros? etc. It is a pedantic point but so is special casing this super common function to mean something that is not consistent with all other languages that use it. Which would be fine if when I did something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. This abstraction is not percolated thru other functions, further giving the feeling of a needless special case in the language writ large. They simply get converted to concrete types for many common functions, when if this is done will vary from func
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
First of all thank everyone for dealing with me! I find this discussion really interesting ... though I hope it is not rehashing to much basic stuff, I am trying my best to get up to speed with the Julian way. I am new to Julia, but have programmed a lot in other languages ... hence my random feelings. Re: turning an arbitrary array into an iterator -> I guess you could simply bind the element generator a la pythons __iter__ so for sin(linspace(0, 0.5, 10)) I would simply have to store the sin function then have this call sin(getitem(_stored > You're making good points for sure - logspace and linspace are > inconsistent wrt return types. > > But I just having trouble seeing how it impacts you as a user of the > language; it's essentially an implementation detail that allows for some > optimizations when performing arithmetic on array-like values known to have > a certain exploitable structure (eg, uniform element spacing). > > Your own code will virtually never make explicit references to Vector or > LinSpace types (except perhaps in specifying the types of fields in new > types you define). If tomorrow logspace was changed to return a special > type or linspace to return a plain array, your code would remain identical > if function arguments were typed as AbstractVector instead of Vector. > > So I can see how it could bother someone's aesthetic knowing that under > the hood these inconsistencies exist (it bothers me a bit). And I can see > how if you're counting on certain optimizations happening, like scalar > multiplication on spaces being O(1), it would be frustrating that it will > depend on if the space was generated by linspace or logspace. But is that > really leading to 'madness'? > > There are other examples of these light-weight wrapper types being used, > especially in the linear algebra routines. The matrix factorization > functions, like qrfact, return special types, for one thing. > > On Wed, Oct 21, 2015 at 4:07 PM, Gabriel Gellner > wrote: > >> That doesn't feel like a reason that they can't be iterators, rather that >> they might be slow ;) a la python. My point is not about speed but the >> consistency of the language. Are there many cases in Julia where there is a >> special type like this because it is convenient/elegant to implement? This >> feels like a recipe for madness, my guess is that this would be crazy rare. >> >> People wondered why people might mind that we get a LinSpace object vs an >> Array. For me it is this strange feeling that I am getting a special case >> that doesn't feel well motivated other than there is a nice way to >> implement it (and that people, again, assumed that it would largely be used >> for looping). If not all things can be made consistently iterators when >> they are vector-like then why not have a special function that returns this >> special type (like your aforementioned linrange)? The fact that I lose my >> iterator when I use certain functions but not others is a way that this >> polymorphism that everyone is describing doesn't feel as nice to me, since >> it will not compose in cases where it likely should, outside of >> implementation details. >> >> >> On Wednesday, 21 October 2015 12:55:35 UTC-7, DNF wrote: >>> >>> The reason why not all arrays can be iterators is that in general arrays >>> can not be 'compressed' like that. A linear range can be compressed to: a >>> start value, an increment, and a length, making it incredibly lightweight. >>> Doing this for sin() is not that easy. Doing it for rand() is simply >>> Impossible. >>> >>> On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner >>> wrote: I have no issue with the LinSpace object, I simply do not see why it is the special case for this kind of object (I imagine the choice was made since it was seen to be used mainly for looping vs being used for array creation like similar functions logspace, zeros, ones, etc). If the iterator version is so good I don't see why all Vectors are not returned as this type for all the reasons you mention. In the current state where only linspace returns this kind of special polymorphic type it simply breaks any feeling of consistency in the language. I so something like x = zeros(10) and I get an array great. the next line I do y = linspace(0, 5, 10) I get a new fangled iterator object. They work the same but how do I get an iterator version of zeros? etc. It is a pedantic point but so is special casing this super common function to mean something that is not consistent with all other languages that use it. Which would be fine if when I did something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. This abstraction is not percolated thru other functions, further giving the feeling of a needless special case in the language writ large. They simply >>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
You're making good points for sure - logspace and linspace are inconsistent wrt return types. But I just having trouble seeing how it impacts you as a user of the language; it's essentially an implementation detail that allows for some optimizations when performing arithmetic on array-like values known to have a certain exploitable structure (eg, uniform element spacing). Your own code will virtually never make explicit references to Vector or LinSpace types (except perhaps in specifying the types of fields in new types you define). If tomorrow logspace was changed to return a special type or linspace to return a plain array, your code would remain identical if function arguments were typed as AbstractVector instead of Vector. So I can see how it could bother someone's aesthetic knowing that under the hood these inconsistencies exist (it bothers me a bit). And I can see how if you're counting on certain optimizations happening, like scalar multiplication on spaces being O(1), it would be frustrating that it will depend on if the space was generated by linspace or logspace. But is that really leading to 'madness'? There are other examples of these light-weight wrapper types being used, especially in the linear algebra routines. The matrix factorization functions, like qrfact, return special types, for one thing. On Wed, Oct 21, 2015 at 4:07 PM, Gabriel Gellner wrote: > That doesn't feel like a reason that they can't be iterators, rather that > they might be slow ;) a la python. My point is not about speed but the > consistency of the language. Are there many cases in Julia where there is a > special type like this because it is convenient/elegant to implement? This > feels like a recipe for madness, my guess is that this would be crazy rare. > > People wondered why people might mind that we get a LinSpace object vs an > Array. For me it is this strange feeling that I am getting a special case > that doesn't feel well motivated other than there is a nice way to > implement it (and that people, again, assumed that it would largely be used > for looping). If not all things can be made consistently iterators when > they are vector-like then why not have a special function that returns this > special type (like your aforementioned linrange)? The fact that I lose my > iterator when I use certain functions but not others is a way that this > polymorphism that everyone is describing doesn't feel as nice to me, since > it will not compose in cases where it likely should, outside of > implementation details. > > > On Wednesday, 21 October 2015 12:55:35 UTC-7, DNF wrote: >> >> The reason why not all arrays can be iterators is that in general arrays >> can not be 'compressed' like that. A linear range can be compressed to: a >> start value, an increment, and a length, making it incredibly lightweight. >> Doing this for sin() is not that easy. Doing it for rand() is simply >> Impossible. >> >> On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner wrote: >>> >>> I have no issue with the LinSpace object, I simply do not see why it is >>> the special case for this kind of object (I imagine the choice was made >>> since it was seen to be used mainly for looping vs being used for array >>> creation like similar functions logspace, zeros, ones, etc). If the >>> iterator version is so good I don't see why all Vectors are not returned as >>> this type for all the reasons you mention. In the current state where only >>> linspace returns this kind of special polymorphic type it simply breaks any >>> feeling of consistency in the language. I so something like x = zeros(10) >>> and I get an array great. the next line I do y = linspace(0, 5, 10) I get a >>> new fangled iterator object. They work the same but how do I get an >>> iterator version of zeros? etc. It is a pedantic point but so is special >>> casing this super common function to mean something that is not consistent >>> with all other languages that use it. Which would be fine if when I did >>> something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. >>> This abstraction is not percolated thru other functions, further giving the >>> feeling of a needless special case in the language writ large. They simply >>> get converted to concrete types for many common functions, when if this is >>> done will vary from function to function, with little semantic reasoning. >>> My feeling is that if what people say is true than why is the default >>> numeric array not have the iterator semantics. As it is linspace is made >>> special for no reason other than it is assumed it will be used for looping. >>> >>> People want to argue that it is more elegant and takes advantage of what >>> makes Julia powerful, which I get, but then why not go all in on this? >>> Mathematica does this. Everything is basically a List which nice behavior >>> for vectors matrices etc. I have no issue with this kind of elegance, but >>> it is rough when the abstraction is
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
I actually tend to think that's a pretty strong reason. On Wednesday, October 21, 2015 at 10:07:23 PM UTC+2, Gabriel Gellner wrote: > > That doesn't feel like a reason that they can't be iterators, rather that > they might be slow ;) a la python. >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wed, Oct 21, 2015, at 04:07 PM, Gabriel Gellner wrote:
> That doesn't feel like a reason that they can't be iterators, rather
> that they might be slow ;) a la python. My point is not about speed
> but the consistency of the language. Are there many cases in Julia
> where there is a special type like this because it is
> convenient/elegant to implement? This feels like a recipe for madness,
> my guess is that this would be crazy rare.
>
> People wondered why people might mind that we get a LinSpace object vs
> an Array. For me it is this strange feeling that I am getting a
> special case that doesn't feel well motivated other than there is a
> nice way to implement it (and that people, again, assumed that it
> would largely be used for looping). If not all things can be made
> consistently iterators when they are vector-like then why not have a
> special function that returns this special type (like your
> aforementioned linrange)? The fact that I lose my iterator when I use
> certain functions but not others is a way that this polymorphism that
> everyone is describing doesn't feel as nice to me, since it will not
> compose in cases where it likely should, outside of implementation
> details.
Can you clarify with an example of when you lose the iterator? IMO that
would be an example of breaking the AbstractArray contract and worthy
of fixing.
I see your point that there are other functions that return array-like
objects that could also be implemented without fully constructing the
array, e.g. `zeros(20)` could return a `Zeros{20}` object that acts like
a length-20 array full of zeros. As far as I know there's no compelling
design reason that's a bad idea, it's just that nobody has done it.
The nice thing is that as long as people don't over-specify their type
annotations that change can be done under-the-hood and it shouldn't
break code. From a pedagogical standpoint I see how these sorts of
things can add confusion and cause new folks to question their
understanding, and more consistency reduces that conceptual friction.
-s
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wed, Oct 21, 2015 at 4:07 PM, Gabriel Gellner wrote: > That doesn't feel like a reason that they can't be iterators, rather that > they might be slow ;) a la python. My point is not about speed but the > consistency of the language. How do you propose making arbitrary arrays into iterators?
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
That doesn't feel like a reason that they can't be iterators, rather that they might be slow ;) a la python. My point is not about speed but the consistency of the language. Are there many cases in Julia where there is a special type like this because it is convenient/elegant to implement? This feels like a recipe for madness, my guess is that this would be crazy rare. People wondered why people might mind that we get a LinSpace object vs an Array. For me it is this strange feeling that I am getting a special case that doesn't feel well motivated other than there is a nice way to implement it (and that people, again, assumed that it would largely be used for looping). If not all things can be made consistently iterators when they are vector-like then why not have a special function that returns this special type (like your aforementioned linrange)? The fact that I lose my iterator when I use certain functions but not others is a way that this polymorphism that everyone is describing doesn't feel as nice to me, since it will not compose in cases where it likely should, outside of implementation details. On Wednesday, 21 October 2015 12:55:35 UTC-7, DNF wrote: > > The reason why not all arrays can be iterators is that in general arrays > can not be 'compressed' like that. A linear range can be compressed to: a > start value, an increment, and a length, making it incredibly lightweight. > Doing this for sin() is not that easy. Doing it for rand() is simply > Impossible. > > On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner wrote: >> >> I have no issue with the LinSpace object, I simply do not see why it is >> the special case for this kind of object (I imagine the choice was made >> since it was seen to be used mainly for looping vs being used for array >> creation like similar functions logspace, zeros, ones, etc). If the >> iterator version is so good I don't see why all Vectors are not returned as >> this type for all the reasons you mention. In the current state where only >> linspace returns this kind of special polymorphic type it simply breaks any >> feeling of consistency in the language. I so something like x = zeros(10) >> and I get an array great. the next line I do y = linspace(0, 5, 10) I get a >> new fangled iterator object. They work the same but how do I get an >> iterator version of zeros? etc. It is a pedantic point but so is special >> casing this super common function to mean something that is not consistent >> with all other languages that use it. Which would be fine if when I did >> something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. >> This abstraction is not percolated thru other functions, further giving the >> feeling of a needless special case in the language writ large. They simply >> get converted to concrete types for many common functions, when if this is >> done will vary from function to function, with little semantic reasoning. >> My feeling is that if what people say is true than why is the default >> numeric array not have the iterator semantics. As it is linspace is made >> special for no reason other than it is assumed it will be used for looping. >> >> People want to argue that it is more elegant and takes advantage of what >> makes Julia powerful, which I get, but then why not go all in on this? >> Mathematica does this. Everything is basically a List which nice behavior >> for vectors matrices etc. I have no issue with this kind of elegance, but >> it is rough when the abstraction is inconsistent (the type is not truly >> perserved in most functions). As people have mentioned clearly logspace >> should be a special object ... but when does this stop? I dislike that this >> feels so arbitrary ... and as a result it is jarring when it turns up. The >> fact that it is polymorphic is only one kind of consistency. >> >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
The reason why not all arrays can be iterators is that in general arrays can not be 'compressed' like that. A linear range can be compressed to: a start value, an increment, and a length, making it incredibly lightweight. Doing this for sin() is not that easy. Doing it for rand() is simply Impossible. On Wednesday, October 21, 2015 at 9:38:07 PM UTC+2, Gabriel Gellner wrote: > > I have no issue with the LinSpace object, I simply do not see why it is > the special case for this kind of object (I imagine the choice was made > since it was seen to be used mainly for looping vs being used for array > creation like similar functions logspace, zeros, ones, etc). If the > iterator version is so good I don't see why all Vectors are not returned as > this type for all the reasons you mention. In the current state where only > linspace returns this kind of special polymorphic type it simply breaks any > feeling of consistency in the language. I so something like x = zeros(10) > and I get an array great. the next line I do y = linspace(0, 5, 10) I get a > new fangled iterator object. They work the same but how do I get an > iterator version of zeros? etc. It is a pedantic point but so is special > casing this super common function to mean something that is not consistent > with all other languages that use it. Which would be fine if when I did > something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. > This abstraction is not percolated thru other functions, further giving the > feeling of a needless special case in the language writ large. They simply > get converted to concrete types for many common functions, when if this is > done will vary from function to function, with little semantic reasoning. > My feeling is that if what people say is true than why is the default > numeric array not have the iterator semantics. As it is linspace is made > special for no reason other than it is assumed it will be used for looping. > > People want to argue that it is more elegant and takes advantage of what > makes Julia powerful, which I get, but then why not go all in on this? > Mathematica does this. Everything is basically a List which nice behavior > for vectors matrices etc. I have no issue with this kind of elegance, but > it is rough when the abstraction is inconsistent (the type is not truly > perserved in most functions). As people have mentioned clearly logspace > should be a special object ... but when does this stop? I dislike that this > feels so arbitrary ... and as a result it is jarring when it turns up. The > fact that it is polymorphic is only one kind of consistency. >
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
I have no issue with the LinSpace object, I simply do not see why it is the special case for this kind of object (I imagine the choice was made since it was seen to be used mainly for looping vs being used for array creation like similar functions logspace, zeros, ones, etc). If the iterator version is so good I don't see why all Vectors are not returned as this type for all the reasons you mention. In the current state where only linspace returns this kind of special polymorphic type it simply breaks any feeling of consistency in the language. I so something like x = zeros(10) and I get an array great. the next line I do y = linspace(0, 5, 10) I get a new fangled iterator object. They work the same but how do I get an iterator version of zeros? etc. It is a pedantic point but so is special casing this super common function to mean something that is not consistent with all other languages that use it. Which would be fine if when I did something like sin(linspace(0, 5, 100)) I got back an iterator but I don't. This abstraction is not percolated thru other functions, further giving the feeling of a needless special case in the language writ large. They simply get converted to concrete types for many common functions, when if this is done will vary from function to function, with little semantic reasoning. My feeling is that if what people say is true than why is the default numeric array not have the iterator semantics. As it is linspace is made special for no reason other than it is assumed it will be used for looping. People want to argue that it is more elegant and takes advantage of what makes Julia powerful, which I get, but then why not go all in on this? Mathematica does this. Everything is basically a List which nice behavior for vectors matrices etc. I have no issue with this kind of elegance, but it is rough when the abstraction is inconsistent (the type is not truly perserved in most functions). As people have mentioned clearly logspace should be a special object ... but when does this stop? I dislike that this feels so arbitrary ... and as a result it is jarring when it turns up. The fact that it is polymorphic is only one kind of consistency. On Wednesday, 21 October 2015 12:09:29 UTC-7, Jonathan Malmaud wrote: > > It’s still hard for me to understanding what the value of returning an > array is by default. > > By getting a structured LinSpace object, it enables things like having the > REPL print it in a special way, to optimize arithmetic operations on it (so > that adding a scalar to a LinSpace is O(1) instead of O(N) in the length of > the space), to inspect its properties by accessing its fields, etc. And on > top of that, it can be used transparently in virtually all expressions > where a concrete array can be used. It’s not like Python where iterators > are generally going to be much slower and clunkier to work with than a > reified Numpy array. > > The only downside really is if your arguments are explicitly and > unnecessarily typed to expect an Array, which is not a great habit to get > into no matter what linspace returns. > > Not trying to be argumentative or dismissive here- just trying to > understand. I would think that if one of your motivations for getting into > Julia is the rich type system compared to Matlab, you’d be happy that Julia > isn’t forced to discard semantic information from operations like linspace > as a result of only only raw numeric vectors being first-class constructs > in the language (as in Matlab and Numpy). > > On Oct 21, 2015, at 2:38 PM, Gabriel Gellner > wrote: > > Wow! Thanks everyone for all the advice!!! Super helpful. I now see that > it is super easy to deal with the LinSpace objects. > > That being said I guess I get scared when the docs tell me to use concrete > types for performance ;) Most of the code I write for myself is working > with Float64 arrays a la Matlab. I am comfortable with duck typing and what > not but one of the things that drew me to Julia (vs lets say Mathematica) > is how in general the type system feels easy to reason about. All the > layers of indirection can scare me as it makes it harder for me to > understand the kinds of performance problems I might face. I like sending > concrete array types for my own code as it often catches bugs (when I maybe > used an int literal and this accidentally created a int array when i really > wanted a float, potentially leading to horrible performance cascades as all > my generic code starts doing the wrong thing...). > > I guess really what this comes down to is the point made by DNF that > changing linspace to an iterator when that name means an array in Matlab > and python is not the path of least surprise. It feels to me like this is a > strange inconsistency as we could just as easily return other common > functions, like `rand` to be an AbstractArray like type, but this would > suck as it does in this case. I guess i
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
It’s still hard for me to understanding what the value of returning an array is
by default.
By getting a structured LinSpace object, it enables things like having the REPL
print it in a special way, to optimize arithmetic operations on it (so that
adding a scalar to a LinSpace is O(1) instead of O(N) in the length of the
space), to inspect its properties by accessing its fields, etc. And on top of
that, it can be used transparently in virtually all expressions where a
concrete array can be used. It’s not like Python where iterators are generally
going to be much slower and clunkier to work with than a reified Numpy array.
The only downside really is if your arguments are explicitly and unnecessarily
typed to expect an Array, which is not a great habit to get into no matter what
linspace returns.
Not trying to be argumentative or dismissive here- just trying to understand. I
would think that if one of your motivations for getting into Julia is the rich
type system compared to Matlab, you’d be happy that Julia isn’t forced to
discard semantic information from operations like linspace as a result of only
only raw numeric vectors being first-class constructs in the language (as in
Matlab and Numpy).
> On Oct 21, 2015, at 2:38 PM, Gabriel Gellner wrote:
>
> Wow! Thanks everyone for all the advice!!! Super helpful. I now see that it
> is super easy to deal with the LinSpace objects.
>
> That being said I guess I get scared when the docs tell me to use concrete
> types for performance ;) Most of the code I write for myself is working with
> Float64 arrays a la Matlab. I am comfortable with duck typing and what not
> but one of the things that drew me to Julia (vs lets say Mathematica) is how
> in general the type system feels easy to reason about. All the layers of
> indirection can scare me as it makes it harder for me to understand the kinds
> of performance problems I might face. I like sending concrete array types for
> my own code as it often catches bugs (when I maybe used an int literal and
> this accidentally created a int array when i really wanted a float,
> potentially leading to horrible performance cascades as all my generic code
> starts doing the wrong thing...).
>
> I guess really what this comes down to is the point made by DNF that changing
> linspace to an iterator when that name means an array in Matlab and python is
> not the path of least surprise. It feels to me like this is a strange
> inconsistency as we could just as easily return other common functions, like
> `rand` to be an AbstractArray like type, but this would suck as it does in
> this case. I guess it all comes down to wishing that, like DNF suggested, we
> had a proper name for linspace and the iterator version was a new name. I
> guess my way forward will likely be to define my own linarray() function,
> sadly this will make my code a bit confusing to my matlab friends ;)
>
> Again thanks though. I really have learned a crazy bunch about the generic
> type system.
>
> On Wednesday, 21 October 2015 08:50:32 UTC-7, Spencer Russell wrote:
> On Wed, Oct 21, 2015, at 11:38 AM, Gabriel Gellner wrote:
>> No that is a good point. Often you can use an iterator where an explicit
>> array would also work. The issue I guess is that this puts the burden on the
>> developer to always write generic code that when you would want to accept an
>> Array you also need to accept a iterator like LinSpace.
>>
>> Maybe this is easier to do than I currently understand?
>
> In general you can just make your function accept an AbstractVector (or more
> generally AbstractArray) and things will just work. If there are places where
> Ranges, LinSpaces, etc. aren't behaving as proper arrays then I think those
> are generally good candidates for Issues (or even better, PRs).
>
> As a related aside:
> For efficiency you'll still want to use a concrete Vector field if you're
> defining your own types, but luckily there seems to be a convert method
> defined so you can do:
>
> type MyArr{T}
> x::Vector{T}
> end
>
> a = MyArr{Int64}(1:4)
>
> And it will auto-convert the range to an array for storage in your type.
>
> -s
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Wow! Thanks everyone for all the advice!!! Super helpful. I now see that it
is super easy to deal with the LinSpace objects.
That being said I guess I get scared when the docs tell me to use concrete
types for performance ;) Most of the code I write for myself is working
with Float64 arrays a la Matlab. I am comfortable with duck typing and what
not but one of the things that drew me to Julia (vs lets say Mathematica)
is how in general the type system feels easy to reason about. All the
layers of indirection can scare me as it makes it harder for me to
understand the kinds of performance problems I might face. I like sending
concrete array types for my own code as it often catches bugs (when I maybe
used an int literal and this accidentally created a int array when i really
wanted a float, potentially leading to horrible performance cascades as all
my generic code starts doing the wrong thing...).
I guess really what this comes down to is the point made by DNF that
changing linspace to an iterator when that name means an array in Matlab
and python is not the path of least surprise. It feels to me like this is a
strange inconsistency as we could just as easily return other common
functions, like `rand` to be an AbstractArray like type, but this would
suck as it does in this case. I guess it all comes down to wishing that,
like DNF suggested, we had a proper name for linspace and the iterator
version was a new name. I guess my way forward will likely be to define my
own linarray() function, sadly this will make my code a bit confusing to my
matlab friends ;)
Again thanks though. I really have learned a crazy bunch about the generic
type system.
On Wednesday, 21 October 2015 08:50:32 UTC-7, Spencer Russell wrote:
>
> On Wed, Oct 21, 2015, at 11:38 AM, Gabriel Gellner wrote:
>
> No that is a good point. Often you can use an iterator where an explicit
> array would also work. The issue I guess is that this puts the burden on
> the developer to always write generic code that when you would want to
> accept an Array you also need to accept a iterator like LinSpace.
>
> Maybe this is easier to do than I currently understand?
>
>
> In general you can just make your function accept an AbstractVector (or
> more generally AbstractArray) and things will just work. If there are
> places where Ranges, LinSpaces, etc. aren't behaving as proper arrays
> then I think those are generally good candidates for Issues (or even
> better, PRs).
>
> As a related aside:
> For efficiency you'll still want to use a concrete Vector field if you're
> defining your own types, but luckily there seems to be a convert method
> defined so you can do:
>
> type MyArr{T}
> x::Vector{T}
> end
>
> a = MyArr{Int64}(1:4)
>
> And it will auto-convert the range to an array for storage in your type.
>
> -s
>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wed, Oct 21, 2015, at 12:08 PM, Steven G. Johnson wrote:
> On Wednesday, October 21, 2015 at 11:50:32 AM UTC-4, Spencer Russell wrote:
>>
>> For efficiency you'll still want to use a concrete Vector field if
>> you're defining your own types, but luckily there seems to be a
>> convert method defined so you can do:
>>
>> type MyArr{T} x::Vector{T} end
>
> Instead, you can do
>
> type MyArr{V:
> and it will allow any vector type (including ranges) but will still be
> fast (i.e. every instance of MyArr will have a concrete type for V).
Ah yes, even better! Thanks.
-s
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wednesday, October 21, 2015 at 11:50:32 AM UTC-4, Spencer Russell wrote:
>
> For efficiency you'll still want to use a concrete Vector field if you're
> defining your own types, but luckily there seems to be a convert method
> defined so you can do:
>
> type MyArr{T}
> x::Vector{T}
> end
>
Instead, you can do
type MyArr{V:
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
All true.
One tip: if you want to find out what abstract type is a parent to all the
concrete types you're interested in, you can use the "typejoin" function.
julia> typejoin(Array, LinSpace)
AbstractArray{T,N}
julia> typejoin(Vector,LinSpace)
AbstractArray{T,1}
On Wednesday, October 21, 2015 at 12:05:06 PM UTC-4, Steven G. Johnson
wrote:
>
>
>
> On Wednesday, October 21, 2015 at 11:57:08 AM UTC-4, Jonathan Malmaud
> wrote:
>>
>> Just to add to Spencer's answer: Is there a particular reason to have
>> your function arguments have type annotations at all in the function
>> definition? You could just write
>>
>> function f(x)
>> y= x[3:5] # or whatever
>> z = length(x)
>> end
>>
>> and now someone could call f with any kind of object that supports
>> indexing and "length" and it will work. This is "duck-typing", if you're
>> familiar with that term, and is the dominant paradigm in Julia precisely
>> since it makes generic programming easier.
>>
>>>
> I agree that duck typing is often a good practice, but there are three
> good reasons to declare argument types:
>
> * Correctness: the code might work but give unexpected results if you pass
> the wrong types. e.g. fib(n) = n < 2 ? one(n) : fib(n-1)+fib(n-2) is a
> function that computes the Fibonacci numbers only for integers — it gives
> an answer for floating-point n, but the answer is probably not what you
> want.
>
> * Clarity: sometimes it is a useful hint to the caller if you indicate the
> expected type.
>
> * Dispatch: you want to do different things for different types, so you
> use the argument types as a filter to indicate which methods should work
> when.
>
> However, in all cases the trick is to declare the widest applicable
> argument type. e.g. use fib(n::Integer), not fib(n::Int), so that any
> integer type will be accepted, rather than the concrete type Int.
>
> In the case of functions accepting vectors, you should almost always
> declare the type as AbstractVector or AbstractArray, not Vector or Array.
> That will let you handle any array-like type. In particular, ranges are a
> subtype of AbstractVector.
>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wednesday, October 21, 2015 at 11:57:08 AM UTC-4, Jonathan Malmaud wrote: > > Just to add to Spencer's answer: Is there a particular reason to have your > function arguments have type annotations at all in the function definition? > You could just write > > function f(x) > y= x[3:5] # or whatever > z = length(x) > end > > and now someone could call f with any kind of object that supports > indexing and "length" and it will work. This is "duck-typing", if you're > familiar with that term, and is the dominant paradigm in Julia precisely > since it makes generic programming easier. > >> I agree that duck typing is often a good practice, but there are three good reasons to declare argument types: * Correctness: the code might work but give unexpected results if you pass the wrong types. e.g. fib(n) = n < 2 ? one(n) : fib(n-1)+fib(n-2) is a function that computes the Fibonacci numbers only for integers — it gives an answer for floating-point n, but the answer is probably not what you want. * Clarity: sometimes it is a useful hint to the caller if you indicate the expected type. * Dispatch: you want to do different things for different types, so you use the argument types as a filter to indicate which methods should work when. However, in all cases the trick is to declare the widest applicable argument type. e.g. use fib(n::Integer), not fib(n::Int), so that any integer type will be accepted, rather than the concrete type Int. In the case of functions accepting vectors, you should almost always declare the type as AbstractVector or AbstractArray, not Vector or Array. That will let you handle any array-like type. In particular, ranges are a subtype of AbstractVector.
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Just to add to Spencer's answer: Is there a particular reason to have your
function arguments have type annotations at all in the function definition?
You could just write
function f(x)
y= x[3:5] # or whatever
z = length(x)
end
and now someone could call f with any kind of object that supports indexing
and "length" and it will work. This is "duck-typing", if you're familiar
with that term, and is the dominant paradigm in Julia precisely since it
makes generic programming easier.
On Wednesday, October 21, 2015 at 11:50:32 AM UTC-4, Spencer Russell wrote:
>
> On Wed, Oct 21, 2015, at 11:38 AM, Gabriel Gellner wrote:
>
> No that is a good point. Often you can use an iterator where an explicit
> array would also work. The issue I guess is that this puts the burden on
> the developer to always write generic code that when you would want to
> accept an Array you also need to accept a iterator like LinSpace.
>
> Maybe this is easier to do than I currently understand?
>
>
> In general you can just make your function accept an AbstractVector (or
> more generally AbstractArray) and things will just work. If there are
> places where Ranges, LinSpaces, etc. aren't behaving as proper arrays
> then I think those are generally good candidates for Issues (or even
> better, PRs).
>
> As a related aside:
> For efficiency you'll still want to use a concrete Vector field if you're
> defining your own types, but luckily there seems to be a convert method
> defined so you can do:
>
> type MyArr{T}
> x::Vector{T}
> end
>
> a = MyArr{Int64}(1:4)
>
> And it will auto-convert the range to an array for storage in your type.
>
> -s
>
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
Actually it is that easy... you just have to get in the habit of doing it:
julia> function mysum(arr::Array)
s = zero(eltype(arr))
for a in arr
s += a
end
s
end
mysum (generic function with 1 method)
julia> mysum(1:10)
ERROR: MethodError: `mysum` has no method matching mysum(::UnitRange{Int64})
julia> mysum(collect(1:10))
55
julia> function mysum_generic(arr::AbstractArray)
s = zero(eltype(arr))
for a in arr
s += a
end
s
end
mysum_generic (generic function with 1 method)
julia> mysum_generic(1:10)
55
julia> mysum_generic(collect(1:10))
55
For something like this, you never need allocate space for the array.
On Wed, Oct 21, 2015 at 11:38 AM, Gabriel Gellner
wrote:
> No that is a good point. Often you can use an iterator where an explicit
> array would also work. The issue I guess is that this puts the burden on
> the developer to always write generic code that when you would want to
> accept an Array you also need to accept a iterator like LinSpace.
>
> Maybe this is easier to do than I currently understand? If not for regular
> scientists like myself I find that this would force me to make my functions
> far more complex than I generally would do in practice. I would often just
> write a signature like func(x::Vector) when I want my code to accept of
> Vector, but if I pass a LinSpace object to this I get a type error. I like
> this type of coding as it fits my model of what I want to do. The
> contortions I would need to do for something as basic as having a nice way
> to get a linear array of floating point numbers to a simple function that
> wants to accept a Vector seems like a wart to me. Am I missing something
> simple?
>
> For the builtins clearly this is not the case which is nice, but it only
> masks the issue of how this would be used by regular users in my opinion.
>
>
> On Wednesday, 21 October 2015 05:59:21 UTC-7, Jonathan Malmaud wrote:
>>
>> Gabriel, I rewrote your code to not ever explicitly convert ranges to
>> arrays and it still works fine. Maybe I'm not quite understanding the issue?
>>
>> function Jakes_Flat( fd, Ts, Ns, t0 = 0, E0 = 1, phi_N = 0 )
>> # Inputs:
>> #
>> # Outputs:
>> N0 = 8; # As suggested by Jakes
>> N = 4*N0+2; # An accurate approximation
>> wd = 2*pi*fd;# Maximum Doppler frequency
>> t = t0 + (0:Ns-1)*Ts;
>> tf = t[end] + Ts;
>> coswt = [ sqrt(2)*cos(wd*t'); 2*cos(wd*cos(2*pi/N*(1:N0))*t') ]
>> temp = zeros(1,N0+1)
>> temp[1,2:end] = pi/(N0+1)*(1:N0)'
>> temp[1,1] = phi_N
>> h = E0/sqrt(2*N0+1)*exp(im*temp ) * coswt
>> return h, tf;
>> end
>>
>> On Wednesday, October 21, 2015 at 3:11:44 AM UTC-4, Gabriel Gellner wrote:
>>>
>>> I find the way that you need to use `linspace` and `range` objects a bit
>>> jarring for when you want to write vectorized code, or when I want to pass
>>> an array to a function that requires an Array. I get how nice the iterators
>>> are when writing loops and that you can use `collect(iter)` to get a array
>>> (and that it is possible to write polymorphic code that takes LinSpace
>>> types and uses them like Arrays … but this hurts my small brain). But I
>>> find I that I often want to write code that uses an actual array and having
>>> to use `collect` all the time seems like a serious wart for an otherwise
>>> stunning language for science. (
>>> https://github.com/JuliaLang/julia/issues/9637 gives the evolution I
>>> think of making these iterators)
>>>
>>>
>>>
>>> For example recently the following code was posted/refined on this
>>> mailing list:
>>>
>>>
>>>
>>> function Jakes_Flat( fd, Ts, Ns, t0 = 0, E0 = 1, phi_N = 0 )
>>>
>>> # Inputs:
>>>
>>> #
>>>
>>> # Outputs:
>>>
>>> N0 = 8; # As suggested by Jakes
>>>
>>> N = 4*N0+2; # An accurate approximation
>>>
>>> wd = 2*pi*fd;# Maximum Doppler frequency
>>>
>>> t = t0 + [0:Ns-1;]*Ts;
>>>
>>> tf = t[end] + Ts;
>>>
>>> coswt = [ sqrt(2)*cos(wd*t'); 2*cos(wd*cos(2*pi/N*[1:N0;])*t') ]
>>>
>>> temp = zeros(1,N0+1)
>>>
>>> temp[1,2:end] = pi/(N0+1)*[1:N0;]'
>>>
>>> temp[1,1] = phi_N
>>>
>>> h = E0/sqrt(2*N0+1)*exp(im*temp ) * coswt
>>>
>>> return h, tf;
>>>
>>> end
>>>
>>>
>>>
>>> From
>>>
>>>
>>>
>>> Notice all the horrible [;] notations to make these arrays … and
>>> it seems like the devs want to get rid of this notation as well (which they
>>> should it is way too subtle in my opinion). So imagine the above code with
>>> `collect` statements. Is this the way people work? I find the `collect`
>>> statements in mathematical expressions to really break me out of the
>>> abstraction (that I am just writing math).
>>>
>>>
>>>
>>> I get that this could be written as an explicit loop, and this would
>>> likely make it faster as well (man I love looping
Re: [julia-users] Re: A question of Style: Iterators into regular Arrays
On Wed, Oct 21, 2015, at 11:38 AM, Gabriel Gellner wrote:
> No that is a good point. Often you can use an iterator where an
> explicit array would also work. The issue I guess is that this puts
> the burden on the developer to always write generic code that when you
> would want to accept an Array you also need to accept a iterator like
> LinSpace.
>
> Maybe this is easier to do than I currently understand?
In general you can just make your function accept an AbstractVector (or
more generally AbstractArray) and things will just work. If there are
places where Ranges, LinSpaces, etc. aren't behaving as proper arrays
then I think those are generally good candidates for Issues (or even
better, PRs).
As a related aside: For efficiency you'll still want to use a concrete
Vector field if you're defining your own types, but luckily there seems
to be a convert method defined so you can do:
type MyArr{T} x::Vector{T} end
a = MyArr{Int64}(1:4)
And it will auto-convert the range to an array for storage in your type.
-s
