Re: [julia-users] sum of 1-element array of composite type returns reference
OK Thanks Milan.
1. I think for now I'll just use subtype of Number.
I have an uneasy feeling this is not entirely the way to go because I get
stuff that might not be valid for my type. An example is convert() see
below. But for my purpose now, it's not really a problem.
2. My comment about both forms of zero() being required for summing
0-element arrays wasn't to suggest it was an issue (I can also now see that
both forms of zero() end up calling convert() as you pointed out. Thanks).
My point was minor and twofold:
a. Summing a 0-element array creates (at least??) 3 objects (zero(T),
zero(Type{T}), and the their addition) which I originally thought
was rather convoluted and inefficient. I'm sure it's to keep code general.
Regardless its really not an issue
b. For non-Number types it does require defining both forms of zero().
Again, not really an issue.
3. Note that if I subtype from Number, then I still need to do one of the
following:
a. define both forms of zero() - I think the better option
b. define convert(MyType, Int) - this is more general than I want. Although
zero() eventually calls convert(MyType, 0) which makes sense, it
doesn't always make sense to define convert for arbitrary Int.
(In my actual case, MyType is a large collection of vectors, 2/3D-arrays
that are counters. I use pmap and want to reuse returned objects after
reducing by sum, which is why I want sum to return a new object).
4. As far as submitting a Pull Request, I think this is way out of my
league. I'm a newcomer to programming, Julia, pull requests, ... (Actually
I didn't really know what a pull request was until I looked it up after
your suggestion). For what it's worth:
a. My main concern is that sum() should always return a new object (so I
don't have to externally check size of array)
b. I feel uneasy about subtyping from Number (I can't really say why,
except that I might be getting more than I want, as in convert example
above)
c. Your solution below seems good.
Thanks for help.
It's much appreciated.
- Greg
If you look at the definition of sum()
that's used in your example, you'll see that it calls _mapreduce(),
which in turn relies on the r_promote() function when only one element
is passed. The code can be seen using edit(Base.r_promote).
For numbers, it indeed does
r_promote(::AddFun, x::Number) = x + zero(x)
But for other types, no copy is done:
r_promote(::AddFun, x) = x
This must be because zero() is not necessarily defined for arbitrary
types. But for the sum to be defined, it looks to me that zero() must be
defined too, and it looks like the code assumes that too. So I'm not
sure why the definition for numbers is not used for everything.
Re: [julia-users] sum of 1-element array of composite type returns reference
Le mercredi 07 janvier 2015 à 01:22 -0800, Greg Plowman a écrit :
OK Thanks Milan.
1. I think for now I'll just use subtype of Number.
I have an uneasy feeling this is not entirely the way to go because I
get stuff that might not be valid for my type. An example is convert()
see below. But for my purpose now, it's not really a problem.
2. My comment about both forms of zero() being required for summing
0-element arrays wasn't to suggest it was an issue (I can also now
see that both forms of zero() end up calling convert() as you pointed
out. Thanks). My point was minor and twofold:
a. Summing a 0-element array creates (at least??) 3 objects
(zero(T), zero(Type{T}), and the their addition) which I
originally thought was rather convoluted and inefficient. I'm
sure it's to keep code general. Regardless its really not an
issue
For standard numbers like integers and floats at least these should go
away during compilation. I'm not sure what it would give for your type,
this would deserve more investigation.
b. For non-Number types it does require defining both forms of
zero(). Again, not really an issue.
There aren't both forms of zero, just two functions to create the same
zero.
3. Note that if I subtype from Number, then I still need to do one of
the following:
a. define both forms of zero() - I think the better option
Yes.
b. define convert(MyType, Int) - this is more general than I
want. Although zero() eventually calls convert(MyType, 0)
which makes sense, it doesn't always make sense to define
convert for arbitrary Int.
(In my actual case, MyType is a large collection of vectors,
2/3D-arrays that are counters. I use pmap and want to reuse
returned objects after reducing by sum, which is why I want
sum to return a new object).
So indeed avoid it and use a. instead.
But you're probably right that your type is not really a number, so
making it inherit from Number is just a temporary stopgap.
4. As far as submitting a Pull Request, I think this is way out of my
league. I'm a newcomer to programming, Julia, pull requests, ...
(Actually I didn't really know what a pull request was until I looked
it up after your suggestion). For what it's worth:
a. My main concern is that sum() should always return a new
object (so I don't have to externally check size of array)
b. I feel uneasy about subtyping from Number (I can't really
say why, except that I might be getting more than I want, as
in convert example above)
c. Your solution below seems good.
OK, that's fine. Could you just open an issue on GitHub, so that the
best solution can be discussed? Thanks!
Regards
Thanks for help.
It's much appreciated.
- Greg
If you look at the definition of sum()
that's used in your example, you'll see that it calls _mapreduce(),
which in turn relies on the r_promote() function when only one
element
is passed. The code can be seen using edit(Base.r_promote).
For numbers, it indeed does
r_promote(::AddFun, x::Number) = x + zero(x)
But for other types, no copy is done:
r_promote(::AddFun, x) = x
This must be because zero() is not necessarily defined for arbitrary
types. But for the sum to be defined, it looks to me that zero() must
be
defined too, and it looks like the code assumes that too. So I'm not
sure why the definition for numbers is not used for everything.
Re: [julia-users] sum of 1-element array of composite type returns reference
Thanks for help and patience Milan. Issue opened: https://github.com/JuliaLang/julia/issues/9669
Re: [julia-users] sum of 1-element array of composite type returns reference
Le lundi 05 janvier 2015 à 18:53 -0800, Greg Plowman a écrit :
The only reason I can think of is that a copy may be
costly for certain
types, and it's not needed in most cases since the
summation will create
a new value in the general case. But as you noted this
is not true when
the array only contains one element. So it looks like
the most efficient
fix would be to copy only when n == 1 in
_mapreduce().
I must admit I don't really understand the code, however it doesn't
look like evaluation would be affected for n=2.
The extra cost would only be for 1-element arrays:
Apparently, for 1-element arrays, zero(::MyType) needs to be defined
For 0-element arrays, both zero(::MyType) and zero(::Type{MyType})
need to be defined
(strangely, for 0-element arrays, mr_empty() calls r_promote(::AddFun,
zero(T)) which effectively calls zero(T) + zero(zero(T)), so both
forms of zero() need to be defined
Yes, but that's not an issue as the definitions are equivalent:
zero(x::Number) = oftype(x,0)
zero{T:Number}(::Type{T}) = convert(T,0)
help? oftype
Base.oftype(x, y)
Convert y to the type of x (convert(typeof(x), y)).
In any case, at the moment I guess I have 2 workarounds:
I could define MyType as a subtype of Number and provide zero()
functions.
However, I'm not sure what the side effects of subtyping are, and
whether this is advisable?
I don't think it would be a problem, it may well make a lot of sense if
your type is similar to a number (which is apparently the case since you
can sum it).
type MyType :Number
x::Int
end
Base.zero(::Type{MyType}) = MyType(0) # required for sum(0-element
array)
Base.zero(::MyType) = MyType(0) # required for sum(0-element
array) and sum(1-element array)
+(a::MyType, b::MyType) = MyType(a.x + b.x)
Alternatively, I could define my own sum() functions, but then if I
want the general functionality of all variants of sum(), this seems
non-trivial.
Indeed. Another solution is to make a pull request with a possible fix,
it could be included quite soon in a 0.3.x minor release so that you can
use it.
Regards
Re: [julia-users] sum of 1-element array of composite type returns reference
Le dimanche 04 janvier 2015 à 19:30 -0800, Greg Plowman a écrit : I'm not sure how general this behaviour is with respect to other types, but I have observed the following with a simple composite type: When summing a 1-element array of a simple composite type, the return is a reference to the single element, rather than a copy of the single element. This seems at odds with the return value of summing an array of any other size (even 0-element array if zero is defined) Is this the desired behaviour? type MyType x::Int end +(a::MyType, b::MyType) = MyType(a.x + b.x) A = [ MyType(i) for i = 1:5 ] sumA = sum(A) A[1].x = 77 sumA # MyType(15), seems reasonable B = [ MyType(i) for i = 1 ] sumB = sum(B) B[1].x = 88 sumB # MyType(88), is this reasonable? sum(B) === B[1] # true I guess for a copy to be returned (rather than a reference) would need to define how to copy. Perhaps require on of the following? * copy constructor * copy() function * zero() and return A[1] + zero() Would this be preferable to returning a reference? If I want to return a copy of single element, do I need to define my own sum() function instead? No, looks like a bug to me. If you look at the definition of sum() that's used in your example, you'll see that it calls _mapreduce(), which in turn relies on the r_promote() function when only one element is passed. The code can be seen using edit(Base.r_promote). For numbers, it indeed does r_promote(::AddFun, x::Number) = x + zero(x) But for other types, no copy is done: r_promote(::AddFun, x) = x This must be because zero() is not necessarily defined for arbitrary types. But for the sum to be defined, it looks to me that zero() must be defined too, and it looks like the code assumes that too. So I'm not sure why the definition for numbers is not used for everything. The only reason I can think of is that a copy may be costly for certain types, and it's not needed in most cases since the summation will create a new value in the general case. But as you noted this is not true when the array only contains one element. So it looks like the most efficient fix would be to copy only when n == 1 in _mapreduce(). This would also fix the same issue with more general mapreduce() operations, when the mapping function does not create a copy (like the identity function). Regards
Re: [julia-users] sum of 1-element array of composite type returns reference
The only reason I can think of is that a copy may be costly for certain
types, and it's not needed in most cases since the summation will create
a new value in the general case. But as you noted this is not true when
the array only contains one element. So it looks like the most efficient
fix would be to copy only when n == 1 in _mapreduce().
I must admit I don't really understand the code, however it doesn't look
like evaluation would be affected for n=2.
The extra cost would only be for 1-element arrays:
Apparently, for 1-element arrays, zero(::MyType) needs to be defined
For 0-element arrays, both zero(::MyType) and zero(::Type{MyType}) need to
be defined
(strangely, for 0-element arrays, mr_empty() calls r_promote(::AddFun,
zero(T)) which effectively calls zero(T) + zero(zero(T)), so both forms of
zero() need to be defined
In any case, at the moment I guess I have 2 workarounds:
I could define MyType as a subtype of Number and provide zero() functions.
However, I'm not sure what the side effects of subtyping are, and whether
this is advisable?
type MyType :Number
x::Int
end
Base.zero(::Type{MyType}) = MyType(0) # required for sum(0-element array)
Base.zero(::MyType) = MyType(0) # required for sum(0-element array)
and sum(1-element array)
+(a::MyType, b::MyType) = MyType(a.x + b.x)
Alternatively, I could define my own sum() functions, but then if I want
the general functionality of all variants of sum(), this seems non-trivial.
[julia-users] sum of 1-element array of composite type returns reference
I'm not sure how general this behaviour is with respect to other types, but I have observed the following with a simple composite type: When summing a 1-element array of a simple composite type, the return is a reference to the single element, rather than a copy of the single element. This seems at odds with the return value of summing an array of any other size (even 0-element array if zero is defined) Is this the desired behaviour? type MyType x::Int end +(a::MyType, b::MyType) = MyType(a.x + b.x) A = [ MyType(i) for i = 1:5 ] sumA = sum(A) A[1].x = 77 sumA # MyType(15), seems reasonable B = [ MyType(i) for i = 1 ] sumB = sum(B) B[1].x = 88 sumB # MyType(88), is this reasonable? sum(B) === B[1] # true I guess for a copy to be returned (rather than a reference) would need to define how to copy. Perhaps require on of the following? - copy constructor - copy() function - zero() and return A[1] + zero() Would this be preferable to returning a reference? If I want to return a copy of single element, do I need to define my own sum() function instead?
