Re: Implementing RFC 272
Buddha Buck wrote: > When I heard about transpose() (as well as reshape(), etc), I was > concerned about the time it would take to execute these complex > operations. To wit, naively, I believed that a lot of data shuffling > would be necessary. If I understand your message correctly this is exactly what PDL, NumPY, etc do right now and is also described in one of my messages I sent yesterday (Re: RFC 272 (v1) Arrays: transpose()) about implementing aliasing ;) Christian
Re: RFC 272 (v1) Arrays: transpose()
Jeremy Howard wrote: > (I'm not assuming the no-flattening thing, since that's another source of > angst altogether!) What is the no-flattening thing? Christian
Re: RFC 272 (v1) Arrays: transpose()
Karl Glazebrook wrote: > the arguments to reshape should be sizes not last elements (i.e. N's > not N-1's). Yup, it's simple: size (N) vs index range (0..N-1) > How does this sound? Logical and consistent ;) Christian
Re: RFC 272 (v1) Arrays: transpose()
Jeremy Howard wrote: > So where is mv(), you ask? If you use the 'reorder' syntax, but don't > specify all of the dimensions in the list ref, then the remaining dimensions > are added in order: That sounds good. I'd say why not also allow the mv syntax? It is syntactically different from the others, may be the least often used variant but then there are N+1 ways to do it ;) But no strong feelings either way... Christian
Re: RFC 272 (v1) Arrays: transpose()
Jeremy Howard wrote:
>
> Karl Glazebrook wrote:
> > you should look at the PDL mv() and xchg() methods
> > and factor this into your thinking!
> >
> Actually, the RFC is based on PDL's xchg()! I forgot to document using
> negative numbers to count from the last dimension--I'll add that into the
> next version. Are there any other differences with xchg() that you think
> would be useful?
>
> I haven't used mv() before, but now I look at it I can see it's pretty
> interesting. Is this used much? If we add it to the RFC, do you think we'd
> want a separate function, or add another arg to transpose:
>
> transpose([$a,$b], 0, @arr); # xchg
> transpose([$a,$b], 1, @arr); # mv
How about (if perl6 allows passing arrays implicitly by reference
without arglist flattening)
transpose @arr, $a, $b; # xchg
transpose @arr, {$a => $b}; # mv
transpose @arr, [0,3,4,1,2]; # PDL reorder
Aliasing for bounded typed arrays is simple:
each array has a block of data (void *) and vectors of shape (int[]),
strides (int[]) and offset (int). To get element arr[i;j;k] Perl
accesses into the typecast block
((type *) data)[offset+i*stride[0]+j*stride[1]+k*stride[2]]
When creating an alias only the shape, strides and offset are
manipulated which tells the new alias how to interpret the same block of
data differently. This kind of implementation is used in PDL, NumPy and
GSL AFAIK.
In PDL we have found it useful to have a method turning the alias into a
being of its own ($pdl->sever, oops undocumented).
Christian
Re: RFC 231 (v1) Data: Multi-dimensional arrays/hashes and slices
Ilya Zakharevich wrote: > > On Fri, Sep 22, 2000 at 05:24:55PM -0400, Karl Glazebrook wrote: > > It's now boiling down to a matter of opinion and we'll have to agree to > > differ. Of course I use array arithmetic all the time as a heavy PDL > > user. > > ...Do you say you are confused by using vectors (=scalars) instead of > arrays? I'm not having a problem with that personally but *many* users of PDL have complained about being confused by this. They assume ndim == array == perl array. Christian
Re: RFC 207 (v1) Array: Efficient Array Loops
Reading through the examples left me wondering about some
technicalities:
> @t[|i;|j] = @a[|j;|i]; # transpose 2-d @a
Written like this it would require that @a is exact 2-dim, i.e. it would
not just swap the first two dims of any n-dim array? I suppose if I'd
want that I'd write
@t[|i;|j;] = @a[|j;|i;]; # trailing ';' implies there might be
trailing dims
> #compute pairwise sum, pairwise product, pairwise difference...
> @sum = @a[|i;|j;|k;|l] + @b[|i;|j;|k;|l];
> @prod= @a[|i;|j;|k;|l] * @b[|i;|j;|k;|l];
> @diff= @a[|i;|j;|k;|l] - @b[|i;|j;|k;|l];
Hm, not sure if I am missing the point of these examples. Is that any
different from the elementwise '+','*','-' apart from being possibly
limited to 4D arrays?
> #print lots of stuff to the screen.
> sub foo { print join(',',@_),"\n"; return 0; }
> $zero[|i;|j;|k;|l;|m;|n;|o;|p] = foo(|i,|j,|k,|l,|m,|n,|o,|p);
Should that be '$zero' or '@zero'?
>
> # Sneaky way to generate dot-product:
> my $dotproduct;
> { my @temp[|i] = $dotproduct += $a[|i] * $b[|i]; }
Hm, how can this work with lazy evaluation? How is Perl supposed to know
that @temp should be transiently created to increment $dotproduct as a
side effect? Also, doesn't the above syntax seem to be conflicting with
perl context rules since it essentially contains a statement
@arr = $scalar
Christian
Re: RFC 82 (v3) Arrays: Apply operators element-wise in a list context
Nathan Wiger wrote: > what people would want to use the ops for, and it's also more usable to > us non-PDLers. I'd like to suggest that it is not a very good idea to start dividing the world into PDLers and non-PDLers. There are a multitude of reasons but I am not keen to go into details. Christian
Re: RFC 82 (v3) Arrays: Apply operators element-wise in a list context
Doug Hunt wrote: > You might also look at APL, a language which has been doing really funky > N-dimensional > array manipulation for longer than anyone... I'd say PDL has probably been influenced quite strongly by APL/J in its PDL threading notion. And with more and more "funny characters" making it into RFCs perl6 might get similar to J ;) Christian
Re: RFC 203 (v1) Arrays: Notation for declaring and creating arrays
Perl6 RFC Librarian wrote: > The bounds of an array or list can be specified at run time, of course: > > my int @t1 :bounds(@dimList) = getFromSomeplace(); Hm, I think some clarification would be good. I'd imagine three cases: (1) getFromSomeplace returns an untyped and unbounded LOL. In that case I'd write my int @t1 : bounds = getFromSomeplace(); and the bounds should be automatically figured out from the returned LOL. (2) getFromSomeplace returns already a typed bounded array: my @t1 = getFromSomeplace(); or (3) getFromSomeplace returns a normal perl list or LOL and I want it to be reshaped (according to @dimList with clipping?) into my new bound array: my int @t1 :bounds(@dimList) = getFromSomeplace(); > > Where the type and bounds of an array can be derived at run time, it is > not necessary to specify them explicitly: > > my int @t1 :bounds(@dimList) = getFromSomeplace(); > my int @t2 :bounds(@dimList) = getFromSomeplaceElse(); > my @prod = @t1 * @t2; # @prod magically has type (int) and :bounds (@dimlist) What happens when pairing typed/untyped bounded/unbounded arrays in such operations. The stricter one wins? I'd imagine the usual type conversions happen automatically. Should there be an RFC detailing the data types and conversions that one would want, e.g. a complex type, etc? > and its bounds can be locked in as well if required: > > my @some_LOL = ([1,2], > [3,4]); > my int @array :bounds(@#some_LOL) = @some_LOL; > Again, couldn't we just imply the bounding, i.e. my int @array :bounds = @rvalue; always meaning my int @array :bounds(@#rvalue) = @rvalue; Christian
Re: RFC 206 (v1) Array: @#arr for getting the dimensions of an array
Perl6 RFC Librarian wrote:
> This RFC proposes using @#array, analogous to $#array, to get the list of
> upper bounds for a multidimensional array @array. The length of @#array
> would indicate the dimensionality of @array.
That's fine. This RFC does not seem to touch on the question what
$#array should be for a multidim array (e.g., $#array == $#{flatten @a}
or rather undef). Or is that dealt with in another RFC?
Christian
Re: RFC 82 (v3) Arrays: Apply operators element-wise in a list context
Nathan Wiger wrote: > > > This RFC proposes that operators in a list context should be applied > > element-wise to the elements of their arguments: > > > > @d = @b * @c; # Returns (2,8,18) > > > > If the lists are not of equal length, an error is raised. > > I've been watching this RFC for a while. I would hesitate to change the > default behavior of * and other operators in so radical a sense, > especially since it would create unexpected error conditions. I think > these operations should remain scalar. I disagree. You end up with a situation where some @a * @b; are in scalar context, some not. And what happens when you say @a = (1,2,2); @b = tie Matrix (1,2,3); @a*@b; Who wins? >From the user point of view a 1D multi-dim array will look just the same as a normal perl list in *all* respects in a would-be perl6. I bet money that users will be highly confused if some arrays enforce scalar context in such operations, others don't. It is *much* clearer to have a clear break with perl5 in this respect (even if code needs to be rewritten). A p52p6 could replace all occurences of @a*@b with scalar(@a)*scalar(@b). Christian
Re: a syntax derived from constant-time hash-based n-dim matrices in perl 5
"David L. Nicol" wrote:
>
> Nathan Wiger wrote:
>
> > Well, this is not bad, only it's not without its problems. Say you
> > wanted to get your indices implicitly:
> >
> > @a[getindices()];
> > @a[$r->get_x, $r->get_y];
>
> @a["@{\(getindices())}"];
> @a[join $",$r->get_x, $r->get_y];
Constructs like this are the reason that we (PDLers) are unhappy with
our current slicing syntax (slice, cf.RFC 115,117). IMHO continuing
along these lines is *not* the way forward.
Christian
Re: Proposed RFC for matrix indexing and slicing
Bart Lateur wrote: > Personally, I would like to have Larry's fiat *before* trying to iron > out the incompatibilities. It could be that you throw away something > that Larry would approve of, and keep something he doesn't like. I'd vote to go for what we think is the best compromise. If Larry interacts as part of the discussion leading there that's fine. We should have some trust in our own judgement. Christian
Re: Some PDL issues (was Re: Test)
Dan Sugalski wrote: > It looks like we need to be able to override operations on arrays, have > multi-dimensional arrays, and do some rather odd slicing operations that > give values still linked to the original matrices. > > Has anyone asked for complex number support yet? It's hidden in one or two lines of RFC 116 I think ;) Christian
Re: RFC 148 (v1) Add reshape() for multi-dimensional array reshaping
"David L. Nicol" wrote: > > Looks like if we give the data type control over what > the meaning of square brackets after it is, the rest > becomes example code. I think this s covered in the > horribly misnamed http://dev.perl.org/rfc/115.pod which > covers overloading bracketing. Agreed. We should rename this one ASAP. Christian
Re: Some PDL issues (was Re: Test)
Karl Glazebrook wrote:
> "->" really sucks as something to routinely type in to a interactive shell
> all the time. I hate it.
I can not agree more. It just seems terribly unnecessary.
Just allow us to directly overload some of the braces/parentheses for
objects.
I would be happy with
$pdl(0:$n,[0],:2)
which would look like a subref call without the & in front. That
was my original DEFAULT method idea which would be called with
a
$(args)
syntax. Just in case people are hung up about allowing us to overload
[] or {}. I really don't have a problem with
$pdl[$a,:2,2:2]
having already another implied meaning for array types in perl5. After
all,
if we take natural languages as our model for perl, natural languages
are full of "exceptions to the rule". Mastering those is at least half
the fun of learning a language.
>
> $pdl[0:$n][0][:]
>
> would be fine as a syntax
Hm, I disagree. Too long. Comma or semicolon as separator (single
character that doesn't require to press shift!!!).
>
> BUT we really want the subscription operation to be fast and efficient -
> which means doing it with one subroutine call not 3. This is where the
> $pdl[0:$n,0,:] form wins as you can imaging the subroutine just seeing
> "0:$n,0,:" in one arg.
>
> Also another common idiom in PDL is:
>
> $pdl[0:$n,(0),:]
>
> where the () means - lose this dimension of size unity. I.e. make the result
> MxN rather than Mx1xN. How would this be handled?
We could replace those with square brackets (returning an array ref)
$pdl[0:$n,[0],:2]
Christian
Re: New variable type: matrix
Karl Glazebrook wrote: > The key from my point of view is to have enough syntactical > hooks in the core so that using it is not like wading through > treacle. Hence the PDL RFCs - especially on [] overloading and ranges. > > The Numerical Python people seem to have accomplished this, and we can't > let them win!! :-) Hear, hear! It would be nice to relax some of the idealistic ideas of what must not be touched or changed and let us do our dirty tricks. And did I say that any extra keystrokes should be avoided at all cost ;-) Christian
Re: Some PDL issues (was Re: Test)
Dan Sugalski wrote: > to make foo and bar 5x5x5 matricies that you casn multiply to get baz then, > well, say it. If that means you need to define a way to provide overridden > operators in the Matrix package, then go for it and say that. > > Let the -internals folks worry about the Weird Magic needed to implement > what you want. (This is a variation of my standard "Don't get hung up on > the implementation, that's my job" speech) Although there is one point that is crucial in a useful implementation -- packed and typed for the reasons I mentioned before. Since we are on language-data speed and efficiency on large chunks of data is essential and can not be completely left to druids/wizards and their weird magic ;) Christian
Some PDL issues (was Re: Test)
> I'm here because:
> * I think the PDL folks have valid needs
> * I think a lot of people are having trouble making suggestions for
>language changes that mesh well with other constructs in the
>language.
> * I'm not able to tell you want you need, but I will try to help
>you come up with language changes that are tasteful.
>
> Nat
These are some of the current issues that we have identified:
(1) The current
$pdl->slice("0:$n,(0)");
syntax sucks. We want something that is much shorter (no ->slice
required).
Why? *Because a module like PDL is often used for interactive
explorartory
data analysis at the (perldl) command line*. Every extra letter is
annoying.
And, honestly, the extra typing is an issue ($pdl[0:$n][0][:] is not an
acceptable possibility).
Our current suggestion: allow us to overload (), [], {} so that we can
say (RFC 115)
$pdl[0:$n,(0)]
or
$pdl(0:$n,(0))
Additionally, allow us to use expressions like
0:$n:3
without raising syntax errors. Our current suggestion is in RFC 117
(which is closely related to RFC 81).
(2) We see the need for a multidimensional, packed and typed array.
Currently this is represented by PDL objects. It needs to be typed and
packed
to allow efficient interfacing to existing libraries (LinAlg, image
processing, etc) and fast dealing with data on the order of 100s of MB
in size.
The need to represent matrices and other higher dim objects of
numerical/
scientific analysis requires a multidim approach (see also RFC 116).
The question is: should such an array type be part of perl6 or should it
be left to a module to implement such types?
A possible approach would be a 1D packed and typed array for perl6 that
a piddle would use as its underlying data array on which it imposes a
multidim structure:
@arr = complex packed 0:1; # some arbitrary syntax suggestion
$pdl = pdlwrap [10,10,100], @arr; # wrap this packed array to make a
[10,10,100] piddle
$arr[100] = 0;
print $pdl[0,0,1]; # print that same element
(3) if we get better support for doing operations between arrays a la
@mult = @a * @b;
@zipped = zip @a, @b;
etc we should strive to make things feel very similar when using arrays
vs
piddles. My feeling is that there will still be the distinction between
piddles and perl arrays since I don't expect all the packed multidim
stuff to make it into the perl6 core.
(4) Get access to the parsers output from perl so that we can do our own
(PDL) optimizations on certain subroutines (the PP equivalent, see RFC
116). I think Damian Conway has mentioned this capability on the wish
list for perl6 already.
So if somebody has good perlish ideas on how to get us these things in
the nicest possible way (that go beyond/against the current RFCs) we'd
love to hear it.
Christian
Re: Test
Karl Glazebrook wrote: > > Just a test to see if this list is actually alive... Seems to be. But no great activity yet. Where to start? Christian
