Re: Sorting a Table in Spark RDD

2014-09-23 Thread Victor Tso-Guillen 
You could pluck out each column in separate rdds, sort them independently,
and zip them :)

On Tue, Sep 23, 2014 at 2:40 PM, Areg Baghdasaryan (BLOOMBERG/ 731 LEX -) <
abaghdasa...@bloomberg.net> wrote:

> Hello,
>
> So I have crated a table in in RDD in spark in thei format:
> col1 col2
> ---
> 1. 10 11
> 2. 12 8
> 3. 9 13
> 4. 2 3
>
> And the RDD is ristributed by the rows (rows 1, 2 on one node and rows 3 4
> on another)
> I want to sort each column of the table so that that output is the
> following:
>
>
> col1 col2
> ---
> 1. 2 3
> 2. 9 8
> 3. 10 11
> 4. 122 13
>
> Is tehre a easy way to do this with spark RDD? The only way that i can
> think of so far is to transpose the table somehow..
>
> Thanks
> Areg
>

Sorting a Table in Spark RDD

2014-09-23 Thread Areg Baghdasaryan (BLOOMBERG/ 731 LEX -) 
Hello,

So I have crated a table in in RDD in spark in thei format:
col1col2
---
1. 10  11
2. 12  8
3.  9   13
4. 2   3

And the RDD is ristributed by the rows (rows 1, 2 on one node and rows 3 4 on 
another)
I want to sort each column of the table so that that output is the following:


col1col2
---
1. 2   3
2.  9   8
3.  10  11
4. 122 13

Is tehre a easy way to do this with spark RDD? The only way that i can think of 
so far is to transpose the table somehow..

Thanks
Areg