Anyone who can guide me how to reduce the Size from Long to Int since I
dont need Long index.
I am huge data and this index talking 8 bytes, if i can reduce it to 4
bytes will be great help?
On 22 April 2015 at 22:46, Jeetendra Gangele gangele...@gmail.com wrote:
Sure thanks. if you can guide
Sure thanks. if you can guide me how to do this will be great help.
On 17 April 2015 at 22:05, Ted Yu yuzhih...@gmail.com wrote:
I have some assignments on hand at the moment.
Will try to come up with sample code after I clear the assignments.
FYI
On Thu, Apr 16, 2015 at 2:00 PM,
Can you please guide me how can I extend RDD and convert into this way you
are suggesting.
On 16 April 2015 at 23:46, Jeetendra Gangele gangele...@gmail.com wrote:
I type T i already have Object ... I have RDDObject and then I am
calling ZipWithIndex on this RDD and getting RDDObject,Long on
Hi Ted.
This works for me. But since Long takes here 8 bytes. Can I reduce it to 4
bytes. its just a index and I feel 4 bytes was more than enough.is there
any method which takes Integer or similar for Index?
On 13 April 2015 at 01:59, Ted Yu yuzhih...@gmail.com wrote:
bq. will return
The Long in RDD[(T, Long)] is type parameter. You can create RDD with
Integer as the first type parameter.
Cheers
On Thu, Apr 16, 2015 at 11:07 AM, Jeetendra Gangele gangele...@gmail.com
wrote:
Hi Ted.
This works for me. But since Long takes here 8 bytes. Can I reduce it to 4
bytes. its just
I type T i already have Object ... I have RDDObject and then I am calling
ZipWithIndex on this RDD and getting RDDObject,Long on this I am running
MapToPair and converting into RDDLong,Object so that i can use it later
for other operation like lookup and join.
On 16 April 2015 at 23:42, Ted Yu
How about using mapToPair and exchanging the two. Will it be efficient
Below is the code , will it be efficient to convert like this.
JavaPairRDDLong, MatcherReleventData RddForMarch
=matchRdd.zipWithindex.mapToPair(new
PairFunctionTuple2VendorRecord,Long, Long, MatcherReleventData() {
bq. will return something like JavaPairRDDObject, long
The long component of the pair fits your description of index. What other
requirement does ZipWithIndex not provide you ?
Cheers
On Sun, Apr 12, 2015 at 1:16 PM, Jeetendra Gangele gangele...@gmail.com
wrote:
Hi All I have an RDD
Hi All I have an RDD JavaRDDObject and I want to convert it to
JavaPairRDDIndex,Object.. Index should be unique and it should maintain
the order. For first object It should have 1 and then for second 2 like
that.
I tried using ZipWithIndex but it will return something like
JavaPairRDDObject, long
