: Data partitioning and node tracking in Spark-GraphX
Thanks a lot for the reply. Indeed it is useful but to be more precise i
have 3D data and want to index it using octree. Thus i aim to build a two
level indexing mechanism i.e. First at global level i want to partition and
send the data
the right node and data partition.
How can i achieve this?
Any help in this regard will be highly appreciated.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Data-partitioning-and-node-tracking-in-Spark-GraphX-tp22527.html
Sent from the Apache Spark User List
How do you intend to fetch the required data - from within Spark or using
an app / code / module outside Spark
-Original Message-
From: mas [mailto:mas.ha...@gmail.com]
Sent: Thursday, April 16, 2015 4:08 PM
To: user@spark.apache.org
Subject: Data partitioning and node tracking
:* MUHAMMAD AAMIR [mailto:mas.ha...@gmail.com]
*Sent:* Thursday, April 16, 2015 4:20 PM
*To:* Evo Eftimov
*Cc:* user@spark.apache.org
*Subject:* Re: Data partitioning and node tracking in Spark-GraphX
I want to use Spark functions/APIs to do this task. My basic purpose is to
index the data
partitioning and node tracking in Spark-GraphX
I have a big data file, i aim to create index on the data. I want to
partition the data based on user defined function in Spark-GraphX (Scala).
Further i want to keep track the node on which a particular data partition
is send and being processed
@spark.apache.org
Subject: Data partitioning and node tracking in Spark-GraphX
I have a big data file, i aim to create index on the data. I want to
partition the data based on user defined function in Spark-GraphX (Scala).
Further i want to keep track the node on which a particular data partition
, April 16, 2015 4:32 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: Data partitioning and node tracking in Spark-GraphX
Thanks a lot for the reply. Indeed it is useful but to be more precise i have
3D data and want to index it using octree. Thus i aim to build a two level
indexing
