On Sun, Mar 23, 2014 at 9:24 PM, Nicholas Chammas
nicholas.cham...@gmail.com javascript:; wrote:
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good
number of partitions
...@gmail.com wrote:
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good
number of partitions is good for performance. If I'm looking to understand
how my pipeline is performing, say
Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good
number of partitions is good for performance. If I'm looking to understand
how my pipeline is performing, say for a parallelized write out to HDFS
at 12:53 AM, Mark Hamstra m...@clearstorydata.com
wrote:
It's much simpler: rdd.partitions.size
On Sun, Mar 23, 2014 at 9:24 PM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good number
of partitions is good for performance. If I'm looking to understand how my
pipeline is performing, say for a parallelized write out
It's much simpler: rdd.partitions.size
On Sun, Mar 23, 2014 at 9:24 PM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good
number
nicholas.cham...@gmail.com wrote:
Hey there fellow Dukes of Data,
How can I tell how many partitions my RDD is split into?
I'm interested in knowing because, from what I gather, having a good
number of partitions is good for performance. If I'm looking to understand
how my pipeline