Re: sparkStreaming how to work with partitions,how tp create partition

2015-08-27 Thread Akhil Das
You can use the driver ui and click on the Jobs - Stages to see the number of partitions being created for that job. If you want to increase the partitions, then you could do a .repartition too. With directStream api i guess the # partitions in spark will be equal to the number of partitions in

sparkStreaming how to work with partitions,how tp create partition

2015-08-22 Thread Gaurav Agarwal
1. how to work with partition in spark streaming from kafka 2. how to create partition in spark streaming from kafka when i send the message from kafka topic having three partitions. Spark will listen the message when i say kafkautils.createStream or createDirectstSream have local[4] Now i