RE: Stage contains task of large size

2016-03-03 Thread Mohammed Guller
/dp/1484209656/> From: Silvio Fiorito [mailto:silvio.fior...@granturing.com] Sent: Wednesday, March 2, 2016 8:43 PM To: Bijuna; user Subject: RE: Stage contains task of large size One source of this could be more than you intended (or realized) getting serialized as part of your operations

Stage contains task of large size

2016-03-02 Thread Bijuna
Spark users, We are running spark application in standalone mode. We see warn messages in the logs which says Stage 46 contains a task of very large size (983 KB) . The maximum recommended task size is 100 KB. What is the recommended approach to fix this warning. Please let me know. Thank