will spark aggregate and treeaggregate case a shuflle action?

2016-11-10 Thread codlife
Hi Users: I'm doubt about whether spark aggregate and treeaggregate case a shuffle action? if not how spark do the combine option in spark internal.Any tips is appreciated, thank you! -- View this message in context:

Why the json file used by sparkSession.read.json must be a valid json object per line

2016-10-15 Thread codlife
Hi: I'm doubt about the design of spark.read.json, why the json file is not a standard json file, who can tell me the internal reason. Any advice is appreciated. -- View this message in context: