Idan Zalzberg created SPARK-5318: ------------------------------------ Summary: Add ability to control partition count in SparkSql Key: SPARK-5318 URL: https://issues.apache.org/jira/browse/SPARK-5318 Project: Spark Issue Type: New Feature Components: SQL Reporter: Idan Zalzberg
When using SparkSql, e.g. sqlContext.sql("..."), spark might need to read hadoop files. However, unlike the hadoopFile API, there is no documented way to set the minimal partition count when reading. There is an undocumented way, though, using "mapred.map.tasks" in hiveConf I suggest we make a documented way to do it, in the exact same way (possibly with a better name) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org