spark git commit: [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py

2015-04-06 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 1cde04f21 -> ab1b8edb8 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py The spark_ec2.py script uses public_dns_name everywhere in the script except for testing ssh availability, which is done using the public ip address

spark git commit: [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py

2015-04-06 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master a0846c4b6 -> 6f0d55d76 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py The spark_ec2.py script uses public_dns_name everywhere in the script except for testing ssh availability, which is done using the public ip address of

spark git commit: [SPARK-6716] Change SparkContext.DRIVER_IDENTIFIER from to driver

2015-04-06 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master e40ea8742 -> a0846c4b6 [SPARK-6716] Change SparkContext.DRIVER_IDENTIFIER from to driver Currently, the driver's executorId is set to ``. This choice of ID was present in older Spark versions, but it has started to cause problems now that

spark git commit: [Minor] [SQL] [SPARK-6729] Minor fix for DriverQuirks get

2015-04-06 Thread adav
Repository: spark Updated Branches: refs/heads/master 30363ede8 -> e40ea8742 [Minor] [SQL] [SPARK-6729] Minor fix for DriverQuirks get The function uses .substring(0, X), which will trigger OutOfBoundsException if string length is less than X. A better way to do this is to use startsWith, wh

spark git commit: [MLlib] [SPARK-6713] Iterators in columnSimilarities for mapPartitionsWithIndex

2015-04-06 Thread meng
Repository: spark Updated Branches: refs/heads/master 9fe412521 -> 30363ede8 [MLlib] [SPARK-6713] Iterators in columnSimilarities for mapPartitionsWithIndex Use Iterators in columnSimilarities to allow mapPartitionsWithIndex to spill to disk. This could happen in a dense and large column - th

spark git commit: SPARK-6205 [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError

2015-04-06 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 728c1f927 -> 1cde04f21 SPARK-6205 [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError Add xml-apis to core test deps to work aroudn UISeleniumSuite classpath issue Author: Sean Owen Closes #4933 from srowen/SP

spark git commit: SPARK-6569 [STREAMING] Down-grade same-offset message in Kafka streaming to INFO

2015-04-06 Thread srowen
Repository: spark Updated Branches: refs/heads/master 49f38824a -> 9fe412521 SPARK-6569 [STREAMING] Down-grade same-offset message in Kafka streaming to INFO Reduce "is the same as ending offset" message to INFO level per JIRA discussion Author: Sean Owen Closes #5366 from srowen/SPARK-6569

spark git commit: [SPARK-6673] spark-shell.cmd can't start in Windows even when spark was built

2015-04-06 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0b5d028a9 -> 49f38824a [SPARK-6673] spark-shell.cmd can't start in Windows even when spark was built added equivalent script to load-spark-env.sh Author: Masayoshi TSUZUKI Closes #5328 from tsudukim/feature/SPARK-6673 and squashes the fo