spark git commit: [SPARK-5878] fix DataFrame.repartition() in Python

2015-02-18 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 9a565b84f - aca799159 [SPARK-5878] fix DataFrame.repartition() in Python Also add tests for distinct() Author: Davies Liu dav...@databricks.com Closes #4667 from davies/repartition and squashes the following commits: 79059fd [Davies

spark git commit: [SPARK-5878] fix DataFrame.repartition() in Python

2015-02-18 Thread rxin
Repository: spark Updated Branches: refs/heads/master de0dd6de2 - c1b6fa983 [SPARK-5878] fix DataFrame.repartition() in Python Also add tests for distinct() Author: Davies Liu dav...@databricks.com Closes #4667 from davies/repartition and squashes the following commits: 79059fd [Davies

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 aca799159 - 3997e74f6 SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master c1b6fa983 - e79a7a626 SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: Avoid deprecation warnings in JDBCSuite.

2015-02-18 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 2bd33ce62 - 9a565b84f Avoid deprecation warnings in JDBCSuite. This pull request replaces calls to deprecated methods from `java.util.Date` with near-equivalents in `java.util.Calendar`. Author: Tor Myklebust tmykl...@gmail.com

spark git commit: Avoid deprecation warnings in JDBCSuite.

2015-02-18 Thread rxin
Repository: spark Updated Branches: refs/heads/master 61ab08549 - de0dd6de2 Avoid deprecation warnings in JDBCSuite. This pull request replaces calls to deprecated methods from `java.util.Date` with near-equivalents in `java.util.Calendar`. Author: Tor Myklebust tmykl...@gmail.com Closes

spark git commit: SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.2 6be36d5a8 - 068ba45cf SPARK-4610 addendum: [Minor] [MLlib] Minor doc fix in GBT classification example numClassesForClassification has been renamed to numClasses. Author: MechCoder manojkumarsivaraj...@gmail.com Closes #4672 from

spark git commit: [SPARK-5507] Added documentation for BlockMatrix

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 661fbd38f - 56f8f295c [SPARK-5507] Added documentation for BlockMatrix Docs for BlockMatrix. mengxr Author: Burak Yavuz brk...@gmail.com Closes #4664 from brkyvz/SPARK-5507PR and squashes the following commits: 4db30b0 [Burak Yavuz]

spark git commit: [SPARK-5519][MLLIB] add user guide with example code for fp-growth

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 9f256ce84 - 661fbd38f [SPARK-5519][MLLIB] add user guide with example code for fp-growth The API is still not very Java-friendly because `Array[Item]` in `freqItemsets` is recognized as `Object` in Java. We might want to define a case

spark git commit: [SPARK-5507] Added documentation for BlockMatrix

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/master 85e9d091d - a8eb92dcb [SPARK-5507] Added documentation for BlockMatrix Docs for BlockMatrix. mengxr Author: Burak Yavuz brk...@gmail.com Closes #4664 from brkyvz/SPARK-5507PR and squashes the following commits: 4db30b0 [Burak Yavuz]

spark git commit: [SPARK-5519][MLLIB] add user guide with example code for fp-growth

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/master 5aecdcf1f - 85e9d091d [SPARK-5519][MLLIB] add user guide with example code for fp-growth The API is still not very Java-friendly because `Array[Item]` in `freqItemsets` is recognized as `Object` in Java. We might want to define a case

spark git commit: [SPARK-4903][SQL]Backport the bug fix for SPARK-4903

2015-02-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 068ba45cf - 36e15b48e [SPARK-4903][SQL]Backport the bug fix for SPARK-4903 The original fix was a part of https://issues.apache.org/jira/browse/SPARK-4912 (commit

spark git commit: [SPARK-5722] [SQL] [PySpark] infer int as LongType

2015-02-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master f0e3b7107 - aa8f10e82 [SPARK-5722] [SQL] [PySpark] infer int as LongType The `int` is 64-bit on 64-bit machine (very common now), we should infer it as LongType for it in Spark SQL. Also, LongType in SQL will come back as `int`. Author:

spark git commit: [SPARK-5722] [SQL] [PySpark] infer int as LongType

2015-02-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 b86e44cd9 - 470cba82c [SPARK-5722] [SQL] [PySpark] infer int as LongType The `int` is 64-bit on 64-bit machine (very common now), we should infer it as LongType for it in Spark SQL. Also, LongType in SQL will come back as `int`.

spark git commit: [SPARK-5840][SQL] HiveContext cannot be serialized due to tuple extraction

2015-02-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a8eb92dcb - f0e3b7107 [SPARK-5840][SQL] HiveContext cannot be serialized due to tuple extraction Also added test cases for checking the serializability of HiveContext and SQLContext. Author: Reynold Xin r...@databricks.com Closes #4628

spark git commit: [SPARK-5840][SQL] HiveContext cannot be serialized due to tuple extraction

2015-02-18 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 56f8f295c - b86e44cd9 [SPARK-5840][SQL] HiveContext cannot be serialized due to tuple extraction Also added test cases for checking the serializability of HiveContext and SQLContext. Author: Reynold Xin r...@databricks.com Closes

spark git commit: SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master 82197ed3b - 5aecdcf1f SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS Correct exclusion path for JBLAS native libs. (More explanation coming soon on the mailing list re: 1.3.0

spark git commit: SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 3997e74f6 - 9f256ce84 SPARK-5669 [BUILD] [HOTFIX] Spark assembly includes incompatibly licensed libgfortran, libgcc code via JBLAS Correct exclusion path for JBLAS native libs. (More explanation coming soon on the mailing list re:

spark git commit: [SPARK-4949]shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block.

2015-02-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master e79a7a626 - 82197ed3b [SPARK-4949]shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block. A variable `shutdownCallback` in SparkDeploySchedulerBackend can be accessed from multiple threads so it should

spark git commit: [SPARK-5846] Correctly set job description and pool for SQL jobs

2015-02-18 Thread lian
Repository: spark Updated Branches: refs/heads/master d12d2ad76 - e945aa613 [SPARK-5846] Correctly set job description and pool for SQL jobs marmbrus am I missing something obvious here? I verified that this fixes the problem for me (on 1.2.1) on EC2, but I'm confused about how others

spark git commit: [SPARK-5846] Correctly set job description and pool for SQL jobs

2015-02-18 Thread lian
Repository: spark Updated Branches: refs/heads/branch-1.2 36e15b48e - f6ee80b18 [SPARK-5846] Correctly set job description and pool for SQL jobs This is #4630 but modified for the 1.2 branch, because I'm guessing it makes sense to fix this issue in that branch (again, unless I missed

spark git commit: [SPARK-5879][MLLIB] update PIC user guide and add a Java example

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 470cba82c - a64f374a3 [SPARK-5879][MLLIB] update PIC user guide and add a Java example Updated PIC user guide to reflect API changes and added a simple Java example. The API is still not very Java-friendly. I created SPARK-5990 for

spark git commit: [SPARK-5879][MLLIB] update PIC user guide and add a Java example

2015-02-18 Thread meng
Repository: spark Updated Branches: refs/heads/master aa8f10e82 - d12d2ad76 [SPARK-5879][MLLIB] update PIC user guide and add a Java example Updated PIC user guide to reflect API changes and added a simple Java example. The API is still not very Java-friendly. I created SPARK-5990 for this