spark git commit: [SPARK-4437] update doc for WholeCombineFileRecordReader

2014-12-16 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master c246b95dd - ed362008f [SPARK-4437] update doc for WholeCombineFileRecordReader update doc for WholeCombineFileRecordReader Author: Davies Liu dav...@databricks.com Author: Josh Rosen joshro...@databricks.com Closes #3301 from

spark git commit: [SPARK-4855][mllib] testing the Chi-squared hypothesis test

2014-12-16 Thread meng
Repository: spark Updated Branches: refs/heads/master ed362008f - cb4844749 [SPARK-4855][mllib] testing the Chi-squared hypothesis test This PR tests the pyspark Chi-squared hypothesis test from this commit: c8abddc5164d8cf11cdede6ab3d5d1ea08028708 and moves some of the error messaging in

spark git commit: [SPARK-3405] add subnet-id and vpc-id options to spark_ec2.py

2014-12-16 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master cb4844749 - d12c0711f [SPARK-3405] add subnet-id and vpc-id options to spark_ec2.py Based on this gist: https://gist.github.com/amar-analytx/0b62543621e1f246c0a2 We use security group ids instead of security group to get around this

spark git commit: [SQL] SPARK-4700: Add HTTP protocol spark thrift server

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master d12c0711f - 17688d142 [SQL] SPARK-4700: Add HTTP protocol spark thrift server Add HTTP protocol support and test cases to spark thrift server, so users can deploy thrift server in both TCP and http mode. Author: Judy Nash

spark git commit: [DOCS][SQL] Add a Note on jsonFile having separate JSON objects per line

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 17688d142 - 1a9e35e57 [DOCS][SQL] Add a Note on jsonFile having separate JSON objects per line * This commit hopes to avoid the confusion I faced when trying to submit a regular, valid multi-line JSON file, also see

spark git commit: [SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 1a9e35e57 - dc8280dcc [SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue Author: jerryshao saisai.s...@intel.com Closes #3698 from jerryshao/SPARK-4847 and squashes the following commits: 4741130 [jerryshao] Make

spark git commit: [SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 4f9916f1e - 1b6fc237c [SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue Author: jerryshao saisai.s...@intel.com Closes #3698 from jerryshao/SPARK-4847 and squashes the following commits: 4741130 [jerryshao]

spark git commit: [SPARK-4812][SQL] Fix the initialization issue of 'codegenEnabled'

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master dc8280dcc - 6530243a5 [SPARK-4812][SQL] Fix the initialization issue of 'codegenEnabled' The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`, which can be override by subclasses. Here is a simple example to show

spark git commit: [SPARK-4527][SQl]Add BroadcastNestedLoopJoin operator selection testsuite

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master b0dfdbdd1 - ea1315e3e [SPARK-4527][SQl]Add BroadcastNestedLoopJoin operator selection testsuite In `JoinSuite` add BroadcastNestedLoopJoin operator selection testsuite Author: wangxiaojing u9j...@gmail.com Closes #3395 from

spark git commit: [SPARK-4483][SQL]Optimization about reduce memory costs during the HashOuterJoin

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ea1315e3e - 30f6b85c8 [SPARK-4483][SQL]Optimization about reduce memory costs during the HashOuterJoin In `HashOuterJoin.scala`, spark read data from both side of join operation before zip them together. It is a waste for memory. We are

spark git commit: [Release] Major improvements to generate contributors script

2014-12-16 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 1b6fc237c - 0fb004739 [Release] Major improvements to generate contributors script This commit introduces several major improvements to the script that generates the contributors list for release notes, notably: (1) Use release tags

spark git commit: [Release] Major improvements to generate contributors script

2014-12-16 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 892685b37 - 581f866d8 [Release] Major improvements to generate contributors script This commit introduces several major improvements to the script that generates the contributors list for release notes, notably: (1) Use release tags

spark git commit: [Release] Cache known author translations locally

2014-12-16 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 581f866d8 - 991748d8b [Release] Cache known author translations locally This bypasses unnecessary calls to the Github and JIRA API. Additionally, having a local cache allows us to remember names that we had to manually discover

spark git commit: [SPARK-4798][SQL] A new set of Parquet testing API and test suites

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master b85044ecf - 3b395e105 [SPARK-4798][SQL] A new set of Parquet testing API and test suites This PR provides a set Parquet testing API (see trait `ParquetTest`) that enables developers to write more concise test cases. A new set of Parquet

spark git commit: [SPARK-4744] [SQL] Short circuit evaluation for AND OR in CodeGen

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 3b395e105 - 0aa834ade [SPARK-4744] [SQL] Short circuit evaluation for AND OR in CodeGen Author: Cheng Hao hao.ch...@intel.com Closes #3606 from chenghao-intel/codegen_short_circuit and squashes the following commits: f466303 [Cheng

spark git commit: [SPARK-4375] [SQL] Add 0 argument support for udf

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ddc7ba31c - 770d8153a [SPARK-4375] [SQL] Add 0 argument support for udf Author: Cheng Hao hao.ch...@intel.com Closes #3595 from chenghao-intel/udf0 and squashes the following commits: a858973 [Cheng Hao] Add 0 arguments support for udf

spark git commit: [SPARK-4866] support StructType as key in MapType

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 770d8153a - ec5c4279e [SPARK-4866] support StructType as key in MapType This PR brings support of using StructType(and other hashable types) as key in MapType. Author: Davies Liu dav...@databricks.com Closes #3714 from

spark git commit: [SPARK-4618][SQL] Make foreign DDL commands options case-insensitive

2014-12-16 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ec5c4279e - 60698801e [SPARK-4618][SQL] Make foreign DDL commands options case-insensitive Using lowercase for ```options``` key to make it case-insensitive, then we should use lower case to get value from parameters. So flowing cmd work

spark git commit: [Release] Update contributors list format and sort it

2014-12-16 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 8a69ed33f - beb75aca6 [Release] Update contributors list format and sort it Additionally, we now warn the user when a duplicate author name arises, in which case he/she needs to resolve it manually. Project:

spark git commit: [HOTFIX] Fix RAT exclusion for known_translations file

2014-12-16 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 beb75aca6 - b5919d1b5 [HOTFIX] Fix RAT exclusion for known_translations file Author: Josh Rosen joshro...@databricks.com Closes #3719 from JoshRosen/rat-fix and squashes the following commits: 1542886 [Josh Rosen] [HOTFIX] Fix RAT

spark git commit: [HOTFIX] Fix RAT exclusion for known_translations file

2014-12-16 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 0efd691d9 - c15e7f211 [HOTFIX] Fix RAT exclusion for known_translations file Author: Josh Rosen joshro...@databricks.com Closes #3719 from JoshRosen/rat-fix and squashes the following commits: 1542886 [Josh Rosen] [HOTFIX] Fix RAT