Repository: spark
Updated Branches:
refs/heads/master 60336e3bc -> b3abf0b8d
[SPARK-7663] [MLLIB] Add requirement for word2vec model
JIRA issue [link](https://issues.apache.org/jira/browse/SPARK-7663).
We should check the model size of word2vec, to prevent the unexpected empty.
CC srowen.
A
Repository: spark
Updated Branches:
refs/heads/master b3abf0b8d -> 09265ad7c
[SPARK-7320] [SQL] Add Cube / Rollup for dataframe
Add `cube` & `rollup` for DataFrame
For example:
```scala
testData.rollup($"a" + $"b", $"b").agg(sum($"a" - $"b"))
testData.cube($"a" + $"b", $"b").agg(sum($"a" - $"b
Repository: spark
Updated Branches:
refs/heads/branch-1.4 86893390c -> 10698e113
[SPARK-7320] [SQL] Add Cube / Rollup for dataframe
Add `cube` & `rollup` for DataFrame
For example:
```scala
testData.rollup($"a" + $"b", $"b").agg(sum($"a" - $"b"))
testData.cube($"a" + $"b", $"b").agg(sum($"a" -
Repository: spark
Updated Branches:
refs/heads/master 09265ad7c -> 3ddf051ee
[SPARK-7533] [YARN] Decrease spacing between AM-RM heartbeats.
Added faster RM-heartbeats on pending container allocations with multiplicative
back-off.
Also updated related documentations.
Author: ehnalis
Closes
Repository: spark
Updated Branches:
refs/heads/master 3ddf051ee -> 589b12f8e
[SPARK-7654] [MLLIB] Migrate MLlib to the DataFrame reader/writer API
parquetFile -> read.parquet rxin
Author: Xiangrui Meng
Closes #6281 from mengxr/SPARK-7654 and squashes the following commits:
a79b612 [Xiangru
Repository: spark
Updated Branches:
refs/heads/branch-1.4 10698e113 -> 996e2d4b3
[SPARK-7654] [MLLIB] Migrate MLlib to the DataFrame reader/writer API
parquetFile -> read.parquet rxin
Author: Xiangrui Meng
Closes #6281 from mengxr/SPARK-7654 and squashes the following commits:
a79b612 [Xia
Repository: spark
Updated Branches:
refs/heads/master 589b12f8e -> 98a46f9df
[SPARK-6094] [MLLIB] Add MultilabelMetrics in PySpark/MLlib
Add MultilabelMetrics in PySpark/MLlib
Author: Yanbo Liang
Closes #6276 from yanboliang/spark-6094 and squashes the following commits:
b8e3343 [Yanbo Lia
Repository: spark
Updated Branches:
refs/heads/branch-1.4 996e2d4b3 -> 606ae3e10
[SPARK-6094] [MLLIB] Add MultilabelMetrics in PySpark/MLlib
Add MultilabelMetrics in PySpark/MLlib
Author: Yanbo Liang
Closes #6276 from yanboliang/spark-6094 and squashes the following commits:
b8e3343 [Yanbo
Repository: spark
Updated Branches:
refs/heads/master 98a46f9df -> b631bf73b
[SPARK-7713] [SQL] Use shared broadcast hadoop conf for partitioned table scan.
https://issues.apache.org/jira/browse/SPARK-7713
I tested the performance with the following code:
```scala
import sqlContext._
import s
Repository: spark
Updated Branches:
refs/heads/branch-1.4 606ae3e10 -> 55bd1bb52
[SPARK-7713] [SQL] Use shared broadcast hadoop conf for partitioned table scan.
https://issues.apache.org/jira/browse/SPARK-7713
I tested the performance with the following code:
```scala
import sqlContext._
impo
Repository: spark
Updated Branches:
refs/heads/branch-1.4 55bd1bb52 -> b40c5ed7a
[SPARK-7537] [MLLIB] spark.mllib API updates
Minor updates to the spark.mllib APIs:
1. Add `DeveloperApi` to `PMMLExportable` and add `Experimental` to `toPMML`
methods.
2. Mention `RankingMetrics.of` in the `Ra
Repository: spark
Updated Branches:
refs/heads/master b631bf73b -> 2ad4837cf
[SPARK-7537] [MLLIB] spark.mllib API updates
Minor updates to the spark.mllib APIs:
1. Add `DeveloperApi` to `PMMLExportable` and add `Experimental` to `toPMML`
methods.
2. Mention `RankingMetrics.of` in the `Rankin
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b40c5ed7a -> ae8a854ca
[SPARK-7579] [ML] [DOC] User guide update for OneHotEncoder
Author: Sandy Ryza
Closes #6126 from sryza/sandy-spark-7579 and squashes the following commits:
5af803d [Sandy Ryza] SPARK-7579 [MLLIB] User guide upd
Repository: spark
Updated Branches:
refs/heads/master 2ad4837cf -> 829f1d95b
[SPARK-7579] [ML] [DOC] User guide update for OneHotEncoder
Author: Sandy Ryza
Closes #6126 from sryza/sandy-spark-7579 and squashes the following commits:
5af803d [Sandy Ryza] SPARK-7579 [MLLIB] User guide update
Repository: spark
Updated Branches:
refs/heads/master 829f1d95b -> 6338c40da
Revert "[SPARK-7320] [SQL] Add Cube / Rollup for dataframe"
This reverts commit 10698e1131f665addb454cd498669920699a91b2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.o
Repository: spark
Updated Branches:
refs/heads/branch-1.4 ae8a854ca -> f84bdbce8
Revert "[SPARK-7320] [SQL] Add Cube / Rollup for dataframe"
This reverts commit 10698e1131f665addb454cd498669920699a91b2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apac
Repository: spark
Updated Branches:
refs/heads/master 6338c40da -> 191ee4745
[SPARK-7511] [MLLIB] pyspark ml seed param should be random by default or 42 is
quite funny but not very random
Author: Holden Karau
Closes #6139 from
holdenk/SPARK-7511-pyspark-ml-seed-param-should-be-random-by-d
Repository: spark
Updated Branches:
refs/heads/branch-1.4 f84bdbce8 -> 096cb127a
[SPARK-7511] [MLLIB] pyspark ml seed param should be random by default or 42 is
quite funny but not very random
Author: Holden Karau
Closes #6139 from
holdenk/SPARK-7511-pyspark-ml-seed-param-should-be-random-
Repository: spark
Updated Branches:
refs/heads/master 191ee4745 -> 9b84443dd
[SPARK-7237] [SPARK-7741] [CORE] [STREAMING] Clean more closures that need
cleaning
SPARK-7741 is the equivalent of SPARK-7237 in streaming. This is an alternative
to #6268.
Author: Andrew Or
Closes #6269 from an
Repository: spark
Updated Branches:
refs/heads/branch-1.4 096cb127a -> 23356dd0d
[SPARK-7237] [SPARK-7741] [CORE] [STREAMING] Clean more closures that need
cleaning
SPARK-7741 is the equivalent of SPARK-7237 in streaming. This is an alternative
to #6268.
Author: Andrew Or
Closes #6269 fro
Repository: spark
Updated Branches:
refs/heads/branch-1.4 23356dd0d -> a502e4b84
[SPARK-7767] [STREAMING] Added test for checkpoint serialization in
StreamingContext.start()
Currently, the background checkpointing thread fails silently if the checkpoint
is not serializable. It is hard to deb
Repository: spark
Updated Branches:
refs/heads/master 9b84443dd -> 3c434cbfd
[SPARK-7767] [STREAMING] Added test for checkpoint serialization in
StreamingContext.start()
Currently, the background checkpointing thread fails silently if the checkpoint
is not serializable. It is hard to debug a
Repository: spark
Updated Tags: refs/tags/rc-test [created] 09a1c6231
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Preparing development version 1.4.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/205ed15f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/205ed15f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/205
Repository: spark
Updated Tags: refs/tags/rc-test [deleted] 09a1c6231
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.4 a502e4b84 -> 205ed15f2
Preparing Spark release rc-test
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/09a1c623
Tree: http://git-wip-us.apache.org/repos/asf/spark/t
Repository: spark
Updated Branches:
refs/heads/master 3c434cbfd -> 7956dd7ab
[SPARK-7698] Cache and reuse buffers in ExecutorMemoryAllocator when using heap
allocation
When on-heap memory allocation is used, ExecutorMemoryManager should maintain a
cache / pool of buffers for re-use by tasks.
Repository: spark
Updated Branches:
refs/heads/branch-1.4 205ed15f2 -> 7cea552e1
[SPARK-7698] Cache and reuse buffers in ExecutorMemoryAllocator when using heap
allocation
When on-heap memory allocation is used, ExecutorMemoryManager should maintain a
cache / pool of buffers for re-use by ta
Repository: spark
Updated Branches:
refs/heads/master 7956dd7ab -> f2faa7af3
[SPARK-7251] Perform sequential scan when iterating over BytesToBytesMap
This patch modifies `BytesToBytesMap.iterator()` to iterate through records in
the order that they appear in the data pages rather than iterati
Repository: spark
Updated Branches:
refs/heads/branch-1.4 7cea552e1 -> 82bc518cf
[SPARK-7251] Perform sequential scan when iterating over BytesToBytesMap
This patch modifies `BytesToBytesMap.iterator()` to iterate through records in
the order that they appear in the data pages rather than ite
Preparing development version 1.4.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/534c787b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/534c787b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/534
Repository: spark
Updated Tags: refs/tags/rc-test [created] 5f4d87f60
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.4 82bc518cf -> 534c787b9
Preparing Spark release rc-test
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5f4d87f6
Tree: http://git-wip-us.apache.org/repos/asf/spark/t
Repository: spark
Updated Tags: refs/tags/rc-test [deleted] 5f4d87f60
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.4 8d6684986 -> 5f64269c5
[SPARK-7762] [MLLIB] set default value for outputCol
Set a default value for `outputCol` instead of forcing users to name it. This
is useful for intermediate transformers in the pipeline. jkbradley
Author: Xiang
Preparing development version 1.4.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8d668498
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/8d668498
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/8d6
Repository: spark
Updated Tags: refs/tags/rc-test [created] ae29aeaf8
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/master f2faa7af3 -> c330e52da
[SPARK-7762] [MLLIB] set default value for outputCol
Set a default value for `outputCol` instead of forcing users to name it. This
is useful for intermediate transformers in the pipeline. jkbradley
Author: Xiangrui
Repository: spark
Updated Branches:
refs/heads/branch-1.4 534c787b9 -> 8d6684986
Preparing Spark release rc-test
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ae29aeaf
Tree: http://git-wip-us.apache.org/repos/asf/spark/t
Repository: spark
Updated Tags: refs/tags/rc-test [deleted] ae29aeaf8
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/rc-test [created] 1e458e355
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.4 5f64269c5 -> 9b37e32c5
Preparing Spark release rc-test
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1e458e35
Tree: http://git-wip-us.apache.org/repos/asf/spark/t
Repository: spark
Updated Tags: refs/tags/rc-test [deleted] 1e458e355
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Preparing development version 1.4.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9b37e32c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9b37e32c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9b3
Repository: spark
Updated Branches:
refs/heads/master c330e52da -> 5196efff5
[SPARK-7719] Re-add UnsafeShuffleWriterSuite test that was removed for Java 6
compat
This patch re-adds a test which was removed in
9ebb44f8abb1a13f045eed60190954db904ffef7 due to a Java 6 compatibility issue.
We
Repository: spark
Updated Branches:
refs/heads/branch-1.4 9b37e32c5 -> e1f7de33b
[SPARK-7719] Re-add UnsafeShuffleWriterSuite test that was removed for Java 6
compat
This patch re-adds a test which was removed in
9ebb44f8abb1a13f045eed60190954db904ffef7 due to a Java 6 compatibility issue.
Repository: spark
Updated Branches:
refs/heads/branch-1.4 e1f7de33b -> 0d061ff9e
[SPARK-7750] [WEBUI] Rename endpoints from `json` to `api` to allow fuâ¦
â¦rther extension to non-json outputs too.
Author: Hari Shreedharan
Closes #6273 from harishreedharan/json-to-api and squashes the foll
Repository: spark
Updated Branches:
refs/heads/master 5196efff5 -> a70bf06b7
[SPARK-7750] [WEBUI] Rename endpoints from `json` to `api` to allow fuâ¦
â¦rther extension to non-json outputs too.
Author: Hari Shreedharan
Closes #6273 from harishreedharan/json-to-api and squashes the followin
Repository: spark
Updated Branches:
refs/heads/master a70bf06b7 -> 895baf8f7
[SPARK-] [STREAMING] Fix the flaky test in
org.apache.spark.streaming.BasicOperationsSuite
Just added a guard to make sure a batch has completed before moving to the next
batch.
Author: zsxwing
Closes #6306 f
Repository: spark
Updated Branches:
refs/heads/branch-1.4 0d061ff9e -> b6182ce89
[SPARK-] [STREAMING] Fix the flaky test in
org.apache.spark.streaming.BasicOperationsSuite
Just added a guard to make sure a batch has completed before moving to the next
batch.
Author: zsxwing
Closes #63
Repository: spark
Updated Branches:
refs/heads/master 895baf8f7 -> 42c592adb
[SPARK-7320] [SQL] Add Cube / Rollup for dataframe
This is a follow up for #6257, which broke the maven test.
Add cube & rollup for DataFrame
For example:
```scala
testData.rollup($"a" + $"b", $"b").agg(sum($"a" - $"
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b6182ce89 -> 4fd674336
[SPARK-7320] [SQL] Add Cube / Rollup for dataframe
This is a follow up for #6257, which broke the maven test.
Add cube & rollup for DataFrame
For example:
```scala
testData.rollup($"a" + $"b", $"b").agg(sum($"a"
Repository: spark
Updated Branches:
refs/heads/master 42c592adb -> ddec173cb
[SPARK-7774] [MLLIB] add sqlContext to MLlibTestSparkContext
to simplify test suites that require a SQLContext.
Author: Xiangrui Meng
Closes #6303 from mengxr/SPARK-7774 and squashes the following commits:
0622b5a
Repository: spark
Updated Branches:
refs/heads/branch-1.4 4fd674336 -> 9711e9bf1
[SPARK-7774] [MLLIB] add sqlContext to MLlibTestSparkContext
to simplify test suites that require a SQLContext.
Author: Xiangrui Meng
Closes #6303 from mengxr/SPARK-7774 and squashes the following commits:
062
Repository: spark
Updated Branches:
refs/heads/master ddec173cb -> d0eb9ffe9
[SPARK-7746][SQL] Add FetchSize parameter for JDBC driver
JIRA: https://issues.apache.org/jira/browse/SPARK-7746
Looks like an easy to add parameter but can show significant performance
improvement if the JDBC drive
Repository: spark
Updated Branches:
refs/heads/branch-1.4 9711e9bf1 -> e70be6987
[SPARK-7746][SQL] Add FetchSize parameter for JDBC driver
JIRA: https://issues.apache.org/jira/browse/SPARK-7746
Looks like an easy to add parameter but can show significant performance
improvement if the JDBC d
Repository: spark
Updated Branches:
refs/heads/master d0eb9ffe9 -> 04940c497
[SPARK-7389] [CORE] Tachyon integration improvement
Two main changes:
Add two functions in ExternalBlockManager, which are putValues and getValues
because the implementation may not rely on the putBytes and getBytes
Repository: spark
Updated Branches:
refs/heads/master 04940c497 -> 8ddcb25b3
[SPARK-7606] [SQL] [PySpark] add version to Python SQL API docs
Add version info for public Python SQL API.
cc rxin
Author: Davies Liu
Closes #6295 from davies/versions and squashes the following commits:
cfd91e6
Repository: spark
Updated Branches:
refs/heads/branch-1.4 e70be6987 -> b0e7c6633
[SPARK-7606] [SQL] [PySpark] add version to Python SQL API docs
Add version info for public Python SQL API.
cc rxin
Author: Davies Liu
Closes #6295 from davies/versions and squashes the following commits:
cfd
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b0e7c6633 -> 64762444e
[SPARK-7753] [MLLIB] Update KernelDensity API
Update `KernelDensity` API to make it extensible to different kernels in the
future. `bandwidth` is used instead of `standardDeviation`. The static
`kernelDensity` m
Repository: spark
Updated Branches:
refs/heads/master 8ddcb25b3 -> 947ea1cf5
[SPARK-7753] [MLLIB] Update KernelDensity API
Update `KernelDensity` API to make it extensible to different kernels in the
future. `bandwidth` is used instead of `standardDeviation`. The static
`kernelDensity` metho
61 matches
Mail list logo
