Repository: spark
Updated Branches:
refs/heads/master d8f528ceb -> f10ae4b1e
[SPARK-6717][ML] Clear shuffle files after checkpointing in ALS
## What changes were proposed in this pull request?
When ALS is run with a checkpoint interval, during the checkpoint materialize
the current state and
Repository: spark
Updated Branches:
refs/heads/branch-2.0 52308103e -> 27efd92e3
[SPARK-6717][ML] Clear shuffle files after checkpointing in ALS
## What changes were proposed in this pull request?
When ALS is run with a checkpoint interval, during the checkpoint materialize
the current state
Repository: spark
Updated Branches:
refs/heads/master f10ae4b1e -> dfd9723dd
[MINOR][DOCS] Fix type Information in Quick Start and Programming Guide
Author: Sandeep Singh
Closes #12841 from techaddict/improve_docs_1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http:/
Repository: spark
Updated Branches:
refs/heads/branch-2.0 27efd92e3 -> 07a02e8bb
[MINOR][DOCS] Fix type Information in Quick Start and Programming Guide
Author: Sandeep Singh
Closes #12841 from techaddict/improve_docs_1.
(cherry picked from commit dfd9723dd3b3ff5d47a7f04a4330bf33ffe353ac)
S
Repository: spark
Updated Branches:
refs/heads/branch-2.0 07a02e8bb -> 38f6e66af
[SPARK-15053][BUILD] Fix Java Lint errors on Hive-Thriftserver module
## What changes were proposed in this pull request?
This issue fixes or hides 181 Java linter errors introduced by SPARK-14987
which copied h
Repository: spark
Updated Branches:
refs/heads/master dfd9723dd -> a74445707
[SPARK-15053][BUILD] Fix Java Lint errors on Hive-Thriftserver module
## What changes were proposed in this pull request?
This issue fixes or hides 181 Java linter errors introduced by SPARK-14987
which copied hive
Repository: spark
Updated Branches:
refs/heads/branch-2.0 38f6e66af -> b802979ad
[SPARK-15081] Move AccumulatorV2 and subclasses into util package
## What changes were proposed in this pull request?
This patch moves AccumulatorV2 and subclasses into util package.
## How was this patch tested?
Repository: spark
Updated Branches:
refs/heads/master a74445707 -> d557a5e01
[SPARK-15081] Move AccumulatorV2 and subclasses into util package
## What changes were proposed in this pull request?
This patch moves AccumulatorV2 and subclasses into util package.
## How was this patch tested?
Upd
Repository: spark
Updated Branches:
refs/heads/master d557a5e01 -> 57ac7c182
[SPARK-14897][CORE] Upgrade Jetty to latest version of 8
## What changes were proposed in this pull request?
Update Jetty 8.1 to the latest 2016/02 release, from a 2013/10 release, for
security and bug fixes. This d
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b802979ad -> f03bf7eac
[SPARK-14897][CORE] Upgrade Jetty to latest version of 8
## What changes were proposed in this pull request?
Update Jetty 8.1 to the latest 2016/02 release, from a 2013/10 release, for
security and bug fixes. Th
Repository: spark
Updated Branches:
refs/heads/branch-1.6 10d513feb -> 2db19a3af
[SPARK-14897][CORE] Upgrade Jetty to latest version of 8
Update Jetty 8.1 to the latest 2016/02 release, from a 2013/10 release, for
security and bug fixes. This does not resolve the JIRA necessarily, as it's
st
Repository: spark
Updated Branches:
refs/heads/master 57ac7c182 -> 46965cd01
[SPARK-15057][GRAPHX] Remove stale TODO comment for making `enum` in
GraphGenerators
## What changes were proposed in this pull request?
This PR removes a stale TODO comment in `GraphGenerators.scala`
## How was th
Repository: spark
Updated Branches:
refs/heads/master 46965cd01 -> d26f7cb01
[SPARK-14971][ML][PYSPARK] PySpark ML Params setter code clean up
## What changes were proposed in this pull request?
PySpark ML Params setter code clean up.
For examples,
```setInputCol``` can be simplified from
```
Repository: spark
Updated Branches:
refs/heads/branch-2.0 f03bf7eac -> 932e1b5b2
[SPARK-14971][ML][PYSPARK] PySpark ML Params setter code clean up
## What changes were proposed in this pull request?
PySpark ML Params setter code clean up.
For examples,
```setInputCol``` can be simplified from
Repository: spark
Updated Branches:
refs/heads/master d26f7cb01 -> 8b6491fc0
[SPARK-15091][SPARKR] Fix warnings and a failure in SparkR test cases with
testthat version 1.0.1
## What changes were proposed in this pull request?
Fix warnings and a failure in SparkR test cases with testthat vers
Repository: spark
Updated Branches:
refs/heads/branch-2.0 932e1b5b2 -> a373c39a9
[SPARK-15091][SPARKR] Fix warnings and a failure in SparkR test cases with
testthat version 1.0.1
## What changes were proposed in this pull request?
Fix warnings and a failure in SparkR test cases with testthat
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a373c39a9 -> 17996e7d0
[SPARK-15088] [SQL] Remove SparkSqlSerializer
## What changes were proposed in this pull request?
This patch removes SparkSqlSerializer. I believe this is now dead code.
## How was this patch tested?
Removed a te
Repository: spark
Updated Branches:
refs/heads/master 8b6491fc0 -> 5503e453b
[SPARK-15088] [SQL] Remove SparkSqlSerializer
## What changes were proposed in this pull request?
This patch removes SparkSqlSerializer. I believe this is now dead code.
## How was this patch tested?
Removed a test c
Repository: spark
Updated Branches:
refs/heads/master 5503e453b -> 5bd9a2f69
[SPARK-14884][SQL][STREAMING][WEBUI] Fix call site for continuous queries
## What changes were proposed in this pull request?
Since we've been processing continuous queries in separate threads, the call
sites are th
Repository: spark
Updated Branches:
refs/heads/branch-2.0 17996e7d0 -> 45bc65519
[SPARK-14884][SQL][STREAMING][WEBUI] Fix call site for continuous queries
## What changes were proposed in this pull request?
Since we've been processing continuous queries in separate threads, the call
sites ar
Repository: spark
Updated Branches:
refs/heads/master 5bd9a2f69 -> 4ad492c40
[SPARK-14716][SQL] Added support for partitioning in FileStreamSink
# What changes were proposed in this pull request?
Support partitioning in the file stream sink. This is implemented using a new,
but simpler code
Repository: spark
Updated Branches:
refs/heads/branch-2.0 45bc65519 -> c5b7e1f70
[SPARK-14716][SQL] Added support for partitioning in FileStreamSink
# What changes were proposed in this pull request?
Support partitioning in the file stream sink. This is implemented using a new,
but simpler c
Repository: spark
Updated Branches:
refs/heads/branch-2.0 c5b7e1f70 -> 31e5a2a76
[SPARK-14860][TESTS] Create a new Waiter in reset to bypass an issue of
ScalaTest's Waiter.wait
## What changes were proposed in this pull request?
This PR updates `QueryStatusCollector.reset` to create Waiter i
Repository: spark
Updated Branches:
refs/heads/master 4ad492c40 -> b545d7521
[SPARK-14860][TESTS] Create a new Waiter in reset to bypass an issue of
ScalaTest's Waiter.wait
## What changes were proposed in this pull request?
This PR updates `QueryStatusCollector.reset` to create Waiter inste
Repository: spark
Updated Branches:
refs/heads/master b545d7521 -> ca813330c
[SPARK-15087][CORE][SQL] Remove AccumulatorV2.localValue and keep only value
## What changes were proposed in this pull request?
Remove AccumulatorV2.localValue and keep only value
## How was this patch tested?
exist
Repository: spark
Updated Branches:
refs/heads/branch-2.0 31e5a2a76 -> 2b715251d
[SPARK-15087][CORE][SQL] Remove AccumulatorV2.localValue and keep only value
## What changes were proposed in this pull request?
Remove AccumulatorV2.localValue and keep only value
## How was this patch tested?
e
Repository: spark
Updated Branches:
refs/heads/master ca813330c -> 439e36101
[SPARK-9819][STREAMING][DOCUMENTATION] Clarify doc for invReduceFunc in
incremental versions of reduceByWindow
- that reduceFunc and invReduceFunc should be associative
- that the intermediate result in iterated appl
Repository: spark
Updated Branches:
refs/heads/branch-2.0 69e030226 -> a08d2c954
[SPARK-15082][CORE] Improve unit test coverage for AccumulatorV2
## What changes were proposed in this pull request?
Added tests for ListAccumulator and LegacyAccumulatorWrapper, test for
ListAccumulator is one s
Repository: spark
Updated Branches:
refs/heads/master 439e36101 -> 84b3a4a87
[SPARK-15082][CORE] Improve unit test coverage for AccumulatorV2
## What changes were proposed in this pull request?
Added tests for ListAccumulator and LegacyAccumulatorWrapper, test for
ListAccumulator is one simil
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2b715251d -> 69e030226
[SPARK-9819][STREAMING][DOCUMENTATION] Clarify doc for invReduceFunc in
incremental versions of reduceByWindow
- that reduceFunc and invReduceFunc should be associative
- that the intermediate result in iterated
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a08d2c954 -> 50adfcdce
[SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to
avoid dead lock
## What changes were proposed in this pull request?
In some cases, fine-grained lock have race condition with class-loa
Repository: spark
Updated Branches:
refs/heads/master 84b3a4a87 -> f5623b460
[SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to
avoid dead lock
## What changes were proposed in this pull request?
In some cases, fine-grained lock have race condition with class-loader
Repository: spark
Updated Branches:
refs/heads/branch-2.0 50adfcdce -> 4bacdebe9
[SPARK-14234][CORE] Executor crashes for TaskRunner thread interruption
## What changes were proposed in this pull request?
Resetting the task interruption status before updating the task status.
## How was this
Repository: spark
Updated Branches:
refs/heads/master f5623b460 -> 659f635d3
[SPARK-14234][CORE] Executor crashes for TaskRunner thread interruption
## What changes were proposed in this pull request?
Resetting the task interruption status before updating the task status.
## How was this patc
Repository: spark
Updated Branches:
refs/heads/master 659f635d3 -> a4aed7171
[SPARK-14521] [SQL] StackOverflowError in Kryo when executing TPC-DS
## What changes were proposed in this pull request?
Observed stackOverflowError in Kryo when executing TPC-DS Query27. Spark thrift
server disable
Repository: spark
Updated Branches:
refs/heads/branch-2.0 4bacdebe9 -> a2369c63f
[SPARK-14521] [SQL] StackOverflowError in Kryo when executing TPC-DS
## What changes were proposed in this pull request?
Observed stackOverflowError in Kryo when executing TPC-DS Query27. Spark thrift
server dis
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a2369c63f -> b67668bc4
[SPARK-11316] coalesce doesn't handle UnionRDD with partial locality properly
## What changes were proposed in this pull request?
coalesce doesn't handle UnionRDD with partial locality properly. I had a user
wh
Repository: spark
Updated Branches:
refs/heads/master a4aed7171 -> 83ee92f60
[SPARK-11316] coalesce doesn't handle UnionRDD with partial locality properly
## What changes were proposed in this pull request?
coalesce doesn't handle UnionRDD with partial locality properly. I had a user
who ha
Repository: spark
Updated Branches:
refs/heads/master 83ee92f60 -> 588cac414
[SPARK-15073][SQL] Hide SparkSession constructor from the public
## What changes were proposed in this pull request?
Users should use the builder pattern instead.
## How was this patch tested?
Jenks.
Author: Andre
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b67668bc4 -> ca9917160
[SPARK-15073][SQL] Hide SparkSession constructor from the public
## What changes were proposed in this pull request?
Users should use the builder pattern instead.
## How was this patch tested?
Jenks.
Author: A
Repository: spark
Updated Branches:
refs/heads/master 588cac414 -> d6c7b2a5c
[SPARK-15095][SQL] drop binary mode in ThriftServer
## What changes were proposed in this pull request?
This PR drop the support for binary mode in ThriftServer, only HTTP mode is
supported now, to reduce the mainta
Repository: spark
Updated Branches:
refs/heads/branch-2.0 ca9917160 -> c50746a4a
[SPARK-15095][SQL] drop binary mode in ThriftServer
## What changes were proposed in this pull request?
This PR drop the support for binary mode in ThriftServer, only HTTP mode is
supported now, to reduce the ma
Repository: spark
Updated Branches:
refs/heads/master d6c7b2a5c -> 2e2a6211c
[SPARK-14973][ML] The CrossValidator and TrainValidationSplit miss the seed
when saving and loading
## What changes were proposed in this pull request?
https://issues.apache.org/jira/browse/SPARK-14973
Add seed sup
Repository: spark
Updated Branches:
refs/heads/branch-2.0 c50746a4a -> f21eddb29
[SPARK-14973][ML] The CrossValidator and TrainValidationSplit miss the seed
when saving and loading
## What changes were proposed in this pull request?
https://issues.apache.org/jira/browse/SPARK-14973
Add seed
Repository: spark
Updated Branches:
refs/heads/master 2e2a6211c -> 71296c041
[SPARK-15056][SQL] Parse Unsupported Sampling Syntax and Issue Better Exceptions
What changes were proposed in this pull request?
Compared with the current Spark parser, there are two extra syntax are
supported
Repository: spark
Updated Branches:
refs/heads/branch-2.0 f21eddb29 -> 70c5ca220
[SPARK-15056][SQL] Parse Unsupported Sampling Syntax and Issue Better Exceptions
What changes were proposed in this pull request?
Compared with the current Spark parser, there are two extra syntax are
suppor
Repository: spark
Updated Branches:
refs/heads/branch-2.0 70c5ca220 -> 51bb0bcc8
[SQL-15102][SQL] remove delegation token support from ThriftServer
## What changes were proposed in this pull request?
These API is only useful for Hadoop, may not work for Spark SQL.
The APIs is kept for source
Repository: spark
Updated Branches:
refs/heads/master 71296c041 -> 028c6a5db
[SQL-15102][SQL] remove delegation token support from ThriftServer
## What changes were proposed in this pull request?
These API is only useful for Hadoop, may not work for Spark SQL.
The APIs is kept for source com
Repository: spark
Updated Branches:
refs/heads/master 028c6a5db -> dbacd9998
[SPARK-15104] Fix spacing in log line
Otherwise get logs that look like this (note no space before NODE_LOCAL)
```
INFO [2016-05-03 21:18:51,477] org.apache.spark.scheduler.TaskSetManager:
Starting task 0.0 in stag
Repository: spark
Updated Branches:
refs/heads/branch-2.0 51bb0bcc8 -> c212307b9
[SPARK-15104] Fix spacing in log line
Otherwise get logs that look like this (note no space before NODE_LOCAL)
```
INFO [2016-05-03 21:18:51,477] org.apache.spark.scheduler.TaskSetManager:
Starting task 0.0 in
Repository: spark
Updated Branches:
refs/heads/master dbacd9998 -> c4e0fde87
[MINOR][DOC] Fixed some python snippets in mllib data types documentation.
## What changes were proposed in this pull request?
Some python snippets is using scala imports and comments.
## How was this patch tested?
Repository: spark
Updated Branches:
refs/heads/branch-2.0 c212307b9 -> 0d16b7f3a
[MINOR][DOC] Fixed some python snippets in mllib data types documentation.
## What changes were proposed in this pull request?
Some python snippets is using scala imports and comments.
## How was this patch test
Repository: spark
Updated Branches:
refs/heads/branch-2.0 0d16b7f3a -> 5625b037a
[SPARK-14422][SQL] Improve handling of optional configs in SQLConf
## What changes were proposed in this pull request?
Create a new API for handling Optional Configs in SQLConf.
Right now `getConf` for `OptionalCo
Repository: spark
Updated Branches:
refs/heads/master c4e0fde87 -> a8d56f538
[SPARK-14422][SQL] Improve handling of optional configs in SQLConf
## What changes were proposed in this pull request?
Create a new API for handling Optional Configs in SQLConf.
Right now `getConf` for `OptionalConfig
Repository: spark
Updated Branches:
refs/heads/branch-2.0 5625b037a -> 4c7f5a74d
[SPARK-14645][MESOS] Fix python running on cluster mode mesos to have non local
uris
## What changes were proposed in this pull request?
Fix SparkSubmit to allow non-local python uris
## How was this patch test
Repository: spark
Updated Branches:
refs/heads/master a8d56f538 -> c1839c991
[SPARK-14645][MESOS] Fix python running on cluster mode mesos to have non local
uris
## What changes were proposed in this pull request?
Fix SparkSubmit to allow non-local python uris
## How was this patch tested?
Repository: spark
Updated Branches:
refs/heads/master c1839c991 -> 0903a185c
[SPARK-15084][PYTHON][SQL] Use builder pattern to create SparkSession in
PySpark.
## What changes were proposed in this pull request?
This is a python port of corresponding Scala builder pattern code. `sql.py` is
m
Repository: spark
Updated Branches:
refs/heads/branch-2.0 4c7f5a74d -> 5e15615d1
[SPARK-15084][PYTHON][SQL] Use builder pattern to create SparkSession in
PySpark.
## What changes were proposed in this pull request?
This is a python port of corresponding Scala builder pattern code. `sql.py` i
Repository: spark
Updated Branches:
refs/heads/master 0903a185c -> 9e4928b7e
[SPARK-15097][SQL] make Dataset.sqlContext a stable identifier for imports
## What changes were proposed in this pull request?
Make Dataset.sqlContext a lazy val so that its a stable identifier and can be
used for im
Repository: spark
Updated Branches:
refs/heads/branch-2.0 5e15615d1 -> 95d359abd
[SPARK-15097][SQL] make Dataset.sqlContext a stable identifier for imports
## What changes were proposed in this pull request?
Make Dataset.sqlContext a lazy val so that its a stable identifier and can be
used fo
Repository: spark
Updated Branches:
refs/heads/branch-2.0 95d359abd -> 940b8f60b
[SPARK-14414][SQL] Make DDL exceptions more consistent
## What changes were proposed in this pull request?
Just a bunch of small tweaks on DDL exception messages.
## How was this patch tested?
`DDLCommandSuite`
Repository: spark
Updated Branches:
refs/heads/master 9e4928b7e -> 6ba17cd14
[SPARK-14414][SQL] Make DDL exceptions more consistent
## What changes were proposed in this pull request?
Just a bunch of small tweaks on DDL exception messages.
## How was this patch tested?
`DDLCommandSuite` et
Repository: spark
Updated Branches:
refs/heads/master 6ba17cd14 -> 348c13898
[SPARK-15095][SQL] remove HiveSessionHook from ThriftServer
## What changes were proposed in this pull request?
Remove HiveSessionHook
## How was this patch tested?
No tests needed.
Author: Davies Liu
Closes #12
Repository: spark
Updated Branches:
refs/heads/branch-2.0 940b8f60b -> fd3accca6
[SPARK-15095][SQL] remove HiveSessionHook from ThriftServer
## What changes were proposed in this pull request?
Remove HiveSessionHook
## How was this patch tested?
No tests needed.
Author: Davies Liu
Closes
Repository: spark
Updated Branches:
refs/heads/master 348c13898 -> 695f0e919
[SPARK-15107][SQL] Allow varying # iterations by test case in Benchmark
## What changes were proposed in this pull request?
This patch changes our micro-benchmark util to allow setting different
iteration numbers for
Repository: spark
Updated Branches:
refs/heads/branch-2.0 fd3accca6 -> 2ff667606
[SPARK-15107][SQL] Allow varying # iterations by test case in Benchmark
This patch changes our micro-benchmark util to allow setting different
iteration numbers for different test cases. For some of our benchmark
Repository: spark
Updated Branches:
refs/heads/master 695f0e919 -> bc3760d40
[SPARK-14237][SQL] De-duplicate partition value appending logic in various
buildReader() implementations
## What changes were proposed in this pull request?
Currently, various `FileFormat` data sources share approxi
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2ff667606 -> 64ad9ba27
[SPARK-14237][SQL] De-duplicate partition value appending logic in various
buildReader() implementations
## What changes were proposed in this pull request?
Currently, various `FileFormat` data sources share app
68 matches
Mail list logo