Repository: spark
Updated Branches:
refs/heads/master ba5487c06 -> 9e266d07a
[SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in
examples
## What changes were proposed in this pull request?
1, Use `SparkSession` according to
Repository: spark
Updated Branches:
refs/heads/branch-2.0 7d187539e -> 86acb5efd
[SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in
examples
## What changes were proposed in this pull request?
1, Use `SparkSession` according to
Repository: spark
Updated Branches:
refs/heads/branch-2.0 0b14b3f13 -> 7d187539e
[SPARK-15072][SQL][PYSPARK][HOT-FIX] Remove SparkSession.withHiveSupport from
readwrite.py
## What changes were proposed in this pull request?
Seems
Repository: spark
Updated Branches:
refs/heads/master f036dd7ce -> ba5487c06
[SPARK-15072][SQL][PYSPARK][HOT-FIX] Remove SparkSession.withHiveSupport from
readwrite.py
## What changes were proposed in this pull request?
Seems
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b2b04c6da -> 0b14b3f13
[SPARK-14346] SHOW CREATE TABLE for data source tables
## What changes were proposed in this pull request?
This PR adds native `SHOW CREATE TABLE` DDL command for data source tables.
Support for Hive tables
Repository: spark
Updated Branches:
refs/heads/master db573fc74 -> ff92eb2e8
[SPARK-15080][CORE] Break copyAndReset into copy and reset
## What changes were proposed in this pull request?
Break copyAndReset into two methods copy and reset instead of just one.
## How was this patch tested?
Repository: spark
Updated Branches:
refs/heads/branch-2.0 114be703d -> b2b04c6da
[SPARK-15080][CORE] Break copyAndReset into copy and reset
## What changes were proposed in this pull request?
Break copyAndReset into two methods copy and reset instead of just one.
## How was this patch
Repository: spark
Updated Branches:
refs/heads/branch-2.0 f8804bb10 -> 114be703d
[SPARK-15072][SQL][PYSPARK] FollowUp: Remove SparkSession.withHiveSupport in
PySpark
## What changes were proposed in this pull request?
This is a followup of https://github.com/apache/spark/pull/12851
Remove
Repository: spark
Updated Branches:
refs/heads/master f14c4ba00 -> 603f4453a
[SPARK-15264][SPARK-15274][SQL] CSV Reader Error on Blank Column Names
## What changes were proposed in this pull request?
When a CSV begins with:
- `,,`
OR
- `"","",`
meaning that the first column names are either
Repository: spark
Updated Branches:
refs/heads/branch-2.0 f9ea54575 -> f763c1485
[SPARK-15276][SQL] CREATE TABLE with LOCATION should imply EXTERNAL
## What changes were proposed in this pull request?
Before:
```sql
-- uses that location but issues a warning
CREATE TABLE my_tab LOCATION
Repository: spark
Updated Branches:
refs/heads/master b9cf617a6 -> f14c4ba00
[SPARK-15276][SQL] CREATE TABLE with LOCATION should imply EXTERNAL
## What changes were proposed in this pull request?
Before:
```sql
-- uses that location but issues a warning
CREATE TABLE my_tab LOCATION
Repository: spark
Updated Branches:
refs/heads/branch-1.6 e2a43d007 -> fd2da7b91
[SPARK-15260] Atomically resize memory pools (branch 1.6)
## What changes were proposed in this pull request?
(This is the branch-1.6 version of #13039)
When we acquire execution memory, we do a lot of things
Repository: spark
Updated Branches:
refs/heads/branch-2.0 4e56857ca -> f9ea54575
[SPARK-15256] [SQL] [PySpark] Clarify DataFrameReader.jdbc() docstring
This PR:
* Corrects the documentation for the `properties` parameter, which is supposed
to be a dictionary and not a list.
* Generally
Repository: spark
Updated Branches:
refs/heads/master 8881765ac -> b9cf617a6
[SPARK-15256] [SQL] [PySpark] Clarify DataFrameReader.jdbc() docstring
This PR:
* Corrects the documentation for the `properties` parameter, which is supposed
to be a dictionary and not a list.
* Generally clarifies
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b1e14d9bf -> 4e56857ca
[SPARK-15257][SQL] Require CREATE EXTERNAL TABLE to specify LOCATION
## What changes were proposed in this pull request?
Before:
```sql
-- uses warehouse dir anyway
CREATE EXTERNAL TABLE my_tab
-- doesn't
Repository: spark
Updated Branches:
refs/heads/master 40ba87f76 -> 8881765ac
[SPARK-15257][SQL] Require CREATE EXTERNAL TABLE to specify LOCATION
## What changes were proposed in this pull request?
Before:
```sql
-- uses warehouse dir anyway
CREATE EXTERNAL TABLE my_tab
-- doesn't actually
Repository: spark
Updated Branches:
refs/heads/master de9c85cca -> 40ba87f76
[SPARK-15278] [SQL] Remove experimental tag from Python DataFrame
## What changes were proposed in this pull request?
Earlier we removed experimental tag for Scala/Java DataFrames, but haven't done
so for Python.
Repository: spark
Updated Branches:
refs/heads/branch-2.0 0699acc1b -> b1e14d9bf
[SPARK-15278] [SQL] Remove experimental tag from Python DataFrame
## What changes were proposed in this pull request?
Earlier we removed experimental tag for Scala/Java DataFrames, but haven't done
so for
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2454f6abf -> 0699acc1b
[SPARK-15270] [SQL] Use SparkSession Builder to build a session with HiveSupport
## What changes were proposed in this pull request?
Before:
Creating a hiveContext was failing
```python
from pyspark.sql import
Repository: spark
Updated Branches:
refs/heads/master 40a949aae -> de9c85cca
[SPARK-15270] [SQL] Use SparkSession Builder to build a session with HiveSupport
## What changes were proposed in this pull request?
Before:
Creating a hiveContext was failing
```python
from pyspark.sql import
Repository: spark
Updated Branches:
refs/heads/master 7ecd49688 -> 40a949aae
[SPARK-15262] Synchronize block manager / scheduler executor state
## What changes were proposed in this pull request?
If an executor is still alive even after the scheduler has removed its
metadata, we may receive
Repository: spark
Updated Branches:
refs/heads/branch-1.6 ced71d353 -> e2a43d007
[SPARK-15262] Synchronize block manager / scheduler executor state
## What changes were proposed in this pull request?
If an executor is still alive even after the scheduler has removed its
metadata, we may
Repository: spark
Updated Branches:
refs/heads/branch-2.0 6e08eb469 -> 2454f6abf
[SPARK-15262] Synchronize block manager / scheduler executor state
## What changes were proposed in this pull request?
If an executor is still alive even after the scheduler has removed its
metadata, we may
Repository: spark
Updated Branches:
refs/heads/master bb88ad4e0 -> 7ecd49688
[SPARK-12200][SQL] Add __contains__ implementation to Row
https://issues.apache.org/jira/browse/SPARK-12200
Author: Maciej Brynski
Author: Maciej BryÅski
Repository: spark
Updated Branches:
refs/heads/branch-2.0 83050ddb8 -> 6e08eb469
[SPARK-12200][SQL] Add __contains__ implementation to Row
https://issues.apache.org/jira/browse/SPARK-12200
Author: Maciej Brynski
Author: Maciej BryÅski
Repository: spark
Updated Branches:
refs/heads/master 81c68eceb -> bb88ad4e0
[SPARK-15260] Atomically resize memory pools
## What changes were proposed in this pull request?
When we acquire execution memory, we do a lot of things between shrinking the
storage memory pool and enlarging the
Repository: spark
Updated Branches:
refs/heads/branch-2.0 6b36185d0 -> 83050ddb8
[SPARK-15260] Atomically resize memory pools
## What changes were proposed in this pull request?
When we acquire execution memory, we do a lot of things between shrinking the
storage memory pool and enlarging
Repository: spark
Updated Branches:
refs/heads/branch-2.0 56e1e2f17 -> 6b36185d0
[SPARK-15248][SQL] Make MetastoreFileCatalog consider directories from
partition specs of a partitioned metastore table
Table partitions can be added with locations different from default warehouse
location of
Repository: spark
Updated Branches:
refs/heads/master 89e67d666 -> 81c68eceb
[SPARK-15248][SQL] Make MetastoreFileCatalog consider directories from
partition specs of a partitioned metastore table
Table partitions can be added with locations different from default warehouse
location of a
http://git-wip-us.apache.org/repos/asf/spark/blob/89e67d66/external/kafka-0-8/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/89e67d66/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/56e1e2f1/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/56e1e2f1/external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
--
diff --git
[SPARK-15085][STREAMING][KAFKA] Rename streaming-kafka artifact
## What changes were proposed in this pull request?
Renaming the streaming-kafka artifact to include kafka version, in anticipation
of needing a different artifact for later kafka versions
## How was this patch tested?
Unit tests
http://git-wip-us.apache.org/repos/asf/spark/blob/56e1e2f1/external/kafka-0-8/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/branch-2.0 e3703c411 -> 56e1e2f17
http://git-wip-us.apache.org/repos/asf/spark/blob/56e1e2f1/external/kafka/src/test/java/org/apache/spark/streaming/kafka/JavaKafkaRDDSuite.java
[SPARK-15085][STREAMING][KAFKA] Rename streaming-kafka artifact
## What changes were proposed in this pull request?
Renaming the streaming-kafka artifact to include kafka version, in anticipation
of needing a different artifact for later kafka versions
## How was this patch tested?
Unit tests
Repository: spark
Updated Branches:
refs/heads/master 6d0368ab8 -> 89e67d666
http://git-wip-us.apache.org/repos/asf/spark/blob/89e67d66/external/kafka/src/test/java/org/apache/spark/streaming/kafka/JavaKafkaRDDSuite.java
--
http://git-wip-us.apache.org/repos/asf/spark/blob/89e67d66/external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/master d8935db5e -> 293143797
[SPARK-15037] [SQL] [MLLIB] Part2: Use SparkSession instead of SQLContext in
Python TestSuites
## What changes were proposed in this pull request?
Use SparkSession instead of SQLContext in Python TestSuites
## How
[SPARK-13522][CORE] Fix the exit log place for heartbeat
## What changes were proposed in this pull request?
Just fixed the log place introduced by #11401
## How was this patch tested?
unit tests.
Author: Shixiong Zhu
Closes #11432 from
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d1654864a -> ced71d353
[SPARK-13519][CORE] Driver should tell Executor to stop itself when cleaning
executor's state
## What changes were proposed in this pull request?
When the driver removes an executor's state, the connection
[SPARK-13522][CORE] Executor should kill itself when it's unable to heartbeat
to driver more than N times
## What changes were proposed in this pull request?
Sometimes, network disconnection event won't be triggered for other potential
race conditions that we may not have thought of, then the
Repository: spark
Updated Branches:
refs/heads/branch-2.0 1b90adc03 -> e3703c411
[SPARK-15259] Sort time metric should not include spill and record insertion
time
## What changes were proposed in this pull request?
After SPARK-14669 it seems the sort time metric includes both spill and
Repository: spark
Updated Branches:
refs/heads/branch-2.0 381a82589 -> 1b90adc03
[SPARK-15037] [SQL] [MLLIB] Part2: Use SparkSession instead of SQLContext in
Python TestSuites
## What changes were proposed in this pull request?
Use SparkSession instead of SQLContext in Python TestSuites
##
Repository: spark
Updated Branches:
refs/heads/branch-2.0 403ba6513 -> 381a82589
[SPARK-15241] [SPARK-15242] [SQL] fix 2 decimal-related issues in RowEncoder
## What changes were proposed in this pull request?
SPARK-15241: We now support java decimal and catalyst decimal in external row,
it
Repository: spark
Updated Branches:
refs/heads/master e1576478b -> d8935db5e
[SPARK-15241] [SPARK-15242] [SQL] fix 2 decimal-related issues in RowEncoder
## What changes were proposed in this pull request?
SPARK-15241: We now support java decimal and catalyst decimal in external row,
it
Repository: spark
Updated Branches:
refs/heads/master a5f9fdbba -> e1576478b
[SPARK-14933][HOTFIX] Replace `sqlContext` with `spark`.
## What changes were proposed in this pull request?
This fixes compile errors.
## How was this patch tested?
Pass the Jenkins tests.
Author: Dongjoon Hyun
Repository: spark
Updated Branches:
refs/heads/branch-2.0 0858a82c1 -> 403ba6513
[SPARK-14933][HOTFIX] Replace `sqlContext` with `spark`.
## What changes were proposed in this pull request?
This fixes compile errors.
## How was this patch tested?
Pass the Jenkins tests.
Author: Dongjoon
Repository: spark
Updated Branches:
refs/heads/branch-2.0 749c29bc0 -> 0858a82c1
[SPARK-15268][SQL] Make JavaTypeInference work with UDTRegistration
## What changes were proposed in this pull request?
We have a private `UDTRegistration` API to register user defined type.
Currently
Repository: spark
Updated Branches:
refs/heads/branch-2.0 3bd7a89bd -> 749c29bc0
[SPARK-14933][SQL] Failed to create view out of a parquet or orc table
## What changes were proposed in this pull request?
Symptom
If a table is created as parquet or ORC table with hive syntaxt DDL, such
Repository: spark
Updated Branches:
refs/heads/master d88afabdf -> 427c20dd6
[SPARK-14933][SQL] Failed to create view out of a parquet or orc table
## What changes were proposed in this pull request?
Symptom
If a table is created as parquet or ORC table with hive syntaxt DDL, such as
Repository: spark
Updated Branches:
refs/heads/master fafc95af7 -> d88afabdf
[SPARK-15150][EXAMPLE][DOC] Update LDA examples
## What changes were proposed in this pull request?
1,create a libsvm-type dataset for lda: `data/mllib/sample_lda_libsvm_data.txt`
2,add python example
3,directly read
Repository: spark
Updated Branches:
refs/heads/branch-2.0 1753f6502 -> 3bd7a89bd
[SPARK-15150][EXAMPLE][DOC] Update LDA examples
## What changes were proposed in this pull request?
1,create a libsvm-type dataset for lda: `data/mllib/sample_lda_libsvm_data.txt`
2,add python example
3,directly
Repository: spark
Updated Branches:
refs/heads/branch-2.0 1e7d8ba5d -> 1753f6502
[SPARK-15238] Clarify supported Python versions
This PR:
* Clarifies that Spark *does* support Python 3, starting with Python 3.4.
Author: Nicholas Chammas
Closes #13017 from
Repository: spark
Updated Branches:
refs/heads/master 33597810e -> fafc95af7
[SPARK-15238] Clarify supported Python versions
This PR:
* Clarifies that Spark *does* support Python 3, starting with Python 3.4.
Author: Nicholas Chammas
Closes #13017 from
Repository: spark
Updated Branches:
refs/heads/master 8beae5914 -> 33597810e
[SPARK-14976][STREAMING] make StreamingContext.textFileStream support wildcard
## What changes were proposed in this pull request?
make StreamingContext.textFileStream support wildcard
like /home/user/*/file
## How
Repository: spark
Updated Branches:
refs/heads/branch-2.0 36f711dc6 -> 1e7d8ba5d
[SPARK-14976][STREAMING] make StreamingContext.textFileStream support wildcard
## What changes were proposed in this pull request?
make StreamingContext.textFileStream support wildcard
like /home/user/*/file
##
Repository: spark
Updated Branches:
refs/heads/branch-1.6 1678bff7f -> d1654864a
[SPARK-14495][SQL][1.6] fix resolution failure of having clause with distinct
aggregate function
Symptom:
In the latest **branch 1.6**, when a `DISTINCT` aggregation function is used in
the `HAVING`
Repository: spark
Updated Branches:
refs/heads/branch-2.0 73dd88939 -> 36f711dc6
[SPARK-15149][EXAMPLE][DOC] update kmeans example
## What changes were proposed in this pull request?
Python example for ml.kmeans already exists, but not included in user guide.
1,small changes like:
Repository: spark
Updated Branches:
refs/heads/master cef73b563 -> 8beae5914
[SPARK-15149][EXAMPLE][DOC] update kmeans example
## What changes were proposed in this pull request?
Python example for ml.kmeans already exists, but not included in user guide.
1,small changes like: `example_on`
Repository: spark
Updated Branches:
refs/heads/branch-2.0 bee2ddb39 -> 73dd88939
[SPARK-14340][EXAMPLE][DOC] Update Examples and User Guide for
ml.BisectingKMeans
## What changes were proposed in this pull request?
1, add BisectingKMeans to ml-clustering.md
2, add the missing Scala
Repository: spark
Updated Branches:
refs/heads/master 875ef7642 -> ad1a8466e
[SPARK-15141][EXAMPLE][DOC] Update OneVsRest Examples
## What changes were proposed in this pull request?
1, Add python example for OneVsRest
2, remove args-parsing
## How was this patch tested?
manual tests
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2d3c69a02 -> bee2ddb39
[SPARK-15141][EXAMPLE][DOC] Update OneVsRest Examples
## What changes were proposed in this pull request?
1, Add python example for OneVsRest
2, remove args-parsing
## How was this patch tested?
manual tests
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a8637f4ac -> 2d3c69a02
[SPARK-15231][SQL] Document the semantic of saveAsTable and insertInto and
don't drop columns silently
## What changes were proposed in this pull request?
This PR adds documents about the different behaviors
Repository: spark
Updated Branches:
refs/heads/master 007882c7e -> 875ef7642
[SPARK-15231][SQL] Document the semantic of saveAsTable and insertInto and
don't drop columns silently
## What changes were proposed in this pull request?
This PR adds documents about the different behaviors
Repository: spark
Updated Branches:
refs/heads/branch-2.0 ca5ce5365 -> a8637f4ac
[SPARK-15189][PYSPARK][DOCS] Update ml.evaluation PyDoc
## What changes were proposed in this pull request?
Fix doctest issue, short param description, and tag items as Experimental
## How was this patch
Repository: spark
Updated Branches:
refs/heads/master ba181c0c7 -> 007882c7e
[SPARK-15189][PYSPARK][DOCS] Update ml.evaluation PyDoc
## What changes were proposed in this pull request?
Fix doctest issue, short param description, and tag items as Experimental
## How was this patch tested?
68 matches
Mail list logo