Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14531
In Hive, `CREATE TABLE LIKE` only copies from the original.source table the
table properties that are specified in the Hive configuration:
`hive.ddl.createtablelike.properties.whitelist`. The
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76554157
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -794,8 +794,10 @@ case class ShowCreateTableCommand(table:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/14388
ping @maver1ck
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/13775
ping @yhuai @liancheng @hvanhovell @cloud-fan Can you take a look at this?
Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/14780
@hvanhovell @cloud-fan Can you help review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/14784#discussion_r76553829
--- Diff: R/pkg/R/sparkR.R ---
@@ -365,6 +365,10 @@ sparkR.session <- function(
}
overrideEnvs(sparkConfigMap, paramMap)
}
+ if
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14835
@QQshu1 I'm not sure what problem you're having but this pr is to solve the
specific problem that the call to get the application list json when loading
the history server takes too long when the
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/14712
yeah I think we should focus in the top priority target in this pr. And the
hive related translation can be addressed in later prs.
---
If your project is set up for it, you can reply to this email
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14856#discussion_r76553172
--- Diff: R/pkg/R/mllib.R ---
@@ -171,7 +172,8 @@ predict_internal <- function(object, newData) {
#' @note spark.glm since 2.0.0
#' @seealso
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/14855#discussion_r76552894
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -794,8 +794,10 @@ case class ShowCreateTableCommand(table:
Github user keypointt commented on a diff in the pull request:
https://github.com/apache/spark/pull/14856#discussion_r76552881
--- Diff: R/pkg/R/mllib.R ---
@@ -171,7 +172,8 @@ predict_internal <- function(object, newData) {
#' @note spark.glm since 2.0.0
#' @seealso
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14856#discussion_r76552745
--- Diff: R/pkg/R/mllib.R ---
@@ -171,7 +172,8 @@ predict_internal <- function(object, newData) {
#' @note spark.glm since 2.0.0
#' @seealso
GitHub user zjffdu opened a pull request:
https://github.com/apache/spark/pull/14857
[SPARK-17261][PYSPARK] Using HiveContext after re-creating SparkContext in
Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a
stopped sparkContext"
## What changes were
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14298#discussion_r76548403
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/PercentileApprox.scala
---
@@ -0,0 +1,462 @@
+/*
+ *
Github user qualiu commented on the issue:
https://github.com/apache/spark/pull/14807
@tritab : Thanks for your reply! Yes, I had tried that last week but not
forgot to put them on the context.
(1) Has space in `spark-submit.cmd` full path.
(2) Cut off quoted argument no
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76545213
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
---
@@ -207,9 +207,11 @@ class
Github user tritab commented on the issue:
https://github.com/apache/spark/pull/14807
Does it work if you escape the internal jdbc quotes with a caret ^ ?
On Aug 28, 2016 8:33 PM, "Quanmao LIU" wrote:
> @tsudukim
Github user qualiu commented on the issue:
https://github.com/apache/spark/pull/14807
@tsudukim : I did the validation mentioned above and snap the picture as
following:
Github user keypointt commented on a diff in the pull request:
https://github.com/apache/spark/pull/13584#discussion_r76543182
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/RFormulaSuite.scala ---
@@ -54,9 +54,6 @@ class RFormulaSuite extends SparkFunSuite with
Github user keypointt commented on a diff in the pull request:
https://github.com/apache/spark/pull/13584#discussion_r76543133
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/r/RWrapperUtilsSuite.scala ---
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software
GitHub user keypointt opened a pull request:
https://github.com/apache/spark/pull/14856
[SPARK-17241][SparkR][MLlib] SparkR spark.glm should have configurable
regularization parameter
https://issues.apache.org/jira/browse/SPARK-17241
## What changes were proposed in this
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/14855
[SPARK-17284] [SQL] Remove Statistics-related Table Properties from SHOW
CREATE TABLE
### What changes were proposed in this pull request?
The statistics-related table properties should be
Github user krishnakalyan3 commented on the issue:
https://github.com/apache/spark/pull/14741
@shivaram thanks for the advice.
Some Issue being faced by me
- While reading a large file from Rstudio and trying to kill the the
process using `Sys.getpid()`, I tried to
Github user eyalfa commented on the issue:
https://github.com/apache/spark/pull/1
@hvanhovell, after a quick 'find references' on CreateStruct it seems that
there are many places whre its constructor being used, but only numerous places
where it's being pattern-matched.
I'm
Github user nblintao commented on the issue:
https://github.com/apache/spark/pull/14204
test this, please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14854#discussion_r76537634
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -148,6 +148,12 @@ object SQLConf {
.booleanConf
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14854#discussion_r76537638
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala ---
@@ -299,9 +299,17 @@ abstract class SparkPlan extends
GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/14854
[SPARK-17283][WIP][Core] Cancel job in RDD.take() as soon as enough output
is received
## What changes were proposed in this pull request?
This patch introduces a new internal
Github user eyalfa commented on the issue:
https://github.com/apache/spark/pull/1
@hvanhovell , if you examine the diffs in this pr, you'll see that
ExpressionEncoder
[uses](https://github.com/apache/spark/pull/1/files#diff-91c617f2464cea010922328f4cdbbda9R136)
CreateStruct
Github user xwu0226 commented on the issue:
https://github.com/apache/spark/pull/14842
I see what you mean. Let me try your approach. Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14841
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14841
LGTM - merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76535329
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -36,11 +40,19 @@ case object Descending extends
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14842
Ok, so ASCENDING and DESCENDING have different NULLS behavior:
- ASCENDING: NULLS FIRST
- DESCENDING: NULLS LAST
I really like to avoid `nulls` being passed around and I also like
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14618
@cloud-fan Let me separate the cleanup task to multiple smaller PRs. We can
decide which ones can be merged at first. Thanks!
---
If your project is set up for it, you can reply to this email
Github user xwu0226 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76534922
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2661,4 +2661,186 @@ class SQLQuerySuite extends QueryTest with
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14712
@wzhfy Yeah, it sounds good to me to split the whole problem into multiple
PRs.
@hvanhovell Sure, let me create the JIRA and I can work on this when the
other dependent JIRAs are
Github user xwu0226 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76534903
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -36,11 +40,19 @@ case object Descending extends
Github user xwu0226 commented on the issue:
https://github.com/apache/spark/pull/14842
@hvanhovell Thank you so much for reviewing and providing the suggestions.
I will separate this into 2 PRs.
For the question of why we need 3 types of NULL ordering, I was thinking
that the
Github user xwu0226 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76534728
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/unsafe/sort/RadixSortSuite.scala
---
@@ -52,6 +52,7 @@ class RadixSortSuite extends
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/14712
I like @hvanhovell 's proposal, since providing a perfect Hive translation
layer is not trivial based on @gatorsmile 's investigation - we need to deal
with different versions of Hive. It is better
Github user tsudukim commented on the issue:
https://github.com/apache/spark/pull/14807
Hi @qualiu, I had a quick look.
I believe spark-submit.cmd that contains space in its path worked fine when
#10789 is merged, so I wonder if it is the problem of `cmd /V /E /C`.
---
If your
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/14619
@cloud-fan I've moved the `InsertRelationScanner` rule to `Analyzer`, after
relations and expressions are resolved. To reuse analyze and optimize rules, I
updated relative rules such as
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76533896
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -401,6 +401,13 @@ private[spark] class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76533832
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
---
@@ -207,9 +207,11 @@ class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76533586
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala
---
@@ -88,14 +87,66 @@ case class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76533558
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala
---
@@ -88,14 +87,66 @@ case class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14712#discussion_r76533400
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala
---
@@ -88,14 +89,70 @@ case class
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14840
Yes, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user HyukjinKwon closed the pull request at:
https://github.com/apache/spark/pull/14840
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14712
@gatorsmile lets also create another ticket for the explicit statistics
updates. I do like the `alter table s update statistics set ...` option.
---
If your project is set up for it, you can
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14712
How about we use our own property names for now, and provide a Hive
translation layer in a different PR. IMO it is fine to break a little bit of
the behavior in master as long as we fix it (or
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14840
@HyukjinKwon could you close this PR (the merge script cannot do this).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14840
LGTM. Merging to 2.0. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532925
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2661,4 +2661,186 @@ class SQLQuerySuite extends QueryTest with
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532909
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2661,4 +2661,186 @@ class SQLQuerySuite extends QueryTest with
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532670
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -58,7 +70,8 @@ case class SortOrder(child:
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532654
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -36,11 +40,19 @@ case object Descending extends
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532409
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -36,11 +40,19 @@ case object Descending extends
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532391
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala
---
@@ -28,6 +28,10 @@ abstract sealed class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/14842#discussion_r76532386
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/unsafe/sort/RadixSortSuite.scala
---
@@ -52,6 +52,7 @@ class RadixSortSuite extends
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/14842
@xwu0226 I glanced over it and this looks like the right approach.
One a high level I would break this up in two separate PRs: one to add
`NULLS FIRST`/`NULLS LAST` sorting to the unsafe
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/1
@eyalfa I would favor removing the `Create*Struct` classes altogether.
Could you elaborate more on how this connects to `Encoders`?
@cloud-fan what is your take on this?
---
If your
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/9571
Test failures timeout related; unlikely to be due to this patch
```
Test Result (2 failures / +2)
org.apache.spark.sql.hive.HiveSparkSubmitSuite.dir
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #64551 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64551/consoleFull)**
for PR 14691 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14746
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64549/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14746
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14746
**[Test build #64549 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64549/consoleFull)**
for PR 14746 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14746
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64547/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14746
**[Test build #64547 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64547/consoleFull)**
for PR 14746 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14746
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14853
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64550/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14853
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14853
**[Test build #64550 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64550/consoleFull)**
for PR 14853 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14531
After reading Hive source codes, the six statistic-related table properties
should be excluded: `numFiles`, `numPartitions`, `totalSize`, `numRows`,
`rawDataSize` and `COLUMN_STATS_ACCURATE`
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14853
**[Test build #64550 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64550/consoleFull)**
for PR 14853 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14746
**[Test build #64549 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64549/consoleFull)**
for PR 14746 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14853
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64548/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14853
**[Test build #64548 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64548/consoleFull)**
for PR 14853 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14853
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14853
**[Test build #64548 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64548/consoleFull)**
for PR 14853 at commit
GitHub user junyangq opened a pull request:
https://github.com/apache/spark/pull/14853
[SparkR][Minor] Fix LDA doc
## What changes were proposed in this pull request?
This PR tries to fix the name of the `SparkDataFrame` used in the example.
Also, it gives a reference url
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14746
**[Test build #64547 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64547/consoleFull)**
for PR 14746 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14801
cc @rxin @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14719
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64546/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14719
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14719
**[Test build #64546 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64546/consoleFull)**
for PR 14719 at commit
87 matches
Mail list logo