[spark] branch master updated (6051755 -> 2e3aa2f)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double add 2e3aa2f [SPARK-32861][SQL] GenerateExec should require column ordering No new revisions were added by this update. Summary of changes: .../sql/execution/RemoveRedundantProjects.scala| 4 +- .../execution/RemoveRedundantProjectsSuite.scala | 54 +++--- 2 files changed, 51 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6051755 -> 2e3aa2f)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double add 2e3aa2f [SPARK-32861][SQL] GenerateExec should require column ordering No new revisions were added by this update. Summary of changes: .../sql/execution/RemoveRedundantProjects.scala| 4 +- .../execution/RemoveRedundantProjectsSuite.scala | 54 +++--- 2 files changed, 51 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6051755 -> 2e3aa2f)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double add 2e3aa2f [SPARK-32861][SQL] GenerateExec should require column ordering No new revisions were added by this update. Summary of changes: .../sql/execution/RemoveRedundantProjects.scala| 4 +- .../execution/RemoveRedundantProjectsSuite.scala | 54 +++--- 2 files changed, 51 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6051755 -> 2e3aa2f)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double add 2e3aa2f [SPARK-32861][SQL] GenerateExec should require column ordering No new revisions were added by this update. Summary of changes: .../sql/execution/RemoveRedundantProjects.scala| 4 +- .../execution/RemoveRedundantProjectsSuite.scala | 54 +++--- 2 files changed, 51 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6051755 -> 2e3aa2f)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double add 2e3aa2f [SPARK-32861][SQL] GenerateExec should require column ordering No new revisions were added by this update. Summary of changes: .../sql/execution/RemoveRedundantProjects.scala| 4 +- .../execution/RemoveRedundantProjectsSuite.scala | 54 +++--- 2 files changed, 51 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-3.0 updated: [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cb6a0d0 [SPARK-32688][SQL][TEST] Add special values to
LiteralGenerator for float and double
cb6a0d0 is described below
commit cb6a0d08cc020d9a2c19173c9023a9f5e565dd6c
Author: Tanel Kiis
AuthorDate: Wed Sep 16 12:13:15 2020 +0900
[SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float
and double
### What changes were proposed in this pull request?
The `LiteralGenerator` for float and double datatypes was supposed to yield
special values (NaN, +-inf) among others, but the `Gen.chooseNum` method does
not yield values that are outside the defined range. The `Gen.chooseNum` for a
wide range of floats and doubles does not yield values in the "everyday" range
as stated in https://github.com/typelevel/scalacheck/issues/113 .
There is an similar class `RandomDataGenerator` that is used in some other
tests. Added `-0.0` and `-0.0f` as special values to there too.
These changes revealed an inconsistency with the equality check between
`-0.0` and `0.0`.
### Why are the changes needed?
The `LiteralGenerator` is mostly used in the
`checkConsistencyBetweenInterpretedAndCodegen` method in
`MathExpressionsSuite`. This change would have caught the bug fixed in #29495 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally reverted #29495 and verified that the existing test cases caught
the bug.
Closes #29515 from tanelk/SPARK-32688.
Authored-by: Tanel Kiis
Signed-off-by: Takeshi Yamamuro
(cherry picked from commit 6051755bfe23a0e4564bf19476ec34cd7fd6008d)
Signed-off-by: Takeshi Yamamuro
---
.../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++--
.../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
index 6a5bdc4..3e2dc3f 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
@@ -260,10 +260,10 @@ object RandomDataGenerator {
new MathContext(precision)).bigDecimal)
case DoubleType => randomNumeric[Double](
rand, r => longBitsToDouble(r.nextLong()), Seq(Double.MinValue,
Double.MinPositiveValue,
- Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0))
+ Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0, -0.0))
case FloatType => randomNumeric[Float](
rand, r => intBitsToFloat(r.nextInt()), Seq(Float.MinValue,
Float.MinPositiveValue,
- Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f))
+ Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f, -0.0f))
case ByteType => randomNumeric[Byte](
rand, _.nextInt().toByte, Seq(Byte.MinValue, Byte.MaxValue, 0.toByte))
case IntegerType => randomNumeric[Int](
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
index d92eb01..c8e3b0e 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
@@ -68,16 +68,27 @@ object LiteralGenerator {
lazy val longLiteralGen: Gen[Literal] =
for { l <- Arbitrary.arbLong.arbitrary } yield Literal.create(l, LongType)
+ // The floatLiteralGen and doubleLiteralGen will 50% of the time yield
arbitrary values
+ // and 50% of the time will yield some special values that are more likely
to reveal
+ // corner cases. This behavior is similar to the integral value generators.
lazy val floatLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Float.MinValue / 2, Float.MaxValue / 2,
-Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity)
+ f <- Gen.oneOf(
+Gen.oneOf(
+ Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity,
Float.MinPositiveValue,
+ Float.MaxValue, -Float.MaxValue, 0.0f, -0.0f, 1.0f, -1.0f),
+Arbitrary.arbFloat.arbitrary
+ )
} yield Literal.create(f, FloatType)
lazy val doubleLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Double.MinValue / 2, Double.MaxValue / 2,
-Double.NaN, Double.PositiveInfinity, D
[spark] branch branch-3.0 updated: [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cb6a0d0 [SPARK-32688][SQL][TEST] Add special values to
LiteralGenerator for float and double
cb6a0d0 is described below
commit cb6a0d08cc020d9a2c19173c9023a9f5e565dd6c
Author: Tanel Kiis
AuthorDate: Wed Sep 16 12:13:15 2020 +0900
[SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float
and double
### What changes were proposed in this pull request?
The `LiteralGenerator` for float and double datatypes was supposed to yield
special values (NaN, +-inf) among others, but the `Gen.chooseNum` method does
not yield values that are outside the defined range. The `Gen.chooseNum` for a
wide range of floats and doubles does not yield values in the "everyday" range
as stated in https://github.com/typelevel/scalacheck/issues/113 .
There is an similar class `RandomDataGenerator` that is used in some other
tests. Added `-0.0` and `-0.0f` as special values to there too.
These changes revealed an inconsistency with the equality check between
`-0.0` and `0.0`.
### Why are the changes needed?
The `LiteralGenerator` is mostly used in the
`checkConsistencyBetweenInterpretedAndCodegen` method in
`MathExpressionsSuite`. This change would have caught the bug fixed in #29495 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally reverted #29495 and verified that the existing test cases caught
the bug.
Closes #29515 from tanelk/SPARK-32688.
Authored-by: Tanel Kiis
Signed-off-by: Takeshi Yamamuro
(cherry picked from commit 6051755bfe23a0e4564bf19476ec34cd7fd6008d)
Signed-off-by: Takeshi Yamamuro
---
.../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++--
.../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
index 6a5bdc4..3e2dc3f 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
@@ -260,10 +260,10 @@ object RandomDataGenerator {
new MathContext(precision)).bigDecimal)
case DoubleType => randomNumeric[Double](
rand, r => longBitsToDouble(r.nextLong()), Seq(Double.MinValue,
Double.MinPositiveValue,
- Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0))
+ Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0, -0.0))
case FloatType => randomNumeric[Float](
rand, r => intBitsToFloat(r.nextInt()), Seq(Float.MinValue,
Float.MinPositiveValue,
- Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f))
+ Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f, -0.0f))
case ByteType => randomNumeric[Byte](
rand, _.nextInt().toByte, Seq(Byte.MinValue, Byte.MaxValue, 0.toByte))
case IntegerType => randomNumeric[Int](
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
index d92eb01..c8e3b0e 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
@@ -68,16 +68,27 @@ object LiteralGenerator {
lazy val longLiteralGen: Gen[Literal] =
for { l <- Arbitrary.arbLong.arbitrary } yield Literal.create(l, LongType)
+ // The floatLiteralGen and doubleLiteralGen will 50% of the time yield
arbitrary values
+ // and 50% of the time will yield some special values that are more likely
to reveal
+ // corner cases. This behavior is similar to the integral value generators.
lazy val floatLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Float.MinValue / 2, Float.MaxValue / 2,
-Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity)
+ f <- Gen.oneOf(
+Gen.oneOf(
+ Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity,
Float.MinPositiveValue,
+ Float.MaxValue, -Float.MaxValue, 0.0f, -0.0f, 1.0f, -1.0f),
+Arbitrary.arbFloat.arbitrary
+ )
} yield Literal.create(f, FloatType)
lazy val doubleLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Double.MinValue / 2, Double.MaxValue / 2,
-Double.NaN, Double.PositiveInfinity, D
[spark] branch master updated (b46c730 -> 6051755)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test add 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++-- .../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++ 2 files changed, 17 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b46c730 -> 6051755)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test add 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++-- .../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++ 2 files changed, 17 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-3.0 updated: [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cb6a0d0 [SPARK-32688][SQL][TEST] Add special values to
LiteralGenerator for float and double
cb6a0d0 is described below
commit cb6a0d08cc020d9a2c19173c9023a9f5e565dd6c
Author: Tanel Kiis
AuthorDate: Wed Sep 16 12:13:15 2020 +0900
[SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float
and double
### What changes were proposed in this pull request?
The `LiteralGenerator` for float and double datatypes was supposed to yield
special values (NaN, +-inf) among others, but the `Gen.chooseNum` method does
not yield values that are outside the defined range. The `Gen.chooseNum` for a
wide range of floats and doubles does not yield values in the "everyday" range
as stated in https://github.com/typelevel/scalacheck/issues/113 .
There is an similar class `RandomDataGenerator` that is used in some other
tests. Added `-0.0` and `-0.0f` as special values to there too.
These changes revealed an inconsistency with the equality check between
`-0.0` and `0.0`.
### Why are the changes needed?
The `LiteralGenerator` is mostly used in the
`checkConsistencyBetweenInterpretedAndCodegen` method in
`MathExpressionsSuite`. This change would have caught the bug fixed in #29495 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally reverted #29495 and verified that the existing test cases caught
the bug.
Closes #29515 from tanelk/SPARK-32688.
Authored-by: Tanel Kiis
Signed-off-by: Takeshi Yamamuro
(cherry picked from commit 6051755bfe23a0e4564bf19476ec34cd7fd6008d)
Signed-off-by: Takeshi Yamamuro
---
.../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++--
.../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
index 6a5bdc4..3e2dc3f 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
@@ -260,10 +260,10 @@ object RandomDataGenerator {
new MathContext(precision)).bigDecimal)
case DoubleType => randomNumeric[Double](
rand, r => longBitsToDouble(r.nextLong()), Seq(Double.MinValue,
Double.MinPositiveValue,
- Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0))
+ Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0, -0.0))
case FloatType => randomNumeric[Float](
rand, r => intBitsToFloat(r.nextInt()), Seq(Float.MinValue,
Float.MinPositiveValue,
- Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f))
+ Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f, -0.0f))
case ByteType => randomNumeric[Byte](
rand, _.nextInt().toByte, Seq(Byte.MinValue, Byte.MaxValue, 0.toByte))
case IntegerType => randomNumeric[Int](
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
index d92eb01..c8e3b0e 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
@@ -68,16 +68,27 @@ object LiteralGenerator {
lazy val longLiteralGen: Gen[Literal] =
for { l <- Arbitrary.arbLong.arbitrary } yield Literal.create(l, LongType)
+ // The floatLiteralGen and doubleLiteralGen will 50% of the time yield
arbitrary values
+ // and 50% of the time will yield some special values that are more likely
to reveal
+ // corner cases. This behavior is similar to the integral value generators.
lazy val floatLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Float.MinValue / 2, Float.MaxValue / 2,
-Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity)
+ f <- Gen.oneOf(
+Gen.oneOf(
+ Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity,
Float.MinPositiveValue,
+ Float.MaxValue, -Float.MaxValue, 0.0f, -0.0f, 1.0f, -1.0f),
+Arbitrary.arbFloat.arbitrary
+ )
} yield Literal.create(f, FloatType)
lazy val doubleLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Double.MinValue / 2, Double.MaxValue / 2,
-Double.NaN, Double.PositiveInfinity, D
[spark] branch master updated (b46c730 -> 6051755)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test add 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++-- .../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++ 2 files changed, 17 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-3.0 updated: [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cb6a0d0 [SPARK-32688][SQL][TEST] Add special values to
LiteralGenerator for float and double
cb6a0d0 is described below
commit cb6a0d08cc020d9a2c19173c9023a9f5e565dd6c
Author: Tanel Kiis
AuthorDate: Wed Sep 16 12:13:15 2020 +0900
[SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float
and double
### What changes were proposed in this pull request?
The `LiteralGenerator` for float and double datatypes was supposed to yield
special values (NaN, +-inf) among others, but the `Gen.chooseNum` method does
not yield values that are outside the defined range. The `Gen.chooseNum` for a
wide range of floats and doubles does not yield values in the "everyday" range
as stated in https://github.com/typelevel/scalacheck/issues/113 .
There is an similar class `RandomDataGenerator` that is used in some other
tests. Added `-0.0` and `-0.0f` as special values to there too.
These changes revealed an inconsistency with the equality check between
`-0.0` and `0.0`.
### Why are the changes needed?
The `LiteralGenerator` is mostly used in the
`checkConsistencyBetweenInterpretedAndCodegen` method in
`MathExpressionsSuite`. This change would have caught the bug fixed in #29495 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally reverted #29495 and verified that the existing test cases caught
the bug.
Closes #29515 from tanelk/SPARK-32688.
Authored-by: Tanel Kiis
Signed-off-by: Takeshi Yamamuro
(cherry picked from commit 6051755bfe23a0e4564bf19476ec34cd7fd6008d)
Signed-off-by: Takeshi Yamamuro
---
.../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++--
.../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
index 6a5bdc4..3e2dc3f 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
@@ -260,10 +260,10 @@ object RandomDataGenerator {
new MathContext(precision)).bigDecimal)
case DoubleType => randomNumeric[Double](
rand, r => longBitsToDouble(r.nextLong()), Seq(Double.MinValue,
Double.MinPositiveValue,
- Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0))
+ Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0, -0.0))
case FloatType => randomNumeric[Float](
rand, r => intBitsToFloat(r.nextInt()), Seq(Float.MinValue,
Float.MinPositiveValue,
- Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f))
+ Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f, -0.0f))
case ByteType => randomNumeric[Byte](
rand, _.nextInt().toByte, Seq(Byte.MinValue, Byte.MaxValue, 0.toByte))
case IntegerType => randomNumeric[Int](
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
index d92eb01..c8e3b0e 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
@@ -68,16 +68,27 @@ object LiteralGenerator {
lazy val longLiteralGen: Gen[Literal] =
for { l <- Arbitrary.arbLong.arbitrary } yield Literal.create(l, LongType)
+ // The floatLiteralGen and doubleLiteralGen will 50% of the time yield
arbitrary values
+ // and 50% of the time will yield some special values that are more likely
to reveal
+ // corner cases. This behavior is similar to the integral value generators.
lazy val floatLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Float.MinValue / 2, Float.MaxValue / 2,
-Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity)
+ f <- Gen.oneOf(
+Gen.oneOf(
+ Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity,
Float.MinPositiveValue,
+ Float.MaxValue, -Float.MaxValue, 0.0f, -0.0f, 1.0f, -1.0f),
+Arbitrary.arbFloat.arbitrary
+ )
} yield Literal.create(f, FloatType)
lazy val doubleLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Double.MinValue / 2, Double.MaxValue / 2,
-Double.NaN, Double.PositiveInfinity, D
[spark] branch master updated (b46c730 -> 6051755)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test add 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++-- .../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++ 2 files changed, 17 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (108c4c8 -> b46c730)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created add b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-3.0 updated: [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double
This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cb6a0d0 [SPARK-32688][SQL][TEST] Add special values to
LiteralGenerator for float and double
cb6a0d0 is described below
commit cb6a0d08cc020d9a2c19173c9023a9f5e565dd6c
Author: Tanel Kiis
AuthorDate: Wed Sep 16 12:13:15 2020 +0900
[SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float
and double
### What changes were proposed in this pull request?
The `LiteralGenerator` for float and double datatypes was supposed to yield
special values (NaN, +-inf) among others, but the `Gen.chooseNum` method does
not yield values that are outside the defined range. The `Gen.chooseNum` for a
wide range of floats and doubles does not yield values in the "everyday" range
as stated in https://github.com/typelevel/scalacheck/issues/113 .
There is an similar class `RandomDataGenerator` that is used in some other
tests. Added `-0.0` and `-0.0f` as special values to there too.
These changes revealed an inconsistency with the equality check between
`-0.0` and `0.0`.
### Why are the changes needed?
The `LiteralGenerator` is mostly used in the
`checkConsistencyBetweenInterpretedAndCodegen` method in
`MathExpressionsSuite`. This change would have caught the bug fixed in #29495 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Locally reverted #29495 and verified that the existing test cases caught
the bug.
Closes #29515 from tanelk/SPARK-32688.
Authored-by: Tanel Kiis
Signed-off-by: Takeshi Yamamuro
(cherry picked from commit 6051755bfe23a0e4564bf19476ec34cd7fd6008d)
Signed-off-by: Takeshi Yamamuro
---
.../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++--
.../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
index 6a5bdc4..3e2dc3f 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
@@ -260,10 +260,10 @@ object RandomDataGenerator {
new MathContext(precision)).bigDecimal)
case DoubleType => randomNumeric[Double](
rand, r => longBitsToDouble(r.nextLong()), Seq(Double.MinValue,
Double.MinPositiveValue,
- Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0))
+ Double.MaxValue, Double.PositiveInfinity, Double.NegativeInfinity,
Double.NaN, 0.0, -0.0))
case FloatType => randomNumeric[Float](
rand, r => intBitsToFloat(r.nextInt()), Seq(Float.MinValue,
Float.MinPositiveValue,
- Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f))
+ Float.MaxValue, Float.PositiveInfinity, Float.NegativeInfinity,
Float.NaN, 0.0f, -0.0f))
case ByteType => randomNumeric[Byte](
rand, _.nextInt().toByte, Seq(Byte.MinValue, Byte.MaxValue, 0.toByte))
case IntegerType => randomNumeric[Int](
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
index d92eb01..c8e3b0e 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala
@@ -68,16 +68,27 @@ object LiteralGenerator {
lazy val longLiteralGen: Gen[Literal] =
for { l <- Arbitrary.arbLong.arbitrary } yield Literal.create(l, LongType)
+ // The floatLiteralGen and doubleLiteralGen will 50% of the time yield
arbitrary values
+ // and 50% of the time will yield some special values that are more likely
to reveal
+ // corner cases. This behavior is similar to the integral value generators.
lazy val floatLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Float.MinValue / 2, Float.MaxValue / 2,
-Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity)
+ f <- Gen.oneOf(
+Gen.oneOf(
+ Float.NaN, Float.PositiveInfinity, Float.NegativeInfinity,
Float.MinPositiveValue,
+ Float.MaxValue, -Float.MaxValue, 0.0f, -0.0f, 1.0f, -1.0f),
+Arbitrary.arbFloat.arbitrary
+ )
} yield Literal.create(f, FloatType)
lazy val doubleLiteralGen: Gen[Literal] =
for {
- f <- Gen.chooseNum(Double.MinValue / 2, Double.MaxValue / 2,
-Double.NaN, Double.PositiveInfinity, D
[spark] branch master updated (b46c730 -> 6051755)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test add 6051755 [SPARK-32688][SQL][TEST] Add special values to LiteralGenerator for float and double No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/RandomDataGenerator.scala| 4 ++-- .../sql/catalyst/expressions/LiteralGenerator.scala | 19 +++ 2 files changed, 17 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (108c4c8 -> b46c730)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created add b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (108c4c8 -> b46c730)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created add b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (108c4c8 -> b46c730)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created add b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (108c4c8 -> b46c730)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created add b46c730 [SPARK-32704][SQL][TESTS][FOLLOW-UP] Check any physical rule instead of a specific rule in the test No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/QueryExecutionSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888b343 -> 108c4c8)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config add 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888b343 -> 108c4c8)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config add 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888b343 -> 108c4c8)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config add 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888b343 -> 108c4c8)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config add 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888b343 -> 108c4c8)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config add 108c4c8 [SPARK-32481][SQL][TESTS][FOLLOW-UP] Skip the test if trash directory cannot be created No new revisions were added by this update. Summary of changes: .../test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6f36db1 -> 888b343)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py add 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 10 ++ .../spark/sql/execution/DataSourceScanExec.scala | 2 +- .../sql/execution/datasources/v2/FileScan.scala| 2 +- .../spark/sql/FileBasedDataSourceSuite.scala | 23 ++ 4 files changed, 35 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6f36db1 -> 888b343)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py add 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 10 ++ .../spark/sql/execution/DataSourceScanExec.scala | 2 +- .../sql/execution/datasources/v2/FileScan.scala| 2 +- .../spark/sql/FileBasedDataSourceSuite.scala | 23 ++ 4 files changed, 35 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6f36db1 -> 888b343)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py add 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 10 ++ .../spark/sql/execution/DataSourceScanExec.scala | 2 +- .../sql/execution/datasources/v2/FileScan.scala| 2 +- .../spark/sql/FileBasedDataSourceSuite.scala | 23 ++ 4 files changed, 35 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6f36db1 -> 888b343)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py add 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 10 ++ .../spark/sql/execution/DataSourceScanExec.scala | 2 +- .../sql/execution/datasources/v2/FileScan.scala| 2 +- .../spark/sql/FileBasedDataSourceSuite.scala | 23 ++ 4 files changed, 35 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (6f36db1 -> 888b343)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py add 888b343 [SPARK-32827][SQL] Add spark.sql.maxMetadataStringLength config No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 10 ++ .../spark/sql/execution/DataSourceScanExec.scala | 2 +- .../sql/execution/datasources/v2/FileScan.scala| 2 +- .../spark/sql/FileBasedDataSourceSuite.scala | 23 ++ 4 files changed, 35 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (316242b -> 6f36db1)
This is an automated email from the ASF dual-hosted git repository. srowen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server add 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py No new revisions were added by this update. Summary of changes: python/pyspark/sql/dataframe.py | 7 --- python/pyspark/storagelevel.py | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (316242b -> 6f36db1)
This is an automated email from the ASF dual-hosted git repository. srowen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server add 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py No new revisions were added by this update. Summary of changes: python/pyspark/sql/dataframe.py | 7 --- python/pyspark/storagelevel.py | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (316242b -> 6f36db1)
This is an automated email from the ASF dual-hosted git repository. srowen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server add 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py No new revisions were added by this update. Summary of changes: python/pyspark/sql/dataframe.py | 7 --- python/pyspark/storagelevel.py | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (316242b -> 6f36db1)
This is an automated email from the ASF dual-hosted git repository. srowen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server add 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py No new revisions were added by this update. Summary of changes: python/pyspark/sql/dataframe.py | 7 --- python/pyspark/storagelevel.py | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (316242b -> 6f36db1)
This is an automated email from the ASF dual-hosted git repository. srowen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server add 6f36db1 [SPARK-31448][PYTHON] Fix storage level used in persist() in dataframe.py No new revisions were added by this update. Summary of changes: python/pyspark/sql/dataframe.py | 7 --- python/pyspark/storagelevel.py | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (99384d1 -> 316242b)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 99384d1 [SPARK-32738][CORE] Should reduce the number of active threads if fatal error happens in `Inbox.process` add 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server No new revisions were added by this update. Summary of changes: .../SparkThriftServerProtocolVersionsSuite.scala | 147 - 1 file changed, 140 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (99384d1 -> 316242b)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 99384d1 [SPARK-32738][CORE] Should reduce the number of active threads if fatal error happens in `Inbox.process` add 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server No new revisions were added by this update. Summary of changes: .../SparkThriftServerProtocolVersionsSuite.scala | 147 - 1 file changed, 140 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (99384d1 -> 316242b)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 99384d1 [SPARK-32738][CORE] Should reduce the number of active threads if fatal error happens in `Inbox.process` add 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server No new revisions were added by this update. Summary of changes: .../SparkThriftServerProtocolVersionsSuite.scala | 147 - 1 file changed, 140 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (99384d1 -> 316242b)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 99384d1 [SPARK-32738][CORE] Should reduce the number of active threads if fatal error happens in `Inbox.process` add 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server No new revisions were added by this update. Summary of changes: .../SparkThriftServerProtocolVersionsSuite.scala | 147 - 1 file changed, 140 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (99384d1 -> 316242b)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 99384d1 [SPARK-32738][CORE] Should reduce the number of active threads if fatal error happens in `Inbox.process` add 316242b [SPARK-32874][SQL][TEST] Enhance result set meta data check for execute statement operation with thrift server No new revisions were added by this update. Summary of changes: .../SparkThriftServerProtocolVersionsSuite.scala | 147 - 1 file changed, 140 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
