Github user janplus commented on the issue:
https://github.com/apache/spark/pull/18376
Hi @srowen, @jiangxb1987. Thanks for quick response.
After further debug and consideration, this fix should be done in the
`spark-solr` project. Since this is a Solr specific issue.
I'll
Github user janplus closed the pull request at:
https://github.com/apache/spark/pull/18376
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user janplus opened a pull request:
https://github.com/apache/spark/pull/18376
[SPARK-21161][CORE] Make the parseHostPort method tolerate non digital
charactors
## What changes were proposed in this pull request?
Fix the following port parse bug when querying Solr
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
Thanks @rxin @dongjoon-hyun @cloud-fan @liancheng
I've learnt a lot from this PR!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
It seems failed the
`org.apache.spark.sql.sources.CreateTableAsSelectSuite.create a table, drop it
and create another one with the same name` test which is irrelevant with this
PR.
Maybe we
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
cc @cloud-fan Thank you.
I have resolved conflicts with master and done some code style fixes as you
suggested.
---
If your project is set up for it, you can reply to this email and have your
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70037559
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,154 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70033835
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
cc @rxin @cloud-fan Thank you for review
I add a new commit doing the following things:
1. Use ExpectsInputTypes instead of ImplicitCastInputTypes.
2. Add some cases for invalid-type
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70028685
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70028094
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70028081
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70027444
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70026365
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70026344
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70021325
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,152 @@ case class StringRPad(str
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
cc @cloud-fan @rxin @liancheng
I did optimization for Literal `part`, so we don't need to check for every
row. But since we may not assume in all circumstances the `part` is Literal, I
keep
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r70018330
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69933267
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69932808
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69932567
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69905666
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69894466
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69857849
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69857153
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69855494
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,145 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69855302
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69851574
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,52 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69851306
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69851163
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69851065
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,160 @@ case class StringRPad(str
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
Resolve conflicts with master.
cc @cloud-fan , is there any more question about this pull request?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69532511
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69532282
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69532226
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69477433
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69476414
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69475224
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +654,163 @@ case class StringRPad(str
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
@cloud-fan Thank you for review.
I did some code style fixes as you suggested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69408201
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69408152
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69408137
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
@dongjoon-hyun @cloud-fan It is nice to have you review my PR. Thank you!
I have add a new commit with following things:
1. Revert driver side's literal key invalidation.
2. Resolve
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69401574
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
@dongjoon-hyun Thank you for review
I have add a new commit which does following things:
1. Implement **Literal** `key` validation in `Driver` side.
2. Correct some failure message
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69386958
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69386564
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69386305
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69386280
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69386079
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,163 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69385849
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69385193
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69384848
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,51 @@ class
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
@dongjoon-hyun and @cloud-fan Thank you for review.
I have add a new commit which does following things:
1. Cache the url and the key pattern use the similar approach of
`XPathBoolean
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69371935
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,129 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69371498
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,129 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69371479
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -652,6 +656,129 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69370542
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69323801
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69314031
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +657,125 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69307883
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69307503
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +657,125 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69307232
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +657,125 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69307187
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +657,125 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69307094
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +657,125 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69297640
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
@rxin and @dongjoon-hyun Thanks for your review.
I have add a new commit which does following things:
1. Put `parse_url` function in the right order.
2. Use `""&quo
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69276506
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -725,4 +725,43 @@ class
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69276469
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69264274
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69264025
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69263461
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69263441
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69251673
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -653,6 +655,128 @@ case class StringRPad(str
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14008#discussion_r69251415
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -285,6 +285,7 @@ object FunctionRegistry
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/14008
cc @rxin and @cloud-fan
Improvements for performance concern
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user janplus opened a pull request:
https://github.com/apache/spark/pull/14008
[SPARK-16281][SQL] Implement parse_url SQL function
## What changes were proposed in this pull request?
This PR adds parse_url SQL functions in order to remove Hive fallback
Github user janplus closed the pull request at:
https://github.com/apache/spark/pull/13999
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/13999
This implementation is not that good. I'll close this one while create a
new pull request.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13966#discussion_r69237086
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -162,6 +163,46 @@ case class ConcatWs
Github user janplus commented on the issue:
https://github.com/apache/spark/pull/13999
cc @rxin and @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user janplus opened a pull request:
https://github.com/apache/spark/pull/13999
[SPARK-16281][SQL] Implement parse_url SQL function
## What changes were proposed in this pull request?
This PR adds `parse_url` SQL functions in order to remove Hive fallback
Github user janplus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13966#discussion_r69098161
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -162,6 +163,46 @@ case class ConcatWs
83 matches
Mail list logo
