itholic commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1519270151
##
python/pyspark/errors/utils.py:
##
@@ -15,12 +15,22 @@
# limitations under the License.
#
+import builtins
import re
-from typing import Dict, Match
+import fun
ueshin commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1518206202
##
python/pyspark/errors/utils.py:
##
@@ -15,12 +15,22 @@
# limitations under the License.
#
+import builtins
import re
-from typing import Dict, Match
+import func
itholic commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-191911
On second thought, we need to keep the previous transformation stacktrace to
provide more accurate context.
Will push more commit to update it.
--
This is an automated message fr
itholic commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1511983720
##
python/pyspark/errors/utils.py:
##
@@ -119,3 +127,73 @@ def get_message_template(self, error_class: str) -> str:
message_template = main_message_templat
itholic commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-1977737988
cc @HyukjinKwon FYI, I'm still working on Spark Connect support and unit
tests but the basic structure is ready for review.
FYI, also cc @MaxGekk as you made a similar contribution
itholic commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-1977735804
I'm still working on Spark Connect support and unit tests, but the basic
structure is ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the me
itholic opened a new pull request, #45377:
URL: https://github.com/apache/spark/pull/45377
### What changes were proposed in this pull request?
This PR introduces an enhancement to the error messages generated by
PySpark's DataFrame API, adding detailed context about the location with
