Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
arzavj commented on PR #45301: URL: https://github.com/apache/spark/pull/45301#issuecomment-1977729895 @HyukjinKwon do you know when I can expect 3.5.2 to be released to be able to take advantage of this bug fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
HyukjinKwon commented on code in PR #45301: URL: https://github.com/apache/spark/pull/45301#discussion_r1505269608 ## python/pyspark/sql/pandas/types.py: ## @@ -993,7 +993,7 @@ def convert_struct(value: Any) -> Any: def convert_timestamp(value: Any) -> Any: if isinstance(value, datetime.datetime) and value.tzinfo is not None: -ts = pd.Timstamp(value) +ts = pd.Timestamp(value) Review Comment: I am fine as is because it's pretty obvious but would be good to have a test at `python/pyspark/sql/tests/test_arrow.py`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
dongjoon-hyun commented on code in PR #45301: URL: https://github.com/apache/spark/pull/45301#discussion_r1505276588 ## python/pyspark/sql/pandas/types.py: ## @@ -993,7 +993,7 @@ def convert_struct(value: Any) -> Any: def convert_timestamp(value: Any) -> Any: if isinstance(value, datetime.datetime) and value.tzinfo is not None: -ts = pd.Timstamp(value) +ts = pd.Timestamp(value) Review Comment: +1 for the above comment about adding a test coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
arzavj commented on code in PR #45301: URL: https://github.com/apache/spark/pull/45301#discussion_r1505353843 ## python/pyspark/sql/pandas/types.py: ## @@ -993,7 +993,7 @@ def convert_struct(value: Any) -> Any: def convert_timestamp(value: Any) -> Any: if isinstance(value, datetime.datetime) and value.tzinfo is not None: -ts = pd.Timstamp(value) +ts = pd.Timestamp(value) Review Comment: Yeah I considered adding a test for this but really what we need here is a linter in CI to catch such typos -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
HyukjinKwon commented on PR #45301: URL: https://github.com/apache/spark/pull/45301#issuecomment-1968241304 Let me just merge this in and follow up with a test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
HyukjinKwon closed pull request #45301: [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo URL: https://github.com/apache/spark/pull/45301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
HyukjinKwon commented on PR #45301: URL: https://github.com/apache/spark/pull/45301#issuecomment-1968241780 Merged to master and branch-3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47202][PYTHON] Fix typo breaking datetimes with tzinfo [spark]
HyukjinKwon commented on PR #45301: URL: https://github.com/apache/spark/pull/45301#issuecomment-1968321173 PTAL: https://github.com/apache/spark/pull/45308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
