Re: [PR] [SPARK-48129][PYSPARK] Provide a constant table schema in PySpark for querying structured logs [spark]
gengliangwang commented on code in PR #46384: URL: https://github.com/apache/spark/pull/46384#discussion_r1589921846 ## python/pyspark/util.py: ## @@ -107,6 +107,22 @@ def majorMinorVersion(sparkVersion: str) -> Tuple[int, int]: ) +class LogUtils: Review Comment: @HyukjinKwon Is there a better file location to put this constant string? There won't be a lot of new code in LogUtils. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[PR] [SPARK-48129][PYSPARK] Provide a constant table schema in PySpark for querying structured logs [spark]
gengliangwang opened a new pull request, #46384: URL: https://github.com/apache/spark/pull/46384 ### What changes were proposed in this pull request? Similar to https://github.com/apache/spark/pull/46375/, this PR provides a constant table schema in PySpark for querying structured logs. The doc of logging configuration is also updated. ### Why are the changes needed? Provide a convenient way to query Spark logs using PySpark. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Manual test ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
