Hi Steve, thanks for the suggestion in this email thread and the SPIP doc! I will read the Audit Log and seek your feedback through PR reviews during the implementation process.
> So worrying about how pass and manage that at the thread level matters. We can have a specific logger for org.apache.spark and only show specific keys from log context (MDC). > The files get really big fast. I'd recommend considering Avro as an option from the outset. Agree, I have mentioned how to address this issue in section Q6. What are the risks? <https://docs.google.com/document/d/1rATVGmFLNVLmtxSpWrEceYm7d-ocgu8ofhryVs4g3XU/edit#bookmark=id.8zbyavz648i6> Thanks, Gengliang On Mon, Mar 11, 2024 at 9:30 AM huaxin gao <huaxin.ga...@gmail.com> wrote: > +1 > > On Mon, Mar 11, 2024 at 7:02 AM Wenchen Fan <cloud0...@gmail.com> wrote: > >> +1 >> >> On Mon, Mar 11, 2024 at 5:26 PM Hyukjin Kwon <gurwls...@apache.org> >> wrote: >> >>> +1 >>> >>> On Mon, 11 Mar 2024 at 18:11, yangjie01 <yangji...@baidu.com.invalid> >>> wrote: >>> >>>> +1 >>>> >>>> >>>> >>>> Jie Yang >>>> >>>> >>>> >>>> *发件人**: *Haejoon Lee <haejoon....@databricks.com.INVALID> >>>> *日期**: *2024年3月11日 星期一 17:09 >>>> *收件人**: *Gengliang Wang <ltn...@gmail.com> >>>> *抄送**: *dev <dev@spark.apache.org> >>>> *主题**: *Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark >>>> >>>> >>>> >>>> +1 >>>> >>>> >>>> >>>> On Mon, Mar 11, 2024 at 10:36 AM Gengliang Wang <ltn...@gmail.com> >>>> wrote: >>>> >>>> Hi all, >>>> >>>> I'd like to start the vote for SPIP: Structured Logging Framework for >>>> Apache Spark >>>> >>>> >>>> References: >>>> >>>> - JIRA ticket >>>> >>>> <https://mailshield.baidu.com/check?q=godVZoGJGzagfL5fHFKDXe8FOsAuf3UaY0E7uyGx6HVUGGWsmD%2fgOW2x6J1A1XYt8pai0Y8FBhY%3d> >>>> - SPIP doc >>>> >>>> <https://mailshield.baidu.com/check?q=qnzij19o7FucfHJ%2f4C2cBnMVM2kxjtEi9Gv4zA05b3oPw5UX986BZOwzaJ30UdGRMv%2fix31TYpjtazJC5uyypG0pZVBCfSjQGqlzkUoZozkFtgMXfpmRMSSp1%2bq83gkbLyrm1g%3d%3d> >>>> - Discussion thread >>>> >>>> <https://mailshield.baidu.com/check?q=6PGfLtMnDpsSvIF5SlbpQ4%2bwdg53GCedx5r%2b7AOnYMjYwomNs%2fBioZOabP9Ml3b%2bE8jzqXF0xR3j607DdbjV0JOnlvU%3d> >>>> >>>> Please vote on the SPIP for the next 72 hours: >>>> >>>> [ ] +1: Accept the proposal as an official SPIP >>>> [ ] +0 >>>> [ ] -1: I don’t think this is a good idea because … >>>> >>>> Thanks! >>>> >>>> Gengliang Wang >>>> >>>>
