bogao007 commented on PR #47804: URL: https://github.com/apache/spark/pull/47804#issuecomment-2299994195
> The logic seems fine but can we reuse some related code path? I believe we have a lot of classes that do the same thing @HyukjinKwon Yeah I included the reason adding a new class in the PR description ``` We added a new class TransformWithStateInPandasWriter rather than reusing the existing ApplyInPandasWithStateWriter because ApplyInPandasWithStateWriter needs to handle state fields together with data fields, the logic there is much more complicated than what we need for TransformWithState. Adding a new class would be much simpler. Also, there's not many existing code can be reused, so no point on adding more complexity to the existing ApplyInPandasWithStateWriter. ``` Do you think if that's valid or is there any other class you think we could reuse? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
