AjayBoddeda4 commented on PR #716: URL: https://github.com/apache/wayang/pull/716#issuecomment-4096414232
Hi Carlo, I studied the new commits carefully — this is excellent progress! I noticed you used Java Records for both Row and Schema which is exactly the direction I suggested on issue #514. The SparkSelectOperator using Dataset[Row] with functions::col is a clean implementation. Looking at SparkSelectOperator, I see getSupportedInputChannels and getSupportedOutputChannels return empty lists — would DatasetChannel descriptors be the right choice here to keep execution within the Dataset world and avoid RDD conversions? This connects to issue #362 about DataFrameChannel that I was studying. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
