alamb commented on issue #15028: URL: https://github.com/apache/datafusion/issues/15028#issuecomment-2703977750
BTW this amazing error message that shows what is actually consuming the memory is thanks to @wiedld 's efforts - #11523 > ``` > Error: Resources exhausted: Additional allocation failed with top memory consumers (across reservations) as: > ParquetSink(ArrowColumnWriter) consumed 65911731 bytes (62.8 MB), > ExternalSorter[0] consumed 23587552 bytes (22.4 MB), > ExternalSorterMerge[0] consumed 14261080 bytes (13.6 MB), > ParquetSink(SerializedFileWriter) consumed 0 bytes. > Error: Failed to allocate additional 1450451 bytes for ParquetSink(ArrowColumnWriter) with 62770337 bytes already allocated for this reservation - 1097237 bytes remain available for the total pool > ``` > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
