Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17329#discussion_r108031525
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java
---
@@ -37,13 +37,24 @@
* A {@link ManagedBuffer} backed by a segment in a file.
*/
public final class FileSegmentManagedBuffer extends ManagedBuffer {
- private final TransportConf conf;
+ private final boolean lazyFileDescriptor;
+ private final int memoryMapBytes;
private final File file;
private final long offset;
private final long length;
public FileSegmentManagedBuffer(TransportConf conf, File file, long
offset, long length) {
- this.conf = conf;
+ this(conf.lazyFileDescriptor(), conf.memoryMapBytes(), file, offset,
length);
+ }
+
+ public FileSegmentManagedBuffer(
--- End diff --
I'm not sure what you're stating, that this is slow? but you also only made
half the changes as you did here. I don't see what the argument is. Removing
the constructor you added can't have an effect on performance either way. In
either case, you access the value once in the constructor.
I think all you're showing is that the first file changed here doesn't
matter. You can revert this entirely too. It's ExternalBlockShufflerManager
that matters.
Make one or the other change and this can proceed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
