Hi, Ignite performs delete in a "soft" fashion: 1) When "remove" command is executed, we propagate it to the secondary file system; 2) For IGFS meta cache, we do not remove all records immediately, but rather execute a single "move" operation and move removed tree to a hidden "TRASH" folder. 3) "TRASH" folder is cleared periodically - this is what you see in logs.
Removal of trash content should not interfere with normal operations anyhow, nor should it cause any performance issues. Do you observe some real slowdown, or you are only concerned with metrics summary? Vladimir. On Thu, Jun 1, 2017 at 8:39 PM, joewang <m...@joewang.net> wrote: > The reads are from a non-IGFS source, but the writes are through IGFS. > Spark > uses Hadoop's FileOutputCommitter to write the output to IGFS. I think what > happens is essentially: > > - During processing, temporary files are written by each of n executors > running on different nodes to some /data/path/output/_temporary/part-n... > - When the job completes, each of the executor performs the final "commit" > by renaming the files under /data/path/output/_temporary/part-n... to > /data/path/output/part-n... and deletes the _temporary directory. > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/igfs-meta-behavior-when-node-restarts- > tp13155p13322.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
