> On Aug. 6, 2012, 12:17 p.m., Sebastian Trueg wrote: > > Apart from the little superfluous return the patch is fine. I did not read > > it properly before. Who would have through that storing dir iterators > > instead of by arrays would make sure a difference? > > Vishesh Handa wrote: > It's not just iterators vs array. > > Here is an example - Say I'm adding watches for a director "DirA" which > has 50,000 subdirectories. The moment addWatchRecursively would enter DirA, > it would store those 50,000 directory urls in the queue. Then it would > process the queue one at a time, during the processing it would again keep > adding directories to the queue. This results in a a large memory requirement. > > In contrast, only one QDirIterator exists. It iterates over all the > subdirectories one at a time, without saving any to a queue/list. > > Do you see what I mean?
Sure, of course. - Sebastian ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://git.reviewboard.kde.org/r/105892/#review16963 ----------------------------------------------------------- On Aug. 6, 2012, 10:03 a.m., Vishesh Handa wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://git.reviewboard.kde.org/r/105892/ > ----------------------------------------------------------- > > (Updated Aug. 6, 2012, 10:03 a.m.) > > > Review request for Nepomuk and Sebastian Trueg. > > > Description > ------- > > kinotify: Do not store the paths to be added > > kinotify used a QQueue<QByteArray> to store all the directories which > need to be added. Since we use QDirIterator, each path is presented as a > QString which is then encoded to its QByteArray (QFile::encodeName). > This results in *large* chunks of memory being allocated, and then > slowly being deallocated. > > Instead, we now use a QDirIterator, and do not store all the directories > which need to be added, we simply iterate over them. This way we do not > allocate large amounts of memory. > > There is a large performance improvement as well. On my system, with > 38829 directories, adding all the watches now takes only about 10 seconds, > instead of about 65. > > Patch possible due to massif output provided by Jure Repinc > <[email protected]>. > Thanks a lot. > > tldr: Use DFS instead of BFS -> Less memory consumption > > BUG: 304476 > > > This addresses bug 304476. > http://bugs.kde.org/show_bug.cgi?id=304476 > > > Diffs > ----- > > services/filewatch/kinotify.cpp e8843c8 > > Diff: http://git.reviewboard.kde.org/r/105892/diff/ > > > Testing > ------- > > kinotify tests pass successfully. > > > Thanks, > > Vishesh Handa > >
_______________________________________________ Nepomuk mailing list [email protected] https://mail.kde.org/mailman/listinfo/nepomuk
