voidtools forum

I've a Windows 10 and have encountered a serious problem during indexing my ReFS partition (I don't know if this happens also with NTFS). I spotted a huge increase of memory usage while reindex is in progress. I diagnosed this problem and came to these conclusions:

• 1. It is not Everything's fault. It's Windows 10's dynamic cache used for file IO that gets bloated (link, link). In short, if there's a lot of quick/frequent IO demand, Windows caches it in RAM. The demand can be caused by, of course, Everything's full partition reindex, but also as simple as selecting all folders on your root partition and invoking Properties. Even if it seems like Windows problem, I still decided to report this here, because I hope Everything code can be upgraded to overcome this issue.

  

  2017.09.23 12.48 - Leak, Reindex Stop.png (39.15 KiB) Viewed 1022 times

  2. The memory section where the leak is happening is metafile section, which is somewhat invisible in Windows Task Manager: It won't show in any column in Processes nor Details tab, but it's a part of the total In use bytes charted on Performance tab. Because of that you can see a 100% memory usage, but if you sum the memory column on Processes tab, it won't match (by many GiB). The missing part is the metafile section. which you can see in Performance Monitor by creating a counter for System Cache Residual Bytes (not Process Private Bytes, where usual memory leaks happen). You can also (and easier) see it using RAMMap app from Windows Internals (link). The problem is clearly visible on the screenshot I attached:

  

  2017.09.23 12.59 - Leak, Before Emptying system working set.png (26.4 KiB) Viewed 1022 times

  3. When I force emptying of this section (Empty > Empty System working set in RAMMap), the memory is released (or rather, flagged as inactive, but Task Manager can see this RAM as now free):

  

  2017.09.23 12.01 - Leak, After Emptying system working set + Chart.png (61.11 KiB) Viewed 1022 times

My guesses are that ReFS partition or Windows 10's kernel are at fault. If you could diagnose this problem deeper and present a workaround in Everything, that would be great. More screenshots and logs for this issue are available at my Dropbox (couldn't attach more than 3 files to this post).

Thanks for dropbox screenshots and bringing this issue to my attention.

I'll look into a fix for Everything 1.5.

I must report a great discovery in this topic. Information from this Microsoft support page resolved the memory leak issue for me. The used RAM still increases during reindex, but now it is being freed (in batches, at random moments). They suggested three different keys to add to registry, I added only the first two (as they said the third is most aggressive and can lead to diminishing performance) and since then the problem is actually gone. I suspect even the first option might be enough (because the second is just a "rate" at which RAM space is reclaimed).

So, maybe Everything installer could check if user has these registry keys and if they don't, then add it? I shall leave the decisions to you, the author, whom I am grateful for this software. I am hardly productive without Everything anymore, for a couple of years now - thank you!