What does "monitor changes" actually monitor?

General discussion related to "Everything".
Post Reply
NotNull
Posts: 1589
Joined: Wed May 24, 2017 9:22 pm

What does "monitor changes" actually monitor?

Post by NotNull » Mon Jun 19, 2017 2:54 pm

Under folder indexing, there is an "Attempt to monitor changes" option.
I have this option enabled and it seems this monitor catches every change on my system.
From time to time I do a manual rescan of the folder indexes, but it seems that's hardly necessary ...

The question(s) (mostly out of curiousity):
What is actually monitored? What does it miss?

( Everything Version 1.4.1.877 (x64); Win7 )

EDIT: NTFS- and ReFS indexing also have a "monitor changes". Is this the same monitor?

void
Site Admin
Posts: 4965
Joined: Fri Oct 16, 2009 11:31 pm

Re: What does "monitor changes" actually monitor?

Post by void » Tue Jun 20, 2017 1:12 am

Generally, Everything will detect all local and remote changes made to an indexed folder while Everything is running.
Everything uses ReadDirectoryChanges to monitor changes to folders.

It is possible for too many changes to occur in a short time frame before Everything has a chance to read all the changes.

With the default settings Everything will just ignore the missed changes and continue monitoring.
With Rescan on full buffer checked, Everything will rescan the entire folder when the full buffer event occurs.

Everything only monitor changes to folder indexes when Everything is running. If you make a change to a folder indexed by Everything when Everything is not running, this folder index will be out of sync the next time you run Everything.

Everything will rescan all folder indexes at 3am everyday by default to make sure folder indexes are up to date.
If this scheduled rescan is missed, a rescan will occur the next time you run Everything.
The rescanning process is done in the background, you can continue to use Everything as normal while this occurs.

Folder monitoring is different to NTFS/ReFS monitoring.
NTFS/ReFS monitoring uses the USN Journal to read changes.

Everything does not need to be running to read all changes as the USN Journal is maintained by the NTFS driver.
Changes made to NTFS/ReFS volumes will not be missed by Everything.

It is possible for the USN Journal to fill up before Everything has a chance to read all changes, however, this is unlikely since the USN Journal is quite large, typically 32MB+ which is enough to store about a week of changes.
In the event the USN Journal has changed too much, Everything will automatically reindex the affected volume.



Folder monitoring and NTFS/ReFS monitoring detect new files, deleted files, renamed files and modified files.

w64bit
Posts: 58
Joined: Wed Jan 09, 2013 9:06 am

Re: What does "monitor changes" actually monitor?

Post by w64bit » Tue Jun 20, 2017 9:20 am

In order to detect as many changes as possible for NTFS/ReFS, what settings shoud be activated?

void
Site Admin
Posts: 4965
Joined: Fri Oct 16, 2009 11:31 pm

Re: What does "monitor changes" actually monitor?

Post by void » Tue Jun 20, 2017 10:47 am

For NTFS/ReFS, nothing needs to be changed.

32768 KB is the default maximum USN Journal size.

If you find Everything reindexing often, you could try increasing your maximum USN Journal size to 65536 KB.

For folder indexes, if Everything is missing changes, you could try increasing the folder change buffer size to 1024 KB.

NotNull
Posts: 1589
Joined: Wed May 24, 2017 9:22 pm

Re: What does "monitor changes" actually monitor?

Post by NotNull » Fri Jun 23, 2017 4:46 pm

void wrote:Generally, Everything will detect all local and remote changes made to an indexed folder while Everything is running.
Everything uses ReadDirectoryChanges to monitor changes to folders.
Thanks a lot! Very clear.
Folder monitoring is different to NTFS/ReFS monitoring.
NTFS/ReFS monitoring uses the USN Journal to read changes.
That was what I thought (/expected), but in the NTFS/ReFS Option dialog "Monitor changes" can be configured even if USN Journal is not enabled.
So I thought it was another kind of monitoring ... But now I'm enlightened :-)

Thanks again!

void
Site Admin
Posts: 4965
Joined: Fri Oct 16, 2009 11:31 pm

Re: What does "monitor changes" actually monitor?

Post by void » Sat Jun 24, 2017 5:23 am

The USN journal is required to monitor NTFS/ReFS volumes.

If you uncheck USN journal, Everything will automatically uncheck Monitor changes.
If you check monitor changes, Everything will automatically check the USN journal.

When monitor changes is enabled, Everything will automatically create the USN Journal if it does not exist.
The USN Journal is automatically created by Windows on installation with Windows Vista or later.

NotNull
Posts: 1589
Joined: Wed May 24, 2017 9:22 pm

Re: What does "monitor changes" actually monitor?

Post by NotNull » Tue Jun 27, 2017 10:45 am

void wrote:The USN journal is required to monitor NTFS/ReFS volumes.
If you uncheck USN journal, Everything will automatically uncheck Monitor changes.
If you check monitor changes, Everything will automatically check the USN journal.
But ... :-)
If you check USN journal, "Monitor changes" will not be checked automatically. Hence my (ex-) confusion.

This is all just nitpicking ... It's just that - in my opinion - the Everything Options dialogs have a very clean and clear UI-language and organisation (there isn't even a miscellaneous tab!) and this one (monitor changes) was less obvious.

A thought: when USN journal and Monitor changes are "hard-linked", you could remove the Monitor changes checkbox altogether and change the "Enable USN Journal" to something like: "Enable USN Journal to monitor changes".

void
Site Admin
Posts: 4965
Joined: Fri Oct 16, 2009 11:31 pm

Re: What does "monitor changes" actually monitor?

Post by void » Fri Jun 30, 2017 2:37 am

If you check USN journal, "Monitor changes" will not be checked automatically. Hence my (ex-) confusion.
The USN Journal is not a part of Everything, it's stored on your hard disk and maintained by the NTFS driver.
You may want to disable "monitor changes" in Everything and keep the USN journal enabled so other applications can use the USN Journal.

NotNull
Posts: 1589
Joined: Wed May 24, 2017 9:22 pm

Re: What does "monitor changes" actually monitor?

Post by NotNull » Sat Jul 01, 2017 1:59 pm

void wrote:You may want to disable "monitor changes" in Everything and keep the USN journal enabled so other applications can use the USN Journal.
Good point! (although every application can create delete or resize the USN Journal at will. (IIRC; might be wrong about that .. ))

void
Site Admin
Posts: 4965
Joined: Fri Oct 16, 2009 11:31 pm

Re: What does "monitor changes" actually monitor?

Post by void » Sun Jul 02, 2017 11:10 am

Each NTFS volume has its own USN journal settings (enabled, maximum size and delta size).
Any application can modify these settings.
Administrative privileges are required to change these settings.

Post Reply