Disable re-scan when changing certain settings

[David.P]

Hello forum,

I am scanning various network folders, some of which are located (via VPN) on other continents...

It seems that Everything re-scans everything every time a file exclusion is added, for example. This makes sense, and is probably needed to apply the respective change in the settings.

However, this can easily take a couple of hours in my case.

Is there any way to disable the instant re-scan, or postpone it to overnight?

Also, what would be the best settings (in particular for "Attempt to monitor changes" and "Fast rescan") for such a slow network drive, that is being scanned nightly, or even only weekly?

[void]

It seems that Everything re-scans everything every time a file exclusion is added, for example. This makes sense, and is probably needed to apply the respective change in the settings.

Everything should only perform a rescan when you remove an exclude.
Adding an exclude should perform a quick reindex without the rescan.

Is there any way to disable the instant re-scan, or postpone it to overnight?

No, Everything must rescan the entire network share.

Also, what would be the best settings (in particular for "Attempt to monitor changes" and "Fast rescan") for such a slow network drive, that is being scanned nightly, or even only weekly?

Only use Fast Rescan if the remote drive is NTFS.
Leave Attempt to monitor changes enabled.



I recommend removing all your excludes, indexing all files and omitting any unwanted results:

• In Everything, from the Tools menu, click Options.
• Click the Exclude tab on the left.
• Uncheck Exclude hidden files and folders.
• Uncheck Exclude system files and folders.
• Uncheck Enable exclude list.
• Click OK.
• From the Index menu, check Enable Omit Results.
• From the Index menu, check Organize Result Omissions.
• Add the files and folders you wish to exclude from your results to this list.
• Click OK.

Toggling Omit Results from the Index menu is very fast and does not require a rescan.



If possible, consider running an Everything Server remotely and including the remote index in your local index.

[David.P]

Everything should only perform a rescan when you remove an exclude.
Adding an exclude should perform a quick reindex without the rescan.

Thank you, got it.

Only use Fast Rescan if the remote drive is NTFS.
Leave Attempt to monitor changes enabled.

Ok. I suppose that it is, but is there any way I can tell whether the remote disk/server actually is NTFS? Or would Everything behave in a certain way if "Fast Rescan" is enabled and the remote disk is not NTFS?

I recommend removing all your excludes, indexing all files and omitting any unwanted results.
[...]
Toggling Omit Results from the Index menu is very fast and does not require a rescan.

Thanks, this is of course the best way.


Would there be a way to scan the entire file server directly, for example above the server with the address \\fileserver1?

Just to give an example, this is possible with Archivarius 3000:

[void]

Ok. I suppose that it is, but is there any way I can tell whether the remote disk/server actually is NTFS? Or would Everything behave in a certain way if "Fast Rescan" is enabled and the remote disk is not NTFS?

Do you have access to \\fileserver\c$ ?
The properties should show the file system.

Otherwise, the file system is unknown.

The file system does not have to be NTFS.
Fast rescan is supported if making a change to a directory causes the parent modified timestamp to update.
FAT, exFAT, FAT32 are not supported.


One way to test if you have write access:
Create a new file in a folder.
Does the folders date modified update?
-If so, use fast rescan.

One way to test if you have read-only access:
Check for a recently created file.
Make sure the date modified for the parent folder is later than or equal to the file creation date.
-If it is, use fast rescan.

If you find Everything missing changes, disable fast rescan.

Would there be a way to scan the entire file server directly, for example above the server with the address \\fileserver1?

To add a root server to your index:

• In Everything, from the Tools menu, click Options.
• Click the Folders tab on the left.
• Click Add Custom....
• Type in:
  \\fileserver1
• Click OK.
• Click OK.

[David.P]

Do you have access to \\fileserver\c$ ?
The properties should show the file system.

Nope, access denied.

Fast rescan is supported if making a change to a directory causes the parent modified timestamp to update.
FAT, exFAT, FAT32 is not supported.

One way to test if you have write access:
Create a new file in a folder.
Does the folders date modified update?

That worked! So I will switch on fast rescan.

To add a root server to your index:

• In Everything, from the Tools menu, click Options.
• Click the Folders tab on the left.
• Click Add Custom....
• Type in:
  \\fileserver1
• Click OK.
• Click OK.

Awesome! This worked as well. Took an entire day to index that server.

However, then I tried to search for stuff -- first I had set Everything to "Show folders".

I then switched to "Show everything", but that made Everything instantly start "Rescanning: \\fileserver1"

These are the current settings for that server:


Can it be said why Everything would re-start scanning here?

[void]

Changing the filter should not cause a rescan.

What is the rebuild reason shown under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason?

[David.P]

Here is the statistics log with the entry "Last rebuild reason: Add folder"

Code: Select all

Build
Count:	19
Total duration:	1:13:29:54
Minimum duration:	0,639486 seconds
Maximum duration:	41.627,028390 seconds
Average duration:	7.104,985268 seconds
Last duration:	41.627,028390 seconds
Last build date:	13.05.2022 22:26
Last rebuild reason:	Add folder.

Folder Index
Path:	\\fileserver1
Drive type:	Remote
Label:	
Index number:	4
Out of date:	No
Online:	
Disk device index:	
Multithreaded:	Separate device thread
Folder count:	122.279
File count:	881.108
Last rescan date:	13.05.2022 22:26
Last rescan successful:	Yes
Last successful rescan date:	13.05.2022 22:26
Last successful rescan duration:	13:04:21.853
Next scheduled rescan date:	15.05.2022 03:00

Folder Index
Path:	C:
Drive type:	Fixed
Label:	
Index number:	5
Out of date:	No
Online:	Yes
Disk device index:	0
Multithreaded:	Separate device thread
Folder count:	183.764
File count:	729.809
Last rescan date:	14.05.2022 11:30
Last rescan successful:	Yes
Last successful rescan date:	14.05.2022 11:30
Last successful rescan duration:	00:01:04.260
Next scheduled rescan date:	15.05.2022 05:00

Thus, a folder had probably been added to the server in the meantime.

And I didn't have "Fast rescan" enabled at that time, so the entire index was rebuilt -- is that assumption correct?

[void]

Here is the statistics log with the entry "Last rebuild reason: Add folder"

Thank you for the stats.

This means a folder was added to Tools -> Options -> Folders.
It is possible this rescan was not a rebuild.

Maybe a rescan was accidentally triggered from Index -> Recsan?

Did you recently add the C: drive to Tools -> Options -> Folders?
Your C: drive should be indexed as an NTFS volume under Tools -> Options -> NTFS.

What settings are shown for your C: drive under Tools -> Options -> Folders?
What settings are shown for your C: drive under Tools -> Options -> NTFS?

Thus, a folder had probably been added to the server in the meantime.

Adding a share to \\fileserver will not cause a full rescan.

And I didn't have "Fast rescan" enabled at that time, so the entire index was rebuilt -- that correct?

Changing Fast rescan should not trigger a rebuild.



I recommend removing \\fileserver from Tools -> Options -> Folders and adding each individual \\fileserver\share.
Detecting changes to \\fileserver is limited. (ShellChangeNotify)
Detecting changes to \\fileserver\share is more robust. (ReadDirectoryChanges)

[David.P]

Thanks very much, will do!

Here are the screenshots of the C: drive settings:

What settings are shown for your C: drive under Tools -> Options -> Folders?
What settings are shown for your C: drive under Tools -> Options -> NTFS?

[void]

Thank you for the screenshots.

Your C: drive is indexed as a "folder index" only.
This is fine.

For the best performance, please consider indexing your C: drive as an NTFS volume:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Everything Service. (Please make sure this is tick-checked and not square-checked)
• Click the Folders tab on the left.
• Select C: and click Remove.
• Click the NTFS tab on the left.
• Select C: and check Include in database.
• Click OK.

[David.P]

O.k. thanks very much for the information.

I believe that I can't add my C: disk as NTFS volume, because I don't have administrator rights on this machine.

Please one more question about the best indexing settings for the remote server \\fileserver1 (over the slow VPN connection):


If both "Attempt to monitor changes" and "Fast rescan" are enabled, is a scheduled rescan (currently every night) useful, or even necessary?

[void]

Are you keeping Everything running 24/7?

If so, you can disable the daily rescan and just rescan once a week/month.
Attempt to monitor changes is quite reliable and will pickup all changes.


Please note:
Indexing your fileserver1 as \\fileserver1 is not as reliable as
indexing your fileserver1 as \\fileserver1\share1 and \\fileserver1\share2 etc..



Everything will miss changes to \\fileserver1 when Everything is not running.
A rescan is necessary to pickup any changes while Everything was not running.

[David.P]

Thanks! Yes, the laptop with Everything runs 24/7.

[David.P]

...another small issue:

It seems that in the Network Drives tab, I can no longer delete the shares that I have added there -- the Remove button is grayed out?


Edit: updated Screenshot

[void]

The Network Drives list shows all online network drives and offline network drives in your index.

Online drives cannot be removed.

Instead of removing these drives, please exclude them from your index:

• In Everything, from the Tools menu, click Options.
• Click the Network Drives tab on the left.
• For each Network drive:
  • Uncheck Include in database.
• Click OK.

[David.P]

I see, thanks!

[David.P]

Please another (related) question:


To me, a folder of interest on a file server is accessible both via the address

\\fileserver1\SomeFolder

, and via the mapped drive letter

K:

.

Does it make a difference for Everything's performance whether I have it scan the server address, or the drive letter?

[void]

Short answer: no, it doesn't matter which path you use.

I have seen SMB cache issues when using a drive letter.
Everything 1.5 may take a few extra seconds to index and update a mapped drive letter.
Everything 1.4 will sometimes miss folders during indexing or updating.

[David.P]

Thanks very much

[David.P]

Hello all,

I just downloaded the latest version of Everything Alpha (portable noadmin installation).

It seems that Everything does a complete scan of all drives after the program update. Since this also includes some slow, remote network drives for me, the scan is going to take several hours to complete.

I believe there is no way to cancel the initial scan and work with the existing database until the next scheduled overnight scan is done:

Is there any way to disable the instant re-scan, or postpone it to overnight?

No, Everything must rescan the entire network share.

Is this new scan normal after a program update? I don't remember this happening with the last updates.

Thanks
David

[NotNull]

With version 1344a the database structure changed and that requires a rebuild if you were running a lower version than that.

[David.P]

I see, thank you. In the meantime, the scan is actually finished, too