Everything reindexing all the file properties for no apparent reason

[Xan]

Hi! I've been using Everything for a while now and really like this simple yet powerful tool. I made an account and this thread because I have an issue that I would like being looked at.

I use the portable version as I don't use Everything every day and I open it a few times a week when I need to look for something. Problem is, some of the time at launch I get the dreaded "Indexing properties" loading bar and as I index a mechanical hard drive, it takes quite a lot of time to finish. This cannot be paused, cancelled or given lower priority as far as I know. What's weird is, I understand that some files might have been added, deleted or changed since last launch but Everything reindexes ALL of the files which seems strange and not efficient at all. I mean, the database is already there with 99.9% of the correct properties data (I at most modify 1000 out of 300000 files), why is this so?

Just in case you need some data, the db weighs 1.13GB. I index many video file properties like Frame Rate, Total Bitrate, length, width, Height and Aspect Ratio. The last rebuild reason of course is: Out of date NTFS volume D: is available

Is there a way to just check what changed and index properties of those files only? Reading all the files properties takes a lot of time and the disk is very busy for a long while. I understand I might be an edge case as I have many properties that I use but still, rereading files that have not been modified in years seems like a waste, the database has already all the properties on hand.


Cheers and thanks for a great tool.

[void]

Thank you for your feedback Xan,

This cannot be paused

Property indexing can be paused under Index menu -> Pause property indexing.

or given lower priority

Tools -> Options -> Advanced -> load_indexed_property_priority

I understand that some files might have been added, deleted or changed since last launch but Everything reindexes ALL of the files which seems strange and not efficient at all.

Everything will use the NTFS USN Journal to keep your index up-to-date.
If this journal is filled while Everything is not running, Everything will have to reindex.

Please check the last rebuild reason under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason
(Out of date NTFS found == too many changes occurred while Everything was not running)



Please try increasing your Maximum USN Journal size:

• In Everything 1.5, from the Tools menu, click Options.
• Click the NTFS tab on the left.
• For each NTFS volume:
  • Change Maximum size to: 262144 KB.
• Click OK.

-or-

keep Everything running in the background.

I mean, the database is already there with 99.9% of the correct properties data (I at most modify 1000 out of 300000 files), why is this so?

Everything will try to keep your indexed properties, but they are all flagged as dirty.
Everything will reindex your drive as it's basically seen as a new drive.

I am going to be rewriting Everything 1.5 to prevent property reindexing.
There's no way I can support the current versions.
It's going to take a long time so please bear with me..

[Xan]

Hey! Thanks for the reply. I will try your suggestions. I found out about pausing just after posting and felt silly. The priority thing its useful.

Please check the last rebuild reason under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason
(Out of date NTFS found == too many changes occurred while Everything was not running)

My first post had the rebuild reason and yes, it is Out of date NTFS.

Please try increasing your Maximum USN Journal size:

I will! It was at 32768 Kb so the extra size will probably help.

keep Everything running in the background.

I'm thinking about setting it up, but the extra overhead for mechanical disks (The everything database is on a mech disk too) and keeping up with all the file changes (Very useful too, see what file was modified in real time) was a little too much. I'm happy if I know it won't reindex all the files.

It's going to take a long time so please bear with me..

Don't ever worry about it! You need to bear with us users, lol.
Thanks a lot for the time and the journal tip, I'll report back if anything worthwhile comes up.

[Xan]

Just to update about this. Increasing the journal size solved all the reindexing issues. Things work beautifully now. Thanks again for the help!

[void]

Everything 1.5.0.1401a rewrites property indexing.

Everything will now remember index properties when doing a reindex.

Properties are now kept when moving a volume to a new drive letter.
Properties are now kept when too many changes have occurred.



Adding a new property to your index will now only index that property.



The database has changed in 1401a.
Previous versions required a full reindex on a database change.
1401a also adds database migration support so the full reindex is not required.
You might see a couple seconds delay to migrate your database when running 1401a for the first time.
-If you change back to 1400a or earlier a full reindex will be required.

[froggie]

After starting 1401 I get this message:

EV.PNG (12.68 KiB) Viewed 3516 times

Restarted, database rebuilds, all is OK.

Went back to 1400, tried it again. It is reproducible.

I don't need it fixed, but I thought you would be interested,

[void]

Thank you for testing 1.5.0.1401a froggie,

Everything is not migrating the extension fast sort database.

I will have a fix soon.

[void]

Everything 1.5.0.1402a will now migrate the extension fast sort database.