Indexing of Properties repeats itself chaotically and causes Everything and system crashes

[Thy Grand Voidinesss]

After my recent update of Everything and used by me various options, my

Tools > Options > Indexes > Properties

settings now have 12 entries. Before this they were working A-OK - but now Everything becomes nearly unusable


I could somewhat understand if re-indexation was happening for some unusual / edge cases like audio files in Recycle Bin re-reporting their audio quality Properties- but when I hover over its progress bar it displays in the tooltip ordinary locations belonging to my all 7 volumes / drives i.e. it just goes again through all of my items. And so usually after a system reset when I open Everything then it once again re-indexes Properties, which takes it dozens of minutes. Sometimes this happens also when I close Everything, wait out hours and reopen it. And by close I mean killing also that other background EXE process of it. And when Everything performs indexation then sometimes my CPU usage can become 100%- Everything spikes for both moments and prolonged periods, grabbing whatever computing power there is available. And I have witnessed also silent crashes of it plus once got a pop-up message produced by Everything informing about a crash- something that has never happened to my before. I even get crashes of Everything that happend seconds after opening it

Recently also blue screen system crashes have been plaguing my machine with unknown root cause, but I am sure that re-indexation in Everything is a catalyst of them- when it is performed, then it often limits my Windows session to merely minutes of work instead of usually hours. And that is another problem, because after interrupted re-indexation I am forced to manually restore my

Everything-1.5a.db

database file from its

Everything-1.5a.backup.db

copy as there is no chance for Everything to finish re-creating the main file before another blue screen of death happens. I also often end up having

Everything-1.5a.db.tmp

instead of the main file i.e. the temporary version, which TMP does not seem to increase its size during re-indexation but only after it is finished

My database file when indexation is finished has size of ~1.66 GB. My system has 64 GB of RAM and I have reserved for Everything minimum of 3 GB and maximum of 6 GB - I gave it more than it needs on a daily basis because I am anticipating future Properties addition and existence of my archival offline drives that I have not re-connected to my machine in a long time due to this re-indexation issues with online drives [viewtopic.php?p=76775#p76775]


To sum up, there are two aspects: re-indexation keeps happening even if has been finished entirely in previous session of Everything / Windows - but it is hard to be become finished because it usually evokes another system crash


This has been happening in version 1.5.0.1397a x64 Portable and is still happening in 1999. As for my operating system, I have been using the same old version as before making changes to my Properties - Windows 10 Enterprise 20H2 x64 [10.0.19042]

[therube]

And by close I mean killing also that other background EXE process of it.

Killing is not nice, even if we have a department of war.
(Being nice is nicer.)

That background EXE process, you mean the Service?

(IMO, it's always better to properly close, rather then killing.)
(And if you are intent on killing, at least kill -9)
(killing also mentioned here, [SOLVED] CMD command of "net stop" for Everything closes also Media Player Classic.)

I even get crashes of Everything that happend seconds after opening it

Do you get .dmp files? If so, someone (not me) might be interested in seeing same?

db.tmp

Then it never finished what it was doing, to safely write to disk (to be renamed back to Everything.db).

IMO, Everything should not be causing crashes, BSOD's, nor be a precursor to such events.

Recently also blue screen system crashes have been plaguing my machine with unknown root cause

Eh, Win10/11 are simply, bullet-proof. They never crash nor BSOD.
(Sorry, I just had to throw that out there .)

limits my Windows session to merely minutes of work instead of usually hours.

On a more serious note, I would expect months.

because after interrupted re-indexation I am forced to manually restore my Everything-1.5a.db from its Everything-1.5a.backup.db

(On the odd time) I have not always been successful on attempting that (as re-indexing did still occur).
(There was a [somewhat fairly] recent thread on what may cause [Property] re-indexing.)
(Maybe this was it, Anyway to prevent intermittent indexing, or to back up the index so it doesn't have to be completely rebuilt?)

[void]

What is the last rebuild reason?:

• In Everything, from the Tools menu, under the Debug submenu, click Statistics.
• What is shown under Build -> Last Rebuild Reason.

See if disabling .metadata.efu helps:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  metadata
• Select: metadata_efu
• Set the value to: false
• Click OK.

[Thy Grand Voidinesss]

[...]
it's always better to properly close, rather then killing
[...]

If closing does not work then I revert to killing. Everything Service does nor close when I use

File > Exit

So if I am experiencing crashes or need all of the computing power that I can get then I must kill

What is the last rebuild reason?:
[...]

The

Tools > Debug > Statistics > Last Rebuild Reason

had showed me

Out of date NTFS volume C: is available

And then I switched to [not] using

Tools > Options > Advanced > metadata_efu > false

And it seemed to work - i.e. after long due re-indexing was finished it did not start again after Everything repeated re-openings and [later on] after user induced system resets. And my system worked without a crash for a 1.5 days. But eventually, as I did experience another crash it came back with a full force: Everything was re-indexing and looked like a catalyst for crashes of system, so I often had to kill it so that I could work. I also experienced one silent crash of Everything itself. And because things got too bad to continue to use Everything I checked if

Everything-1.5a.db.tmp

was created - but it was not. But then I made the stunning discovery of

Tools > Options > Advanced > metadata_efu > true

i.e. this option somehow has turned itself back on. So I turned it off again 5 days ago and since then it has not changed back itself true, but Everything still behaves in the same way i.e. sometimes after a BSOD it re-indexes and sometimes it does not, sometimes it silently crashes during it and sometimes I can use it without a sweat, sometimes it creates the TMP database file. Plus today after a night [of downloading torrents] I noticed one weird glitch [when I turned monitor on]: Everything was taking 36.7 GB out of my 64 GB memory, despite it having been told to use only up to 6 GB, which setting

Tools > Options > Advanced > max_working_set_size > 6442450944

I have just double checked. And I still have

Last rebuild reason: Out of date NTFS volume C: is available

in the debug after all these newest re-indexations


Should I perhaps perform the experiment of going back to using my older settings - but also with older version of Everything? And then, assuming it stops repeating indexation, go back to newest version of the program with still those old settings? And then load back my newer settings to newest version of program?

[Thy Grand Voidinesss]

It is now obvious that it is not the Everything solely that causes all these BSODs plaguing eve my restored from an image system, but I stand reassured it is [along with programs like BackUp Maker and SmoothVideo Project] a catalyst - as empirical experience has shown me, in hopes of just few hours of uninterrupted work I must kill preemptively [along with such others]; assuming of course it does not again crash itself silently at automatic startup

But I did get a new symptom once - something I have never experienced before: a pop-up error saying

.\src\db.c(24355): _db_corrupt(): Fatal error: remove fast sort file date modified: Everything database is corrupt. Please click OK and restart Everything to rebuild the database.

which resulted with me having to restore it from a backup drive because both the

Code: Select all

Everything-1.5a.db

and

Code: Select all

Everything-1.5a.backup.db

files got deleted by Everything. (I keep my portable Everything on the C volume / drive but its databases on S used as auxiliary system one with stuff like e-mails and recent backups)

[Thy Grand Voidinesss]

In version 1.5.0.1400a x64 Portable the repeated re-indexation is still happening


But what is interesting, it apparently skips my volume C despite that volume having benn treated with

Code: Select all

chkdsk /f

with which CMD command some repairing has been performed

[void]

Thanks for the issue reports Thy Grand Voidinesss,

Everything was taking 36.7 GB out of my 64 GB memory, despite it having been told to use only up to 6 GB, which setting

Tools > Options > Advanced > max_working_set_size > 6442450944

This doesn't limit the memory, this setting defines how much memory to keep in physical RAM.
This means almost all of Everything's memory will get paged to disk.
That is going to kill Everything and system performance.
I recommend leaving this setting off.



Could you please send a debug log where the repeated reindexing occurs:

• In Everything 1.5.0.1400a, wait for the repeated reindexing to occur.
• From the Tools menu, under the Debug submenu, click Start Debug Logging...
• Wait for Everything to do two reindexes.
• From the Tools menu, under the Debug submenu, click Stop Debug Logging...
  --This will open your %TEMP%\Everything Debug Log.txt in Notepad.
• Please upload this file in a bug report.

[Thy Grand Voidinesss]

[...]
Everything was taking 36.7 GB out of my 64 GB memory, despite it having been told to use only up to 6 GB, which setting

Tools > Options > Advanced > max_working_set_size > 6442450944
[...]

[...]
This doesn't limit the memory, this setting defines how much memory to keep in physical RAM.
This means almost all of Everything's memory will get paged to disk.
[...]

It does not limit the memory? I do not understand then: because if I set max amount of RAM that Everything is suppose to have access to - then I expect any requests for more memory above that limit to be accommodated with substitute in form of space on disk - i.e. that Everything will use the pagefile.sys after exceeding the limit imposed with that option

Does Everything use pagefile.sys in general, but when this option is used then it does not - and thus it is possible for it to choke / crash itself and / or the whole systems? Or perhaps it creates some sort of its own version of pagefile.sys - and if yes, then can user pick which volume / path should be used for this?

That is going to kill Everything and system performance.
I recommend leaving this setting off.

What will do the killing? The mere fact of using this feature - or the amount of max RAM value [in relation to my database size] I have chosen?

[And I think that the above portion of this discussion should be taking place in the viewtopic.php?t=15931 topic]

[...]
please send a debug log
[...]

Sure, I can and want to

But I just do not know how long I would have to wait for re-indexation to occur. Yes, I can easily evoke it by closing and re-opening of Everything - but this initial one will end at some point, after which Everything seems lately to be working A-OK in this regard. Or is the Debug Logging sustained between sessions of Everything? Because if it is sustained then I think it will be better to start it after one re-indexation is finished, so that [next] 1st one will be clean recorder from the get-go

[void]

It does not limit the memory? I do not understand then: because if I set max amount of RAM that Everything is suppose to have access to - then I expect any requests for more memory above that limit to be accommodated with substitute in form of space on disk - i.e. that Everything will use the pagefile.sys after exceeding the limit imposed with that option

Yes, you are correct, max_working_set_size will limit the physical memory.
Everything can still allocate more memory, but this will page other memory to disk.
You never want this to happen, it will kill system performance.

Does Everything use pagefile.sys in general, but when this option is used then it does not

The OS manages memory, RAM and the pagefile.
I don't really have control.
You can use min_working_set_size, max_working_set_size and virtual_lock to hint to the OS to never page to disk, but if Everything allocates large amounts of ram, paging can still occur.

If you want to use min_working_set_size and max_working_set_size, please make sure they are set to value higher than the RAM usage of Everything. (Never less than the RAM usage of Everything -it's fine to overshoot, so set it to your total RAM size)

- and thus it is possible for it to choke / crash itself and / or the whole systems?

The whole system will feel sluggish, it shouldn't crash, unless there's something up with your disk or a very low resource issue.

Or perhaps it creates some sort of its own version of pagefile.sys - and if yes, then can user pick which volume / path should be used for this?

No, Everything will never do this.
Everything wants to be in RAM.
Searching from disk defeats the purpose of Everything.
Search the Windows Index with si: in Everything if you want to store indexed information on disk.

What will do the killing?

Accessing memory that has been paged to disk.
Windows is very good at knowing what to page to disk.
However, when Everything needs to access memory that has been paged to disk, it means the system has to swap out something else to disk and load in the desired memory from disk. This will cause Everything to hang, and depending on your disk, will also cause the system to hang.

The mere fact of using this feature - or the amount of max RAM value [in relation to my database size] I have chosen?

Your min_working_set_size and max_working_set_size are too low, set them to the size of your total RAM.
but be aware that if Everything actually allocates all your RAM its going to leave the system with very low resources.

please send a debug log

Sure, I can and want to

But I just do not know how long I would have to wait for re-indexation to occur.

Ah, I thought Everything was stuck in a reindexing loop and the reindexing occurred immediately.



So if Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason is still showing:
Out of date NTFS volume C: is available

Then the issue is either:
Everything is not always running in the background.

-Please make sure Tools -> Options -> UI -> Run in background is enabled.
-Please make sure Tools -> Options -> General -> Run on system startup is enabled.

Too many changes occurred while Everything was not running.

-Please make sure there's no stuck Windows updates.

Something is deleting your USN Journal (not likely)

-Check the USN Journal ID is the same between Windows sessions under
Tools -> Debug -> Statistics -> C: -> USN Journal ID.

Something is spamming your file system with changes.

Monitor changes to your file systems (C: drive) from Index -> Index Journal

Everything is not saving your index to disk on exit.

-Please check the last save date under:
Tools -> Debug -> Statistics -> Save -> Last save date.
-Please make sure this is recent.

Yes, I can easily evoke it by closing and re-opening of Everything - but this initial one will end at some point, after which Everything seems lately to be working A-OK in this regard. Or is the Debug Logging sustained between sessions of Everything? Because if it is sustained then I think it will be better to start it after one re-indexation is finished, so that [next] 1st one will be clean recorder from the get-go

The log is overwritten if you exit and restart Everything.
One way to capture exactly what is going on is:

• Wait for Everything to do a reindex.
• Terminate Everything:
  • In Everything, type in the following search and press ENTER:
    
    /exit-now
• Launch Everything with:
  
  C:\Program Files\Everything\Everything.exe -debug-log
• Everything should immediately start reindexing.
• Wait for Everything to complete reindexing. (don't worry about property indexing completing)
• From the File menu, under the Debug submenu, click Stop Debug Logging...
  ---This will open your %TEMP%\Everything Debug Log.txt in Notepad.
• Please upload this file in a bug report.

[Thy Grand Voidinesss]

Well, now I have no idea what is happening


I have been busy with cleaning of my system / settings and drives in preparation for installing of new one on a brand new machine, thus opened Everything for the first time since exactly 2 weeks - and I was greeted with a pop-up window informing me that Everything needs to be run with privileges or the Everything Service has to be installed in order for indexation of NTFS to happen

After choosing the manual option I then went straight away to

Tools > General > Everything Service

and changed it from [half] indeterminate to [fully] tick-checked. I do not know why it was like this


I also made sure that at

Tools > General > Database location

the folder path points out to existing new location albeit with old database file. And I changed the

min_working_set_size

to 8000000000 while

max_working_set_size

to 16000000000 - i.e. to 8 GB and 16 GB, while still having physically 64 GB


But using the

Tools > General > Force Rebuild

button makes it happens only briefly for Recycle Bin folders - thus I am left with apparently old database being used and stuck in the past


I have not updated Windows in-between while my Everything 1.5.0.1400a x64 Portable version is the same as before. Task Manager is showing me that the Everything.exe which takes up ~830,836 K of memory is not elevated, while the other one that takes ~2,700 K has privileges

[void]

changed it from [half] indeterminate

Something likely uninstalled the Everything Service, which would explain the UAC prompt.

Tools > General > Force Rebuild

button makes it happens only briefly for Recycle Bin folders

What happens? the recycle bin still shows old files after a rebuild?

Please try a fresh database:
Exit Everything (File -> Exit)
In Windows Explorer, type in the following address and press ENTER:
%LOCALAPPDATA%\Everything
Rename your Everything-1.5a.db to Everything-1.5a.db.old
Rename your Everything-1.5a.backup.db to Everything-1.5a.backup.db.old
Restart Everything.

Does the Recycle bin update after Everything reindexes?

[Thy Grand Voidinesss]

I only now have spotted that all my Everything shows is content of Recycle Bin folders

But the rebuild of Recycle Bin multiple [drive's] folders seems to semi-fail at noticing new arrivals to it, no matter if deletion is performed before I hit the Force Rebuild button or after it finishes indexing. Because I do see a deleted item when I use the default Filter named Everything - but when I use my tested positively over and over again in the past Filter of

Code: Select all

:\$recycle.bin\ !$I <<search:> | displayname:<search:> | original-location:<search:>> columns:dm;extension;date-deleted;path;size;length;displayname;original-location;name;type

then the list becomes empty

[Thy Grand Voidinesss]

And I think I just have found the culprit of that newly arose issue: under the

Tools > Options > Indexes > NTFS

all of my new and old drives, no matter if they are online and offline, are listed with correct current letters - but they all had their

Include in database

option somehow turned off. It was not me who deselected this basic vital option

I reckon the culprit is what I did with my drives: [while still on old machine] I changed their letters, added new ones, removed some old ones; changed letters again and disconnected some new drives. I also changed the location of folder with my portable Everything. Plus I changed the secondary location in which I keep the database itself. But nevertheless I did not change in any way my system and torrent drives - and yet these two also got somehow de-included. So some of those action must have glitched / reset Everything in this regard altogether

After turning back that option the indexation is happening [right now] for locations also other than the Recycle Bin. But I do not know if this old falling apart machine will withstand the entirety of this process

[void]

Everything uses the volume name (\\?\Volume{01234567-8901-2345-6789-012345678901}) to track volumes.

This volume name will change between computers.
You will have to manually recheck "Include in database" when changing computers.

[Thy Grand Voidinesss]

But that is the point: I am still at the old machine - so how did all the drives, even the ones I did not mingled with, got all de-included?

And also now another shenanigans is happening: yesterday despite repeating the process of waiting for re-indexing to be finished, closing the program and re-opening it, the new database was not being created - the indexing results were apparently stored in RAM because no new

Everything-1.5a.db

file was being created on my drives. And yes, I used also a third party tool to look for this file and again checked if the path entered in the field of

Tools > General > Database location

is valid one [by coping its value and pasting it address bar of a file manager]. But when I woke up in the morning the database file has finally made its way to that location, with a timestamp from middle of the night. But then again: after opening of Everything re-indexing started happening but the database file did not get its timestamp updated after the process became finished - thus indicating it did not got detected changes saved to it

So once again I am now after yet another re-indexation which results are stored apparently in the RAM, because that DB file has unchanged timestamp

[void]

After choosing the manual option I then went straight away to

Choosing the manual configuration will remove all NTFS/ReFS volumes from your index.
This option should really be called "No system NTFS volume indexing"
-I will rename this for the next alpha update.

closing the program and re-opening it, the new database was not being created

Everything stores its database in memory.
Exiting Everything will save your index disk.

Closing the search window will not save your index to disk.

Did you just close the window or exit out of Everything (File -> Exit)?

Everything will also auto save your index to disk daily at 4am.
If this scheduled save is missed, then Everything will save the index to disk the next time you close an Everything window.

[Thy Grand Voidinesss]

I think we are getting closer to the core of these issues


A] My resets / shutdown are performed with BAT script that perform handful of necessary tasks - and then end up with

Code: Select all

shutdown /r /f /t 0 >nul

Code: Select all

shutdown /s /f /t 0 >nul

B] My system closing BAT scripts do not mingle with Everything. I have just re-checked this to be sure


C] To close Everything, when I need to free up resources or test something, I usually use such BAT script

Code: Select all

taskkill /IM Everything.exe
timeout /T 1 >nul
taskkill /IM Everything.exe /T /F
net stop "Everything (1.5a)"

D] Up to around those 2 weeks ago my machine was running constantly; albeit often without Everything in order lessen the number of processes that seem to be catalysts for the crashes plaguing this junk of mine. But now I do turn it off after midnight and turn back on in the morning [that database created in middle of the night that I mentioned in previous post was from 1 AM]


E] Aside from using system killer-scripts, my machine have been constantly experiencing crashes for months now. I cannot give the exact dates, but if I recall correctly they do more or less correlate with arrival of those overall problems that I started to have with previously correctly working Everything



And so:

#1] If I do remember precisely that for the last few days I have been closing Everything with my kill-script [thus never through its menu] - then how did it after all managed to create that database? And also a new was created today in the morning, some minutes after booting-up. How can Everything store indexing results from RAM between sessions of system? Does it use some temporary dump file - which apparently in my case gets rarely saved to the

Everything-1.5a.db

form


#2] I just performed a test of using the

File > Exit

option. The result was new size and timestamp applied to the

Everything-1.5a.db

. And after re-opening of Everything no re-indexation occurred. But then I again used the Everything-killer script. The result was some relatively quick indexation of described vaguely

Indexing properties... 99%
X:

where X is a letter of a drive from the pool of those marked for listing


#3] If my modus operandi relays on using kill-script for closing of system, because I do not like wasting time on prompts from dozen of opened programs, then am I doomed to have problems with databases / re-indexation in Everything?


#4] Can than 3 AM mandatory update of database be changed to user to some other hour? I myself perform automated backups at 20:00, so would set the Everything dump to be performed just before it

[void]

Please don't terminate Everything with taskkill.

You will lose all database changes.

To exit Everything gracefully, call:

Code: Select all

Everything.exe -exit -wait
net stop "Everything (1.5a)"

then how did it after all managed to create that database?

It didn't, the Everything.db will be the last scheduled daily save. (if there was one)

How can Everything store indexing results from RAM between sessions of system? Does it use some temporary dump file

Everything stores your index in %LOCALAPPDATA%\Everything\Everything.db
Everything will first save your database to Everything.db.tmp, then rename this to Everything.db
Your taskkill bat file will request Everything to exit, Everything will start saving your database to Everything.db.tmp, but then you only wait one second and terminate Everything, so you'll be left with the unusable Everything.db.tmp
Maybe you are seeing this Everything.db.tmp

If my modus operandi relays on using kill-script for closing of system, because I do not like wasting time on prompts from dozen of opened programs, then am I doomed to have problems with databases / re-indexation in Everything?

Yes.

#4] Can than 3 AM mandatory update of database be changed to user to some other hour? I myself perform automated backups at 20:00, so would set the Everything dump to be performed just before it

Tools -> Options -> Advanced -> db_auto_save_at_hour

db_auto_save_at_hour

-or-

Call

Everything.exe -save-db

if you would like to trigger a save externally.

-save-db

[Thy Grand Voidinesss]

Thank you for the info


It is true that sometimes I was seeing experiencing DB.TMP file showing up in the folder where I keep my

Everything-1.5a.db

database, without seeing a pattern in its occurrences. But as I have just checked this, I do not have the folder of

C:\Users\YOUR-USER-NAME\AppData\Local\Everything\

[As a reminder: I am using portable version]


I will now start testing in my daily usage the BAT modified to

Code: Select all

Everything.exe -exit
timeout /t 11 >nul
net stop "Everything (1.5a)"

If this will not prove be enough I will increase the cool-down period and / or add the

Code: Select all

Everything.exe -save-db

before the

net stop

command


I have also set

Tools > Options > Advanced > db_auto_save_at_hour > 19
Tools > Options > Advanced > db_auto_save_at_minute > 59

But on the side note: the default value of

db_auto_save_at_hour

in my Everything is

4

, not

3

. Maybe that is due to the daylight saving time difference?

[void]

But on the side note: the default value of

db_auto_save_at_hour

in my Everything is

4

, not

3

. Maybe that is due to the daylight saving time difference?

The default is

4

I did say

3

in my previous post, sorry.



Instead of the

timeout /t 11 >nul

call, please consider the following:

Code: Select all

Everything.exe -exit -wait
net stop "Everything (1.5a)"

-wait will wait for Everything to exit.
This means Everything will not be running for your

net stop "Everything (1.5a)"

call.

[Thy Grand Voidinesss]

This version

Code: Select all

"C:\QQ\Everything\Everything.exe" -exit -wait
net stop "Everything (1.5a)"

seems to work. I will continue to test-run this solution

[Thy Grand Voidinesss]

So: apparently when things are OK, then Everything seems to behave as expected and is stable i.e. it opens with immediately available results because it does not indulge in re-indexations of all items. This upholds also when a

Code: Select all

shutdown /r /f /t 0 >nul

is executed or a single BSODs occurs


But today I experienced something like 4 BSODs in 1 hour, due to whatever was the reason. So after yet another disruptive boot-up I opened Everything - and this time it was performing re-indexation. And the Task Manager was showing me again that

[...]
when Everything performs indexation then sometimes my CPU usage can become 100%- Everything spikes for both moments and prolonged periods, grabbing whatever computing power there is available
[...]

And so, yet again I am forced to hiatus my usage of Everything, because I have important work to do


Thus: is there a way to limit Everything's access to CPU resources, so that it can calmly heal its database? And afterwards, due to few BSODs in a row evoked by whatever culprits, prevent it from falling it again into another loop of re-indexations catalyzing system crashes - which crashes in turn create a need for database rebuilds, because Everything was not closed gracefully too many times in a row?


And what about the fact that

[...]
I do not have the folder of

C:\Users\YOUR-USER-NAME\AppData\Local\Everything\
[...]

?

[void]

this time it was performing re-indexation

What is shown under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason ?

is there a way to limit Everything's access to CPU resources

To limit Everything to 4 indexing threads:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  max threads
• Select: content_max_threads
• Set the value to: 4
• Select: index_max_threads
• Set the value to: 4
• Click OK.

Adjust the value 4 to how ever many threads you want.
I recommend half of your total available CPUs.

I do not have the folder of

C:\Users\YOUR-USER-NAME\AppData\Local\Everything\

What is your database location under:

Tools -> Options -> General -> Database location ?

Please make sure your Everything.db (Everything-1.5a.db in the alpha version) is saved to this location and is not deleted after a reboot.

[Thy Grand Voidinesss]

[...]
this time it was performing re-indexation
[...]

What is shown under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason ?

Back then I left Everything running by accident - and it did finish that re-indexation. But later on I was experiencing an unrelated weird glitch on one of the drives that made even the chkdsk hang up - which forced my to do a hard reset or two. And now that I finally got rid of the glitch, the Everything is again performing re-indexation - which is expected after some Journal rebuilds, right? So I reckon that right now the

Last rebuild reason: Out of date NTFS volume Q: is available

info does not tell us anything of value, because it is a rare occasion that I know what happened?

To limit Everything to 4 indexing threads:
[...]

I just set

Tools > Options > Advanced > content_max_threads > 5
Tools > Options > Advanced > index_max_threads > 5

as both my old currently used and already bought new CPU have 8 of them

But why not just use the

max_threads

?

And on the side note: if you pasted the name of these settings from some template file, then you have a typo in it - because you wrote context instead of content

[...]
make sure your Everything.db (Everything-1.5a.db in the alpha version) is saved to this location and is not deleted after a reboot.

For the last couple of weeks I had the folder with the database pinned as a tab in my filemanager- thus sometimes making sure of its presence and checking its timestamps when things went south. Amidst all of my issues with Everything and Windows, I just do not see a possibility of experiencing also unwanted deletions / disappearances of the DB file - and to which attest e.g. the fact that the just checked by me 5 full backups from different days when I was using Everything, do contain the Database file

I am using

C:\QQ\Everything\Everything.exe

with its

Tools > Options > General > Database location

set to

Q:\Configs\Everything\Database

and currently when re-indexation is happening I have

Q:\Configs\Everything\Database\Everything-1.5a.backup.db
Q:\Configs\Everything\Database\Everything-1.5a.db.tmp

[void]

now that I finally got rid of the glitch, the Everything is again performing re-indexation - which is expected after some Journal rebuilds, right?

Yes, Everything will reindex your drive if the journal is corrupted and rebuilt.

Try increasing the USN Journal Maximum Size on your Q: drive:

• In Everything, from the Tools menu, click Options.
• Click the NTFS tab on the left.
• Select your Q: drive
• Set maximum size to:
  131072
  KB.
• Click OK.

-this will not help against corruption, it will only help prevent the reindex if many changes occur.

But why not just use the
max_threads?

Setting max_threads will make your searches much slower.

And on the side note: if you pasted the name of these settings from some template file, then you have a typo in it - because you wrote context instead of content

That was a typo, sorry.
I fixed my OP.

Q:\Configs\Everything\Database\Everything-1.5a.db.tmp

The presence of this file indicates Everything was not shutdown correctly.
Everything was terminated before the database was completely written to disk.

You will still have your Everything-1.5a.db, and any recent changes will be lost.
Everything will have to rescan from the USN Journals, which takes time and if too many changes have occurred, Everything will have to reindex.

[Thy Grand Voidinesss]

[...]
Try increasing the USN Journal Maximum Size on your Q: drive:
[...]

I would rather refrain from dealing with this on a per volume basis. There is no way I will remember to keep track of this value when in the future I will add / remove disks to my machine and rearrange volumes within system. Thus: I spotted that my already enabled

Tools > Options > Indexes > Enable Journal

has

Maximum size

of merely 1024 KB - could this be the core issue and / or I could increase just this value?


As for the

Tools > Options > Indexes > NTFS > [A DRIVE]

I have a few volumes and each has its

Enable USN Journal

option selected - and thus active fields of

Maximum size

and

Allocation delta

. To these size values override the above? The Q volume has currently 66560 KB set, while every other has its own different value. Where do these size values come from? I do not remember setting them for the old ones and surely I did not in the past month set them for the new ones. The same goes for delta- what apparently automatically set it to their current values?

And I have noticed now that my T volume, that hosts torrents and downloads from YouTube, has a standing out

Maximum size

of 4194304 KB and

Allocation delta

of 488192 KB. And this single volume drive started these few days ago to have a new problem: despite me using the

Tools > General > Force Rebuild

and seeing it becoming finished, Everything still does not see changes- it it showing me files deleted from the T while also not seeing newly created ones, when at the same time other volumes were scanned and are monitored correctly [that is when I yet again manage to break away from that re-indexation loop]

[void]

Tools > Options > Indexes > Enable Journal has Maximum size of merely 1024 KB - could this be the core issue and / or I could increase just this value?

No, this is the Everything Journal and is unrelated.

Where do these size values come from?

Typically from Windows.
Same with the delta values.

These values can be set by any program.

Everything still does not see changes- it it showing me files deleted from the T while also not seeing newly created ones

Sounds like Everything is using the existing index instead of getting a fresh one.
I would need to see debug logs to work out the issue.
Tools -> Debug -> Start Debug Logging
Tools -> Options -> Indexes -> Force Rebuild
--wait for indexing to complete
Tools -> Debug -> Stop Debug Logging
Upload %TEMP%\Everything Debug log.txt in a bug report.

[Thy Grand Voidinesss]

And of course now Everything managed to finish re-indexation without a sweat and it seems to be working A-OK

So I should hold on to the log file for now and create another one if this happens again i.e. if after a freshly executed forced re-indexation I will spot that Everything is ignoring changes on any of the volumes? So that I will be able to send logs for two different outcomes?

[void]

I would need to see the log where a reindex is reusing the old T: index.

If Everything stops detecting changes on any volume, please capture a debug log:
Tools -> Debug -> Start Debug Logging
Make a change to your volume (eg: make a New Text Document.txt)
--wait for 1 minute
Tools -> Debug -> Stop Debug Logging
Upload %TEMP%\Everything Debug log.txt in a bug report.

[Thy Grand Voidinesss]

It seems that I am in a junky and stretched in time but nevertheless endless loop: of Everything working A-OK, then breaking after apparently usually multiple BSODs, then re-indexing itself and either working again as it should or requiring manual intervention from me

A couple of days ago I was again in situation where despite having performed manual re-indexation one of the drives was apparently again not responding in Everything to occurring changes but I was very busy thus unable to tackle this issue deeper - and when I finally found time I discovered that it again fixed itself on its own

I do not know if I will keep troubleshooting this when still on old machine or just wait it out for when I finally assemble the new one. But if I will start using the new one and these issues will do disappear due to lack of constant system crashes, then I will come back to this topic to happily report the I have no more data to share

[Thy Grand Voidinesss]

I finally moved on from my old machine with Windows 10 Enterprise 20H2 x64 [10.0.19042] to a brand new hardware with Windows 10 IOT Enterprise LTSC 21H2 x64 [19044.6575]

I have been using Everything 1.5.0.1400a x64 Portable on it for the last two weeks now, which is the same version with the same settings as on the previous PC setup. Although something might come up later in relation to this transition, any issues with indexation seem to be all gone - despite indexing now ~100% more items due to additional backup drives and despite using for the first time a ReFS volume

If my old machine would be a life organism, then it would had to be marked as terminally ill - it got worse and worse in so many ways that I do not even know where to begin trying to describing its issues. And when I came back to using it for further tests, mid-through them I got locked out of it, because for whatever reason I cannot even restore system from an image using my Acronis True Image DVD - and without such restoration using it is futile, because that system is plain unusable

Long story short, it seems I have experienced a self-reinforcing circle of hardware power failures and corruptions of vital system files that spiraled down to the bitter end - and all that is left for me to do now is to dismantle the old junk and sell it on a per-device basis, with hopes that whatever mechanical / electrical root cause there was it will not manifest itself in a different hardware / software configuration within the setups of aftermarket buyers', so that I will not have to give back them their money

All in all, this re-indexation became an unresolved mystery - because like I said, I am unable to boot up to old system any more, while a mere fought of spending any more time on trying to revive it makes me sick