"Saving to disk" ...... running for many hours

[Shooter3k]

I've been having this strange problem where once in awhile, I'll get up in the morning and Everything will show it's "Saving to disk". This will continue for about half the day and then everything will be fine again. I'm not sure if this is from doing a large index update or what but is there any way to determine what it's doing?

When this happens, it's using about 1.8% CPU and 0.1MB/s disk writing. It appears that it increases usage of ram until it finally finishes <???>. Right now, it's using about 61GB of ram (I have 1tb of ram). My everything database is about 5gb.

The biggest problem when this happens is that I'm unable to use Everything until it completes. Meaning, it won't allow me to search.

[void]

Please send your Help -> Troubleshooting information.

[Shooter3k]

Everything 1.5.0.1391a (x64)
OS: Windows NT 10.0 26100 (x64)
Admin: 0
Service: 1 (connected / installed and running)
Command line: -startup
Binary: C:\Program Files\Everything 1.5a\Everything64.exe
Profile: C:\Users\home\AppData\Roaming\Everything\Everything-1.5a.ini
Database: C:\Users\home\AppData\Local\Everything\Everything-1.5a.db
Instance: 1.5a

Config:
check_for_updates_on_startup=1
auto_include_fixed_volumes=0
auto_remove_offline_ntfs_volumes=0
auto_move_ntfs_volumes=0
auto_move_refs_volumes=0
auto_move_fat_volumes=0
allow_literal_operators=1
index_date_created=1
filter=EVERYTHING
plugins=0
columns=[{"name":"Name","width":703},{"name":"Path","width":447},{"name":"Size","width":251},{"name":"Extension","width":96},{"name":"Date Modified","width":153},{"name":"Date Created","width":153}]
ntfs_volumes=[{"path":"C:","volume_name":"\\\\?\\Volume{1a}"},{"path":"M:","volume_name":"\\\\?\\Volume{fd}"}]
folders=[{"path":"D:","buffer_size":65536000,"update_day":7,"update_at":0,"fast_update":1},{"path":"G:","buffer_size":65536000,"update_day":7,"update_at":0,"fast_update":1},{"path":"J:","buffer_size":65536000,"update_day":7,"update_at":0,"fast_update":1},{"path":"L:","buffer_size":65536000,"update_day":7,"update_at":0,"fast_update":1}]
network_indexes=[{"host":"basementpc","username":"home","password":"********"}]

Policy:

Plugins:

[therube]

Might it be the Folder or Network indexes?

The Folders are only set to update once a week, & maybe they're never getting caught up on the changes there, timely?

[Shooter3k]

Might it be the Folder or Network indexes?

The Folders are only set to update once a week, & maybe they're never getting caught up on the changes there, timely?

I'm not sure but it's still running the 'saving to disk' and the ram usage is up to 74gb now

[void]

Thank you for the troubleshooting information.

Everything will auto save nightly.
This can be changed under Tools -> Options -> advanced -> db_auto_save

This nightly save should be rather quick.
Under 1 minute for 10million files.

How many files are you indexing? (shown in the status bar when your search is empty)



When Everything auto saves to disk, any pending folder changes and network index changes are processed.
Is something flooding your file system with changes?

When Everything recovers check your Index -> Index Journal.



Could be a corrupt index? circular folder reference somewhere?
Please try terminating Everything, deleting your %LOCALAPPDATA%\Everything\Everything.db and restarting Everything for a fresh index.

"buffer_size":65536000,"

While 64MB ReadDirectoryChange buffer (Tools -> Options -> Folders -> Buffer Size) should be fine, I don't recommend going over 64KB.
Also Everything will automatically drop this down to 64KB for network folders.
Network folders have a limit of 64KB.

A 64MB buffer is just wasted memory. Everything can process the 64KB buffer fast enough before it is completely filled.

[Shooter3k]

Thank you for the troubleshooting information.

Everything will auto save nightly.
This can be changed under Tools -> Options -> advanced -> db_auto_save

This nightly save should be rather quick.
Under 1 minute for 10million files.

How many files are you indexing? (shown in the status bar when your search is empty)



When Everything auto saves to disk, any pending folder changes and network index changes are processed.
Is something flooding your file system with changes?

When Everything recovers check your Index -> Index Journal.



Could be a corrupt index? circular folder reference somewhere?
Please try terminating Everything, deleting your %LOCALAPPDATA%\Everything\Everything.db and restarting Everything for a fresh index.

"buffer_size":65536000,"

While 64MB ReadDirectoryChange buffer (Tools -> Options -> Folders -> Buffer Size) should be fine, I don't recommend going over 64KB.
Also Everything will automatically drop this down to 64KB for network folders.
Network folders have a limit of 64KB.

A 64MB buffer is just wasted memory. Everything can process the 64KB buffer fast enough before it is completely filled.

There's at least 120 million files and yes, there are a lot of changes happening.

Is there a way to add multiple folders to Everything at the same time?

[void]

Please check your debug output:
In Everything, from the Tools menu, under the Debug submenu, click Console.
Wait for Everything to show Saving to disk.
Monitor console output.
You might see a lot of update events getting flushed to your index.
Look for the following lines:

db save (x) folders, (y) files
-This is shown when saving to disk actually starts.

saved db: (x) seconds
-Shown when finished saving to disk.
What time is shown?
The time shown doesn't include flushing update events to your index.

If the console is too busy, please try Tools -> Debug -> Start Debug Logging instead.
Click Tools -> Debug -> Stop Debug Logging after Everything has finished saving your index to disk.

Note: Enabling Tools -> Debug -> Verbose will output more debug information about processing update events.
However, enabling will make the console very busy.
-It might be useful to see exactly what Everything is doing during "Saving to disk".



Consider disabling auto save under Tools -> Options -> Advanced -> db_auto_save_type=Disabled

Is there a way to add multiple folders to Everything at the same time?

Add folders to your index?
You can drag multiple folders to Tools -> Options -> Folders.
-or-
You can edit your %APPDATA%\Everything\Everything-1.5a.ini directly.