Index journal exclusions, size increase, journal-only db and full export timestamps?

[Herkules97]

I don't remember if I was meant to make one post for each question, seems faster to merge them all. Some of these are duplicates from before and act more as re-checks.

If I exclude C:\Users\User\AppData\Local\Temp\AJC* in the exclusion list, maybe using the "Exclude files" wildcard box, does it apply to the journal when the db monitors using the NTFS tab? Asking in case I can avoid having to make a test folder.

Is there a plan to increase journal size maximum? I thought I read about it in a post, but todo list does not seem to have it.
I would probably do 32GB if I could, to prevent overwriting older entries before I can separate.

Speaking of that, can I run a db that only keeps index journal entries? Right now the monolithian db I run starts at 8GB any time I want to separate the db and start a new journal. If I could run one that only keeps index journal entries on all devices in the NTFS tab(or cmd line if it creates dbs that can be read with the GUI later), there would be no cost in separating .

Finally, journal still exports only down to seconds when the full range is visible in the db.
Is there already a way to get a csv out that contains the full range?
The full range in one file is 2022-08-30 15:42:53.3600024 and exports via GUI stops at 53.
Never used the cmd line, can I do it there?

[void]

If I exclude C:\Users\User\AppData\Local\Temp\AJC* in the exclusion list, maybe using the "Exclude files" wildcard box, does it apply to the journal when the db monitors using the NTFS tab?

Yes.
Excluded files are not added to your index journal.
The Index Journal is an audit of your index.

Is there a plan to increase journal size maximum?

Not at this stage because the journal is stored in RAM.
The default is 1MB.
I recommend 32MB if you find the journal index useful.
Please don't go over 1GB.
Use Everything journal logging or other tools to log changes.

Speaking of that, can I run a db that only keeps index journal entries?

Same as above, I don't recommend going over 1GB.
Use logging.

Finally, journal still exports only down to seconds when the full range is visible in the db.
Is there already a way to get a csv out that contains the full range?

Currently, no.
I will consider support for higher resolution.

Never used the cmd line, can I do it there?

Not implemented yet.

[Herkules97]

Is there a plan to increase journal size maximum?

Not at this stage because the journal is stored in RAM.
The default is 1MB.
I recommend 32MB if you find the journal index useful.
Please don't go over 1GB.
Use Everything journal logging or other tools to log changes.

I've been using 3.8GB on all dbs just fine for 2 years or however long it's been, but logging is inside the Advanced tab?
How does the logging work? It places logs in a folder named for each day? Maybe there is a guide on it, Idk where that would be.

Speaking of that, can I run a db that only keeps index journal entries?

Same as above, I don't recommend going over 1GB.
Use logging.

But if only db has the full time and I don't want to abandon that, is there a way to run a db that keeps nothing in the main db? I am presuming logging has the same nerfed time as GUI exports do.
So without any journal entries, the db might only be a few KBs up to a MB.
I believe when I was testing properties with the monolithian db, I had disabled some properties like date modified but the journal still indexed those properties.
If so all that should be missing is excluding files in main db but not in the journal.
Would mean separating every 3.8GB journal costs nothing extra.
My monolithian db is 8GB any time I separate, so every db copied is that big at least.

[void]

Journal logging

Journal logging creates a daily log.
The default location is %LOCALAPPDATA%\Everything
The location can be customized.

But if only db has the full time and I don't want to abandon that, is there a way to run a db that keeps nothing in the main db?

The journal will log changes to your index.
You will need to index your files. (the db cannot be empty)



The journal log can be configured to save full resolution timestamps.

[Herkules97]

Journal logging

Journal logging creates a daily log.
The default location is %LOCALAPPDATA%\Everything
The location can be customized.

But if only db has the full time and I don't want to abandon that, is there a way to run a db that keeps nothing in the main db?

The journal will log changes to your index.
You will need to index your files. (the db cannot be empty)



The journal log can be configured to save full resolution timestamps.

So I tried it instead of asking further about it, I was going to ask if it's on-disk or in-ram like EBV normally is.
After some oddities with maybe Windows Explorer I did find that it works on-disk.
It is a bit unusable given the amount that is written to have it constantly do that, no way to make it in-ram and dump it at the end of the day or when manually told so via the search bar like /save-log?

What I am not seeing is path, new path(Granted maybe these two aren't really needed but Idk how to work with multi-GB files to regex separate the path from the files if I do want to read the path and filename separately), size, date modified/created/accessed, attributes and the two parent date modifieds.
So while the GUI journal export does lack the full timestamp, it has all that other data.
I tested at least #size: and that did nothing.

For me, when I temporarily did GUI exports until I remembered that partial timestamps are why I never did it in the first place, I just placed dbs inside of new folders with an added "(export)". That way I can export while the live dbs still runs on a different instance.
Granted you can do it in the same folder probably, but that makes it messier.
If the exporting ever gets the full timestamp range, I will start copying in old dbs and exporting.

What I wonder is the thing about not going over 1GB.
It seems like a mysterious issue, but are you just thinking of RAM usage?
The only memory leaking I've had has come from Stalker 2, no issues from EBV.
Any massive RAM climb, from sorting filenames, I already know about. My monolithian db climbs from about 31GB to 52GB or whatever RAM.
This happened recently when I told it to stop saving those AJC- temp files.
It still reads them, given the CPU usage climbs whenever I am restoring from an AJC archive. But at least it doesn't keep any of it in the journal.
Reading a full journal might be about as much, 3.8GB, RAM which mostly or entirely goes down when you exit the journal window.
But if there is anything else, I might not be aware or I've forgotten.

[void]

What I wonder is the thing about not going over 1GB.

The index is stored in memory.
I do not recommend going over 1GB.
Everything is not designed to index over 1GB efficiently.



dumpusn might be more suitable.

dumpusn can be called daily to dump the USN Journal for an NTFS volume.
The USN Journal will contain all changes to the NTFS volume.
No need to run Everything.

[Herkules97]

What I wonder is the thing about not going over 1GB.

The index is stored in memory.
I do not recommend going over 1GB.
Everything is not designed to index over 1GB efficiently.



dumpusn might be more suitable.

dumpusn can be called daily to dump the USN Journal for an NTFS volume.
The USN Journal will contain all changes to the NTFS volume.
No need to run Everything.

Ah, ok well I can report EBV works fine at 3.8GB, the only concern is that it will overwrite old entries which is solved by purging.
Alternatively I could just let it continue overwriting entries.
In the past I did do that, I copied off a recently saved db when it reached near the last saved dbs' newest id entries.
If I can remember those ids or around there. Maybe some extra loss, I could lazily just check for the millionth number. If last db saved at 23mil, save the next db when the bottom ids reach 22mil. I probably only stopped because more than just the journal was kept for every new db copied. But this is only maybe 100MB without journal so there is little loss.
Another factor was probably that I had to read every journal individually and I can't use my memory method this way. But for just one db it would be fine.

Wouldn't be surprised if it works fine above 10GB even. Though there was that issue with the stat page, the presumption I get with the journal is that it scales well.

Would be sweet to have a way to grab every instances' journal id amount into one pop-up window to quickly know which are reaching the overwriting threshold. Less guesswork, but might be too much of a hassle to make. Or EBV needs to read the entire journals to know the amount..That could max out memory usage at least for my system.
I currently use a memory method, if most instances are above 3GB private bytes in Process Explorer it can mean they're nearing.

It is fortunate the journal still saves all the data even if the main db has all of that data disabled.
It still saves the size, datetimes and such. Hopefully that doesn't change, even if it is a bug (Not that it is a bug here..But a reference to Firefox's decision to remove bookmark tags in the history window making it less convenient to know which recent URLs have bookmark tags)