Index Journal Suggestions

Discussion related to "Everything" 1.5 Alpha.
Post Reply
adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Index Journal Suggestions

Post by adamantine » Mon Mar 15, 2021 1:45 am

(i use win-server 2003 r2 sp2, 32-bit, 32 gb ram) (Everything-1.5.0.1238a.x86 portable)

thank you for the very promising version 1.5 (at the moment i may use it for the tests only)
the 'index journal' feature is probably the most interesting for me (along with the 'content indexing' feature)

speaking about 'index journal':
- since 2010 i had been constantly using the 'cenvorox file monitor' program (version 3.2.4.2) (made in 2006) (i still have to use it)
- not a single attempt (to substitute it with something more convenient and modern) was successful in these 11 years
- the 'index journal' is a ray of hope for sure
but right now there are quite a lot of negative aspects/questions/ideas that i would like to mention:

1) in my case the 'Export...' button allows to save a CSV-file with this content only:
ID,Date,Action,Name,Path,New Name,New Path,Size,Date Modified,Date Created,Date Accessed,Attributes,Parent Date Modified,New Parent Date Modified
this is the 1st line (the 2nd line is empty) (there are 2 lines only, while the 'index journal' window displays hundreds/thousands of ID's)

2) will it be possible (later) to choose which columns to show and which not to show?
for example, i certainly don't need these: Date Accessed, Attributes; and most likely several others too
while the horizontal space is quite valuable there


3) will it be possible (later) to export to usual .txt-file? (not only csv)

this is the pattern/example that i suggest to consider for such txt-file (which in fact will serve us as a log):
1   2021.03.14  09.42.25   File Rename   D:\Programs\my-test-9.txt   D:\Programs\my-test-10.txt
2   2021.03.14  09.42.25   Folder Modify   D:\Programs
but it even could be implemented in a lot better way (more convenient to look through a long txt-file whose content looks like a huge log; because in majority of cases the content of 1 line will fit that line horizontally):
1   2021.03.14  15.01.05   D:\Programs\my-test-9.txt   Rename From
2   2021.03.14  15.01.05   D:\Programs\my-test-10.txt   Rename To
3   2021.03.14  15.01.05   D:\Programs   Modify
the rest of example could look like this:
4   2021.03.14  17.42.25   D:\Programs\my-test-11.txt   Add
5   2021.03.14  20.03.00   D:\Programs\my-test-9.txt   Delete
6   2021.03.15  00.42.25   D:\Programs\my-test-10.txt   Delete
7   2021.03.15  00.42.26   D:\Tests\my-test-10.txt   Add
8   2021.03.15  03.00.59   D:\Programs\my-test-12.txt   Add
(yes, i think that all the time/date-related symbols have to be highly customizable: order, user-defined delimiters/spaces, presence of seconds...)

here ID's 6 and 7 are technically the 'move' action (this is just an alternative way to display the ID's, thus not using the 'move' definition at all)
the idea is based on the long experience which i have with the 'cenvorox file monitor' program

thanks to it, i also have the saved history of all my file/folder events since august of 2010: it occupies 1 gb in txt-files but they are not archived
(as you understand, they may occupy much less in a rar/zip-format)
but 'cenvorox' is not convenient in many ways: it constantly requires manual tidying up, it demands to be a service, it demands to have 1/2 processes in a loaded state, it makes impossible to rename any root folder in any drive (to actually rename any of it, 'cenvorox' may be completely unloaded but this is the additional and slightly tedious action)


4) so there is a very important feature that may be added to 'index journal':

a. possibility to manage/save/keep all the file-system-events (often it it necessary to return to some point and see exactly in tiniest details what was happening with this or that file/folder, even if it was long time ago)
b. i suggest to implement some convenient system that will allow to keep absolutely all ID's in automatically created (easy readable) txt-files
c. this system/feature will be aimed at advanced users, and it has to be very customizable
there should be a possibility to choose the folder where those txt-files will be created by ET, according to user-defined settings like:

- how many maximum ID's a txt-file may contain (for instance, we choose 20.000, then a new txt will be auto-created after each 20.000 ID's)
(my personal choice would have been 200.000 or even much more)
- what is the name pattern for such txt-file? for instance, i would have decided to see it this way: "2021.03.14  19.42.25.txt"

right after the auto-creation of a consecutive txt-file, the 'index journal' clears its content (which was already copied to a just saved txt-file)
but of course, the clearing must be done by ET only when the 'index journal' window is not opened

5) the upper 'Action' line could be improved
there is a lot of free space there to the right side, and thus the drop-down list is a pure non-ergonomic evil in this case
all its 11 entries could be made available directly as small and accurate buttons: horizontally placed, 1 by 1, with a corresponding inscriptions like: all actions, folder create, folder delete...


6) idea: if some line is selected (in the result field of 'index journal'), press on F2 may give this whole line the 'inline rename' state
(this will allow to select any part of a path/name/date... and send it to clipboard: this option could become a nice time-saver in many situations)

7) now the title bar is 'Index Journal'. it may become something like this: Index Journal - 3217
so it will immediately prompt a user: how many file/folder events are displayed right now in a long list (no need to scroll it down)
it may be useful if ID's sort is ascending (0 on top) which is my case always
(the '3217' value should not be auto-changed during the current session of the 'Index Journal' window)
(this suggestion is quite arguable and should be treated as least important)

8) something like the 'Import...' button may be considered too
those automatically created/named txt-files will be used with this new 'Import...' setting
(here the main sense is probably in the fast search/filter that is already implemented in the 'index journal' window)
but importing should not re-write the current (not saved so far) view in the 'index journal' window
so seemingly there should be some additional button like 'Switch To Current Events' (near the 'Import...' button)

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Mon Mar 15, 2021 10:58 am

Thank you for your feedback adamantine,

The Index Journal was created so you could monitor changes to your file systems.
I had never intended for it to log changes to disk. However...
1) in my case the 'Export...' button allows to save a CSV-file with this content only:
this is the 1st line (the 2nd line is empty) (there are 2 lines only, while the 'index journal' window displays hundreds/thousands of ID's)
I swear I tested this! It was a very recent sort optimization that broke it..

Please try the latest Everything 1.5 Alpha.

Version 1.5.0.1242a fixes an issue with exporting the index journal when no search is active.
2) will it be possible (later) to choose which columns to show and which not to show?
for example, i certainly don't need these: Date Accessed, Attributes; and most likely several others too
while the horizontal space is quite valuable there
Added to my TODO list.
3) will it be possible (later) to export to usual .txt-file? (not only csv)
Done, please try the latest Everything 1.5 Alpha.

Please let me know what you think, what needs to change.
Is there a standard log format going around?
Do I need to export the Journal Item ID for each line?

-I can't split the journal up internally to Rename To and Rename From as this needs to be a single event for the Everything Server. However, I do split them into separate To/From lines for the txt export.
(yes, i think that all the time/date-related symbols have to be highly customizable: order, user-defined delimiters/spaces, presence of seconds...)
I'm using rfc 3339 for the time format. Is there any reason to use something else?
Do you need more precision than seconds? -I can go to 100 nanoseconds.
a. possibility to manage/save/keep all the file-system-events (often it it necessary to return to some point and see exactly in tiniest details what was happening with this or that file/folder, even if it was long time ago)
I've added a journal_log_file ini option to specify a log file on disk to the latest Everything 1.5 Alpha.
Set this to the filename of your log file, eg: D:\Logs\Everything Index Journal.txt
I still need to add log file rolling... added to my TODO list.
b. i suggest to implement some convenient system that will allow to keep absolutely all ID's in automatically created (easy readable) txt-files
Keep item IDs, noted.
Currently, if you change your index, the Journal will be reset and the next item ID will start at 0.
I will look into keeping the journal through index setting changes.
I can't do this due to the Everything Server, clients use the Index Journal to synchronize their local indexes.
An index setting change may just have to force log rolling.
c. this system/feature will be aimed at advanced users, and it has to be very customizable
there should be a possibility to choose the folder where those txt-files will be created by ET, according to user-defined settings like:
- how many maximum ID's a txt-file may contain (for instance, we choose 20.000, then a new txt will be auto-created after each 20.000 ID's)
(my personal choice would have been 200.000 or even much more)
Added to my TODO list: max line count, max size limits for log rolling
- what is the name pattern for such txt-file? for instance, i would have decided to see it this way: "2021.03.14 19.42.25.txt"
Added to my TODO list: auto name for log file (eg: date), daily logs.
right after the auto-creation of a consecutive txt-file, the 'index journal' clears its content (which was already copied to a just saved txt-file)
but of course, the clearing must be done by ET only when the 'index journal' window is not opened
This would be handled with log file roll over.
I am consider another tool to do this... I will need to create an API to pull this data out of Everything.
5) the upper 'Action' line could be improved
there is a lot of free space there to the right side, and thus the drop-down list is a pure non-ergonomic evil in this case
all its 11 entries could be made available directly as small and accurate buttons: horizontally placed, 1 by 1, with a corresponding inscriptions like: all actions, folder create, folder delete...
I'll consider checkboxes instead of the dropdown combobox.
6) idea: if some line is selected (in the result field of 'index journal'), press on F2 may give this whole line the 'inline rename' state
(this will allow to select any part of a path/name/date... and send it to clipboard: this option could become a nice time-saver in many situations)
This already exists, unless I'm missing something? -It's not inline though, it brings up a rename dialog.
There's also Ctrl + C to copy the path of the currently selected items.
Right clicking gives more options.
7) now the title bar is 'Index Journal'. it may become something like this: Index Journal - 3217
so it will immediately prompt a user: how many file/folder events are displayed right now in a long list (no need to scroll it down)
it may be useful if ID's sort is ascending (0 on top) which is my case always
(the '3217' value should not be auto-changed during the current session of the 'Index Journal' window)
(this suggestion is quite arguable and should be treated as least important)
I wanted this too. I was thinking of a status bar, the window caption will work for now..
Done in the latest Everything 1.5 Alpha.
8) something like the 'Import...' button may be considered too
those automatically created/named txt-files will be used with this new 'Import...' setting
(here the main sense is probably in the fast search/filter that is already implemented in the 'index journal' window)
but importing should not re-write the current (not saved so far) view in the 'index journal' window
so seemingly there should be some additional button like 'Switch To Current Events' (near the 'Import...' button)
I'll consider it.

Thank you for your helpful suggestions.
I hope I can make the Index Journal work for you.

mtu
Posts: 5
Joined: Mon Mar 15, 2021 4:40 pm

Re: Index Journal Suggestions

Post by mtu » Mon Mar 15, 2021 7:36 pm

i'm testing the alpha and appreciating this feature as an Avafind refugee this reminds me of the must-have feature(for me) that it used to have, the Scout Bot. it enabled a user to monitor the system for created and modified folders and files and it is great to see such a feature finding its way to Everything, Thank you.

i'd like to offer my suggestions on this feature

- would it be possible to enable the Index Journal window to be dockable ? it would be great to have it always visible and attached to the main window
- i'd also like to be able to create custom Actions. let's say i'd like to view only deleted files and folders, currently we can only view one or the other but with the ability to see what Actions that we want to see we can mix and match the Actions that would suit our needs.

version 1.5 is shaping up to be an exciting release and hopefully i can fully let go of the memories of the user experience i had with Avafind(specifiacally the Scout Bot) all those years ago and go all in with Everything.

NotNull
Posts: 2861
Joined: Wed May 24, 2017 9:22 pm

Re: Index Journal Suggestions

Post by NotNull » Mon Mar 15, 2021 8:01 pm

mtu wrote:
Mon Mar 15, 2021 7:36 pm
- would it be possible to enable the Index Journal window to be dockable ? it would be great to have it always visible and attached to the main window
A different approach to that:
When you configred Everything to run in the background (Menu:Tools > Options > UI ), you can even close the Everything Search window.
The Index Journal will stay opened.

To open a new search window, double-click the Everything system-tray icon (for example; there are more ways).

mtu
Posts: 5
Joined: Mon Mar 15, 2021 4:40 pm

Re: Index Journal Suggestions

Post by mtu » Mon Mar 15, 2021 8:36 pm

whew, that was fast! appreciate you getting back to me.

yes i've noticed that the Index Journal remains open when Everything's main window is closed but i'd like to simulate the Scout Bot feature from Avafind. here the Scout Bot showing recent filesystem changes was to the right of the search results.

Ava Find.JPG
Ava Find.JPG (55.52 KiB) Viewed 8440 times

NotNull
Posts: 2861
Joined: Wed May 24, 2017 9:22 pm

Re: Index Journal Suggestions

Post by NotNull » Mon Mar 15, 2021 9:09 pm

For the record, @void is the developer of Everything.
I am just someone who helps here answering questions and have no influence on how Everything evolves (which is a good thing! )

Some time ago I played around with Ava Find. I did like the ScoutBot, but also think the Index Journal will be much more powerful in the end (I like teh extra information)

Your suggestion to select multiple actions will certainly help with that (missed that too a couple of times).
The user interface is still under heavy development - it is an alpha release after all - so we will have to wait what @void has to say if and when this gets implemented (there is still a lot to do .. )


FWIW: I liked Ava Find's way to present filenames (in two lines; for others, see here for an example)

mtu
Posts: 5
Joined: Mon Mar 15, 2021 4:40 pm

Re: Index Journal Suggestions

Post by mtu » Mon Mar 15, 2021 9:54 pm

NotNull wrote:
Mon Mar 15, 2021 9:09 pm
For the record, @void is the developer of Everything.
I am just someone who helps here answering questions and have no influence on how Everything evolves (which is a good thing! )
have seen some of your posts around the forum and suspected as much but your input/help is still very much appreciated it is good to have your queries answered in a matter of hours and not days, if ever.
NotNull wrote:
Mon Mar 15, 2021 9:09 pm

Some time ago I played around with Ava Find. I did like the ScoutBot, but also think the Index Journal will be much more powerful in the end (I like teh extra information)
yes already the Index Journal already offers more than the Scout Bot which is why i'm excited about this feature and the release of version 1.5.
NotNull wrote:
Mon Mar 15, 2021 9:09 pm

Your suggestion to select multiple actions will certainly help with that (missed that too a couple of times).
The user interface is still under heavy development - it is an alpha release after all - so we will have to wait what @void has to say if and when this gets implemented (there is still a lot to do .. )
let's hope he takes it on board as this is the perfect time to forward these suggestions before the door is closed near to final release.
NotNull wrote:
Mon Mar 15, 2021 9:09 pm

FWIW: I liked Ava Find's way to present filenames (in two lines; for others, see here for an example)
had forgotten how convenient it was to have information presented that way, it saves horizontal space and looks a bit better. hopefully this two line display makes its way into everything one day.

NotNull
Posts: 2861
Joined: Wed May 24, 2017 9:22 pm

Re: Index Journal Suggestions

Post by NotNull » Mon Mar 15, 2021 10:22 pm

mtu wrote:
Mon Mar 15, 2021 9:54 pm
let's hope he takes it on board as this is the perfect time to forward these suggestions before the door is closed near to final release.
Good point!

I find it really amazing that on the third day after the very first release of an alpha version, we are already talking about suggestions for new functionality. The forum should be full of bug reports!
Which means that @void has done a really excellent job here (but undoubtedly more bugs will still come to light ...)

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Tue Mar 16, 2021 2:09 am

void, i'm grateful for the detailed answer and for your serious approach to various aspects
even if some of them are not that big and may seem not that important

1 of my main goals is to never lose a single line/event from the history
so now i'm trying to understand for myself: will it be possible at all to switch to the 'index journal'?
and if 'yes', then: how exactly should i maintain this whole new system? (without the 'cenvorox file monitor' program)
which means: i need to be sure that ET won't get me into some real trouble like this:
Currently, if you change your index, the Journal will be reset and the next item ID will start at 0
i didn't knew about this aspect while writing my previous message in this thread

but at the moment the 'index journal' is still in the process of implementation
and some most essential things are not ready yet
so that's why in my posts here i mention various small questions/ideas too
(though i'm much more worried about the main things like: correct exporting and further convenient reading of the saved txt-file)

export to txt-file is successful now (the yesterday's csv-bug is gone)
(i don't think that we need to be bothered about 'standard log format' because the txt-format probably should be enough)


1) i just noticed that 'index journal' has severe trouble with the date-time:
they are displayed according to the moment when the 'index journal' was opened (not when the events were actually happening)

this behavior can be seen if you load the 'index journal' after some period of time: and see at the dates-times of the just added (to the list) ID's
they all will have equal date-time (the exact date-time of the moment when you opened the 'index journal')

and even more: the exported txt-file also has these numerous equal dates-times
while it is clear that they do not reflect the real state of things:
the real events were happening earlier (and in very different hours/minutes/seconds)


2) (inside the exported txt-file) all notes like 'File Move From', 'File Modify'... are strangely placed/arranged
sometimes the gap looks like 2-3 empty symbols (and sometimes 8 or more)
i propose to change it so that there were always exactly 9 spaces before such notes (namely spaces, not some 1 huge 'tab' symbol)
while 6 spaces may be placed between date-time and path

currently all lines in this txt-file begin with the date: i ask to add the number to start of each line
3 spaces may be placed between the number and the date

speaking about the view of date-time. now it looks like this:
in the main window of 'index journal': 2021.03.16 00.58
while in the exported txt: 2021-03-16 00:58:03
is it possible to have these 2 various versions look identically? exactly like this: 2021.03.16   00.58.03

so here's the example of the desired display (in the exported txt):
1   2021.03.15   22.58.03      D:\Images\2021.03.15\example-8.png         File Create
2   2021.03.15   23.04.58      D:\Images\2021.03.15\example-8.png         File Delete
3   2021.03.16   01.02.21      D:\Images\2021.03.16\example-number-9.png         File Rename From
4   2021.03.16   01.02.21      D:\Images\2021.03.16\example-number-10.png         File Rename To

so, to sum up:
3 spaces between number of line and date
3 spaces between date and time (if possible)
6 spaces between time and path
9 spaces between path-file and action

all these small aspects will help a lot because this txt should be created in an easy readable/searchable way


3) a small idea: new button 'pause' (in the main window of 'index journal')
sometimes it may become useful when concentrating on some details inside the numerous lines of 'index journal'
(without any kind of distraction like: constantly resizing vertical scroll, constantly changing number in the title bar)
(and it seems that further press on 'pause' should load all the "missing" events to the list)
(so this is the pause for the view only: generally the info about 'file system events' should not be lost)

4) a tiny question: now ID's are starting with 0. is it possible to have them always starting from 1?

5) some people may like to use the drop-down list (not the checkboxes)
so maybe it's better to implement it as an ini-setting (if it is technically possible):
option-1: current state (drop-down list only)
option-2: 11 checkboxes only
option-3: both the drop-down list and 11 checkboxes

and as was mentioned above in the other post: some additional and combined actions probably may be useful
for example: Folder+File Create, Folder+File Delete...
i propose to name them this way: Items Create, Items Delete...


6) i'm also thinking about the following aspect:
my main ET is 1.4.1.963 and i always use it with these 4 columns only: name, extension, size, date modified
as you can see, there is no 'path' column (because it's combined with the 'name' column)
there's a corresponding ini-option to do it (i became slightly happier when i found it by chance)

i had been using this approach/view for a year or 2 (and i find it very-very convenient)
in no way i'm going to return to the standard 'path column + name column' system
so maybe later we should consider/discuss something like this for the 'index journal' too

(i apologize for possible mistakes and for possible not very logical moments in my message:
there are too many nuances here, while i have to describe them all not in my native language)

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Tue Mar 16, 2021 1:25 pm

Thanks for the feedback adamantine,

I've simplified the logging so I have something working for now in Everything 1.5.0.1243a.
More options to come...

To enabling journal logging:
  • In Everything, from the Tools menu, under the Debug submenu, click Config
  • This will open your Everything configuration file in Notepad.
  • Change the following lines:
    journal_log=0
    to:
    journal_log=1
  • Save changes and close Notepad.
  • Accept the prompt in Everything to reload your configuration
When journal logging is enabled, Everything will create the following file:
%LOCALAPPDATA%\Everything\index-journal-2021-03-16.log
-where 2021-03-16 is the current date.

A new log will be created for each day. (this will probably need to be monthly)

The contents of the log file look like:

Journal-ID Item-ID Date Filename File/Folder Action

For example:
01d71987c50842f9 999 2021-03-14 09:42:25 D:\Programs\my-test-10.txt File Delete

The Journal-ID with an Item-ID will give you a reference to a unique change (Since the Journal-ID can change and the Item-ID will reset to 1).
The Date is now the USN time-stamp, not the Everything log time.
2) (inside the exported txt-file) all notes like 'File Move From', 'File Modify'... are strangely placed/arranged
I'm using TABs so it is easier to parse the data. I will consider spaces.
If I use spaces, I would put the action before the filename, eg:

1 2021.03.15 22.58.03 File Create D:\Images\2021.03.15\example-8.png

The action could be space padded so the filename always starts on the same column.
speaking about the view of date-time. now it looks like this:...is it possible to have these 2 various versions look identically? exactly like this: 2021.03.16 00.58.03
I'm using RFC 3339 for the date format for easy parsing.
2021.03.16 00.58.03 is your local format.
I will consider using the local format.
Will you ever parse or analyze these files later? how will you read them?

3) a small idea: new button 'pause' (in the main window of 'index journal')
You can pause the journal from main Everything search window -> Index -> Pause updating.
I will consider adding this option to the Index Journal somewhere..
4) a tiny question: now ID's are starting with 0. is it possible to have them always starting from 1?
Done, I'm still using indexes internally, so ID = internal_index + 1
i propose to name them this way: Items Create, Items Delete...
I'll consider this, thanks for the suggestions.

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Tue Mar 16, 2021 5:14 pm

1) now ID's are starting with 1, thank you

2) at the first glance, the "severe trouble with the date-time" seemed to be gone in the new '1243'-alpha-version (i'm not completely sure about it)

but when i again loaded ET (after several hours), i saw that many entries are arranged very strangely (inside the 'date' column). example:
1425 2021.03.16 17.58 File Create some-file-1
1426 2021.03.16 17.58 File Delete some-file-2
1427 2021.03.16 16.04 File Create some-file-3
1428 2021.03.16 16.05 File Modify some-file-4
the earlier events (from 16.04 and 16.05) are placed after the newer events (which is not normal/acceptable)


3) if i had the possibility, then i would have changed the following things:

a. remove the notes like "01d71a6b2ddb91ca", "01d71987c50842f9"
(horizontal space is very important in the exported txt-file: often the path-name may be quite long) (moreover, these notes look awful)
i'm trying to understand: is 'a reference to a unique change' that necessary?

b. not use "%LOCALAPPDATA%\Everything\"
(all related folders/files should be placed in the ET-folder) (it's the portable application after all, at least in my case)

c. not use the .log extension (the .txt extension is enough)

d. bring back the 'File Rename From', 'File Rename To' instead of these new variants: 'File Rename-From', 'File Rename-To'

i even propose to consider these simplified variants: Rename From, Rename To, Create, Delete...
(without specifying if it's a Folder or a File)
(in the overwhelming majority of cases we already have a dot and an extension if it's a File, and thus we may distinguish a File from a Folder)

(i also think that it's better to use 'Add' instead of 'Create')

this will probably need to be monthly
'monthly' is of course a long way better than 'each day'. but 'weekly' seems to be the optimal choice
(or we may return to the idea about limits: '20.000 or 200.000' from the 1st message of this thread)
I will consider spaces
ok. i hope to see them someday
(generally, the current display of the exported txt-file is really not good for me: i hope that none of us will give up and that this part of ET will be functioning right eventually. i'm not proud of it, but i was very close to giving up after i saw this new alpha-version)
Will you ever parse or analyze these files later? how will you read them?
speaking honestly, there's nothing serious/interesting here in my case. i will just open these exported txt-files from time to time (often) (sometimes very often) and analyze some part of it visually
(i also may edit them or copy something from them to clipboard)
(i also search inside such history-files a lot: using my Notepad2 or using the content-search of ET)

so sometimes it is kind of a reference, while in other cases it can also be a way to get the info that helps me to repair some mistakes, accidentally made by me or by some program or by some script or maybe even by the OS
I will consider adding this option to the Index Journal somewhere..
i propose to consider adding it as a button with the unpressed/pressed state
(Index Journal should remember a state of the button across the sessions of ET)

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Fri Mar 19, 2021 9:49 am

Thank you for your feedback adamantine, and for persisting with Everything 1.5a..

I've made some changes to log file layout in Everything 1.5.0.1245 as I now understand you are only going to view these logs (not analyze/parse them):
I'm using 3 spaces now.
Please let me know if this layout is an improvement, how is the 'action' column positioning?
I've also added the option to toggle the Index Journal columns and I've disabled some of the unnecessary ones by default.
There's still a lot to do here, I hoping I'm making some progress..
2) at the first glance, the "severe trouble with the date-time" seemed to be gone in the new '1243'-alpha-version (i'm not completely sure about it)
I'm looking into this issue. Are these files on different drives? Everything will update one drive at a time, so it is possible you'll see a bunch of updates for one drive, and then another drive. Keeping Everything running in the background should help prevent this as there will be less updates to process.

a. remove the notes like "01d71a6b2ddb91ca", "01d71987c50842f9"
This is the Journal ID. Combined with the item ID, this will give you a unique reference to an index change.
I've removed this Journal ID column. However, keep in mind the ID could reset to one due an index setting change.
I will look into keeping a unique ID across index setting changes.
a reference to a unique change' that necessary?
For viewing the logs, no.
For analyzing the logs, yes.
The Journal ID could go in the filename.
b. not use "%LOCALAPPDATA%\Everything\"
With the latest update, Everything 1.5 will now store the logs in the "logs" folder in the same location as your Everything for the portable version.
c. not use the .log extension (the .txt extension is enough)
I will try .txt
FYI the -1.5 in the filename is just the instance name, this will be removed in beta/release.
d. bring back the 'File Rename From', 'File Rename To' instead of these new variants: 'File Rename-From', 'File Rename-To'
Done, I was trying to make it parse-able, but now I know this is not necessary.
(without specifying if it's a Folder or a File)
Done.
'monthly' is of course a long way better than 'each day'. but 'weekly' seems to be the optimal choice
I've abandon the monthly idea, the logs will grow way too large and will be unviewable.
I'll explore storing single day logs in a weekly or monthly subfolder.
(or we may return to the idea about limits: '20.000 or 200.000' from the 1st message of this thread)
It's on my TODO list.
Thank you for your suggestions.

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Sat Mar 20, 2021 4:18 am

here's the description of my situation (just in case):
a) i have 0 ssd. currently i have 5 hdd (all ntfs) (just 5 logical drives: c, d, e, f, g) (totally 600.000 objects: folders and files)
b) i never use: ET service, ETP/FTP server, HTTP server
c) i never use: installed ET (portable only)
d) i never have ET running in background (luckily, i don't even have the notification area in my OS anymore)
here i mean: the "run in background" setting is always disabled

e) i have the "create a new window when running Everything" setting always enabled
f) i constantly have 0 or 1 or 2 or even more instances of ET opened (with various searches in each of them)
g) when i finish some current task in 1 instance of ET, i usually press Num5. which means Alt+F4 in my main ahk-script
h) when i need to search for something in ET (while other instances of ET are busy or absent), i press the / button (near the right shift)
which means "run ET" in my main ahk-script

Are these files on different drives?
yes


in 1.5.0.1245 i created an exported txt-file. there were 232 actions. it turned out that the txt contained 244 lines:
the last line is empty (which is bearable but not good), while there were also defective/duplicated numbers of lines. examples:
4   2021-03-19 21:41:31   Create        F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini.tmp
5   2021-03-19 21:41:31   Rename From   F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini.tmp
5   2021-03-19 21:41:31   Rename To     F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini
6   2021-03-19 21:41:32   Create        C:\Program Files\Cenvo-Rox-File-Monitor\data\cr_file_events.cdb-journal

30   2021-03-19 21:43:57   Delete        F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini
31   2021-03-19 21:43:57   Rename From   F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini.tmp
31   2021-03-19 21:43:57   Rename To     F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini
32   2021-03-19 21:43:57   Create        D:\Images\2021.03.19\2021.03.19   21.43.57.png

78   2021-03-19 21:46:50   Delete        F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini
79   2021-03-19 21:46:50   Rename From   F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini.tmp
79   2021-03-19 21:46:50   Rename To     F:\New-4\Everything-1.5.0.1245a.x86\Everything-1.5a.ini
80   2021-03-19 21:46:50   Create        C:\Program Files\Cenvo-Rox-File-Monitor\data\cr_file_events.cdb-journal
as i understand, these duplications are only related to the "Rename From"/"Rename To"-actions (for some reason)
(added later: the duplications were already present in the 1.5.0.1243 version)


here are my thoughts concerning the general appearance of the exported txt-file:
it looks not good if the "action" column is placed before the "path-name" column
it is also harder to visually compare adjacent paths/names (which sometimes may be quite similar), while for example i have to find a small difference
so i ask to bring back the previous approach (when the "action" column was after the "path-name" column)

currently the situation with spaces is not optimal:
- 3 spaces (between number of line and date) look good

- 1 space (between date and time) (instead of 3 spaces) looks not good (but kind of bearable)
not equal separators (hyphens in date, colons in time) look not good (but kind of bearable)
just 4 dots (instead of 2 hyphens and 2 colons) would have been the better variant
but frankly speaking, even the current display of date-time is already better than in the "cenvo-rox" program

- 3 spaces (between time and next column) are quite normal
(i was previously asking for 6 spaces but this is the aspect where i'm completely ready for a compromise)

- i noticed that the quantity of spaces is very different between various "actions" and the "path-name" column:
8 spaces after Delete, 8 spaces after Create, 3 spaces after Rename From, 5 spaces after Rename To, 8 spaces after Modify
this issue may be solved this way:

the "action" column becomes the last column
6 spaces are added between "path-name" and any type of "action"
if it's technically hard to implement 6 spaces here, then let it be 3 spaces (but it's visually not convenient here: 3 is too little)
(i was previously asking for 9 spaces but this is the aspect where i'm also ready for a compromise: 6 will be quite good, 3 will be kind of bearable)


I'll explore storing single day logs in a weekly or monthly subfolder
actually, it's not that easy for me to define the best solution right now
briefly i'll try to describe my long-term practice with that "cenvo-rox" program:

(approximately once a week) (or sometimes less frequently) i manually export all actions to the txt-file
and right after that i clear those very events actions in the program's GUI (using 1 simple setting in the main menu of the program)
sometimes the txt-file may contain ~50.000 actions (lines), sometimes it's 200.000 or 300.000
(added later: in case of some error or some my mistake, i may manually export all actions to the txt-file substantially more often than just once a week)

in the program's GUI: the quantity of actions is always static, so i never lose a single action
1 of the main problems is: the rarer i perform such cleaning, the longer i wait for the program's GUI to load all the numerous lines (with actions)
(added later: 1 line contains 1 action)
after the load: i see the appearing of a complete list of actions that were done right after my previous manual export to a txt-file

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Mon Mar 22, 2021 7:48 am

Thank you for your feedback adamantine,
a) i have 0 ssd. currently i have 5 hdd (all ntfs) (just 5 logical drives: c, d, e, f, g) (totally 600.000 objects: folders and files)
Thank you for sharing your setup.
Would it be preferred to a have separate logs for each drive?
-This would keep Dates in sequence.

My current thoughts are to make a separate program to log the USN Journal to suit your needs.
in 1.5.0.1245 i created an exported txt-file. there were 232 actions. it turned out that the txt contained 244 lines:
the last line is empty (which is bearable but not good), while there were also defective/duplicated numbers of lines.
I could use a unique Log ID for each line. However, these would not match the IDs in the Index Journal UI.
I am assuming you would need the Log ID to match the ID in the Index Journal UI. -Please let me know if this is not the case.
The Index Journal must have a single ID for a complete rename operation (old name and new name) for Everything Server synchronization.
it looks not good if the "action" column is placed before the "path-name" column
it is also harder to visually compare adjacent paths/names (which sometimes may be quite similar), while for example i have to find a small difference
I assumed you are using a mono-spaced font for reading the logs.
What about using a 3 letter appreviation, eg: ADD/DEL/RNT (rename to)/RNF (rename from)/MVT (move to)/MVF (move from)
- 1 space (between date and time) (instead of 3 spaces) looks not good (but kind of bearable)
I need to add custom date/time formatting code to do this, its on my TODO list.
The current version is using rfc 3339. The plan is to use your local format, my local time formatting code currently does not support seconds so it's on my TODO list.
the "action" column becomes the last column
I find this hard to read, maybe using [DELETE] after the filename with square brackets might help.
(approximately once a week) (or sometimes less frequently) i manually export all actions to the txt-file
Thanks for sharing your cenvo-rox usage.
I'll look into some sort of simple mechanism to call Everything daily-weekly to update a log.

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Mon Mar 22, 2021 10:49 am

Would it be preferred to a have separate logs for each drive?
no, just 1 log is needed (otherwise, i'll constantly have 5 times bigger concern/worry)
I am assuming you would need the Log ID to match the ID in the Index Journal UI
yes, this assumption is correct. i would like to add a few thoughts here:
- this match seems to be a very natural and useful thing
- but if it's technically very difficult to achieve, i may try to test some future implementation where the Log ID's are totally absent
(this could be the last resort)


these days i'm much more worried about the fact that the 'index journal' is maybe not very reliable
(even if we speak about it's main functionality)
here's the example of what i mean:
- in the evening: i perform some file/folder-actions
- in the night: ET is completely absent in the taskbar (both the regular 1.4-version and the 1.5-alpha)
- in the morning: i load the 1.5-alpha, open its 'index journal', and see that only some part (of my evening file/folder-actions) is present
- simultaneously i also notice that all not numerous 'night'-actions (auto-saves of my opened firefox, for example) are totally absent
(while they are present in the 'cenvo-rox' program)

- so i see a huge period of time (many hours) which is not reflected/reported in the 'index journal' in any way
(while some actions were certainly performed during that period of time)
- in the end of the 'index journal' i see several very fresh actions (belonging to the minutes when i loaded the current instance of 1.5-aplha)

(i'll continue to observe this strange behavior of the 'index journal')
The Index Journal must have a single ID for a complete rename operation
yes, i understood that
(i'm trying to comprehend if those duplicated numbers are very big evil
i don't know, maybe i'll manage to get used to them somehow)
I assumed you are using a mono-spaced font for reading the logs
here my knowledge is only enough to say that i use Notepad2 with these 'default font' settings:
font - tahoma, font style - regular, size - 12, script - cyrillic
but i suspect that this info is not important at all in this thread
What about using a 3 letter appreviation, eg: ADD/DEL/RNT (rename to)/RNF (rename from)/MVT (move to)/MVF (move from)
i feel that it's definitely not for me (i also never liked any kind of abbreviations)
I find this hard to read
i can understand this. what about trying it this way:
the "action" column is placed before the 'path-name' column (like you prefer)
but at the same time the beginning of the 'path-name' column is arranged so that all paths begin very-very equally (vertically)

there will be rare but obvious exceptions in cases when there is passage from 9 to 10, 99 to 100, 999 to 1000...
(these exceptions will not exist if we really decide to get rid of the Log ID's)

my quick calculations show that we need to add 12 spaces after any action:
Create, Modify, Delete, Rename From, Rename To, Move From, Move To
or if the spaces are not good enough to accurately arrange the paths vertically: maybe we should try again your initial approach (TABs instead of spaces before the 'path-name' column)
maybe using [DELETE] after the filename with square brackets might help
i don't think that square brackets (or any other brackets) (or the action displayed in the upper case) are a good idea

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Tue Mar 23, 2021 11:33 am

Thank you for your feedback adamantine,
these days i'm much more worried about the fact that the 'index journal' is maybe not very reliable
I will look into this.
It could be Everything performing a fresh reindex -it would be useful to log the journal ID to know for sure. -Check the Last build date from Tools -> Debug -> Statistics -Is this build date recent? -What is the last reported rebuild reason?
- so i see a huge period of time (many hours) which is not reflected/reported in the 'index journal' in any way
Was there actions in the journal with a date before this missing time period?



I've added a journal_log_format option to Everything 1.5.0.1249a.

Please try the following format:
  • In Everything, type in the following search and press ENTER:

    Code: Select all

    /journal_log_format=#split:#id:   #date:<YYYY.MM.DD   hh.mm.ss>   #filename:      #action:
    
    Type in the following search and press ENTER:
    /restart
Syntax is:
#split: = use two lines for rename / move
#id:<width> = item ID
#date:<format> = date in rfc 3339 or in the specified format. For example: <YYYY-MM-DD hh:mm:ss.SSS>
#action:<width> = action text.
#filename:<width> = old filename (old and new if using split)
#newfilename:<width> = new filename.

<width> (optional) = pad text to width characters. Prefix with - to left align. For example, show action left aligned to 6 characters:
#action:<-6>

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Wed Mar 24, 2021 5:54 am

It could be Everything performing a fresh reindex
i may be wrong but i don't think that this is the reason
Is this build date recent?
info from Tools -> Debug -> Statistics:
Last build date: 2021.03.24 05.16

info from index journal:
1   2021.03.24 05.16   File Create         Path: D:\Images\2021.03.24         Name: 2021.03.24   05.16.33.png

info from exported txt:
1   2021.03.24   05.16.33   D:\Images\2021.03.24\2021.03.24   05.16.33.png      Create
What is the last reported rebuild reason?
the 'Last rebuild reason:' line is empty
Was there actions in the journal with a date before this missing time period?
yes, many of them. another very fresh example:
i still keep the earlier 1245-alpha (it had been untouched by me for many-many hours: but i don't remember the exact intermission)
so i just opened it and saw the following situation: there are 103.810 actions

the actions from 1 to 103.696 belong to this period of time: from 2021.03.19 21.41 to 2021.03.23 19.10

the action 103.697: 2021.03.24 01.37 (here i have the obvious absence of numerous actions that were actually done after 2021.03.23 19.10)
the action 103.698: 2021.03.24 01.38
the action 103.699: 2021.03.24 05.11 (here i also have the obvious absence of numerous actions that were actually done)

the last action 103.810: 2021.03.24 07.06
and even in the very recent 2 hours: there were numerous actions which are totally absent here (above the last action)

(these aspects are slightly complicated for me so please ask more questions in case my info is not very clear)


journal_log_format option
thank you for adding it
from the beginning i was thinking about something like this but didn't pay a substantial attention to such possible approach
the setting adds an important flexibility and enlarges the user friendliness

(at the moment, i failed to find any meaningful use for the <width> setting
but that's not a big deal. maybe i'll look into it later)


i noticed that with these settings in the ini-file:
journal_log=1
journal_log_directory=F:\New-4\Everything-1.5.0.1249a\my-jl
the automatically created txt-file (inside the just created 'my-jl' folder) gets modified literally every second
(as long as 1.5-alpha is opened)
(there's something wrong here. besides that, this is absolutely not the behavior that any hdd/ssd likes, i think)


in the main window of index journal:
a) sort by date doesn't work normally
i mean that sort by date should take into consideration the 'time' as well (but it doesn't)
so currently: sort by date seems to be senseless (both ascending and descending)
because there are cases when, for example:
2021.03.24 06.03 is present before the 2021.03.24 05.46
while later there is another 2021.03.24 06.03 that is placed after the above-mentioned 2021.03.24 05.46

b) the 'time' should have seconds, in my opinion
c) there should be slightly bigger visual delimiter (between date and time) than just 1 space
yes, this is important in the main window (of index journal) too
3 spaces could be a best variant (maybe 2 is normal but certainly not 1)

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Sat Mar 27, 2021 10:57 am

i may be wrong but i don't think that this is the reason
Can you confirm there's a gap of missing events where the item ID is still in sequence.
For example:

Item ID / Date / Action / Filename
100 / 2021.03.24 05.16.33 Delete / foobar.txt
... gap of missing changes for 2 hours ...
101 / 2021.03.24 07.16.33 Delete / foobar2.txt

Is the Item ID consecutive?
Please make sure the action combo is set to all actions and there is no active search.
It could be a monitoring issue which I am still investigating..
the automatically created txt-file (inside the just created 'my-jl' folder) gets modified literally every second
(as long as 1.5-alpha is opened)
(there's something wrong here. besides that, this is absolutely not the behavior that any hdd/ssd likes, i think)
Everything will constantly stream to this log file (WriteFile)
Data is not flushed to disk and should be buffered.

Is the log file constantly reported in your Index Journal as Modified? -If so, it may indicate an issue with closing and reopening of the same file over and over.
a) sort by date doesn't work normally
The date sort is currently the same as the ID sort.
I haven't updated since changing the date to use the USN Journal date.
...Added to my TODO list.
b) the 'time' should have seconds, in my opinion
Added to my TODO list: Use long time format for index journal.

To set Everything to show seconds in the time format:
  • In Everything, type in the following search and press ENTER:

    Code: Select all

    /time_format=HH.mm.ss
Note: this format is used for all times in Everything.

Time Format.
c) there should be slightly bigger visual delimiter (between date and time) than just 1 space
yes, this is important in the main window (of index journal) too
3 spaces could be a best variant (maybe 2 is normal but certainly not 1)
Everything uses your Windows Region and Language settings to format the date and time.
To override your region and language format:
  • In Everything, type in the following search and press ENTER:

    Code: Select all

    /date_format=yyyy.MM.dd  
    Note the double space at the end.
Note: this format is used for all dates in Everything.

Date Format.

Thank you for your suggestions.

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Sun Mar 28, 2021 4:40 am

i'm not sure that the following screenshots help us but nevertheless here they are just in case:
1) the id's 1-3780 are absent for some reason:
2021.03.27   16.45.02.png
2021.03.27 16.45.02.png (190.57 KiB) Viewed 6515 times
2) another example of that not normal order in the date column:
2021.03.27   16.40.22.png
2021.03.27 16.40.22.png (195.98 KiB) Viewed 6515 times
3) those screenshots are from yesterday, while this is the just created screenshot:
2021.03.28   06.57.27.png
2021.03.28 06.57.27.png (187.36 KiB) Viewed 6515 times
Can you confirm there's a gap of missing events where the item ID is still in sequence
it's quite hard to find any useful info (including the 'gap of missing events') because of not adequate sort in the date column
but anyway, it seems that i can confirm there's a gap of missing events


moreover, currently (under such circumstances) there's not much sense in the id column for me (and unfortunately i cannot even disable it)


in case i'm doing something wrong: is there any relatively simple algorithm that i may try to perform?
(so that you got more useful answers) (and so that this whole situation stopped being too complicated for me)

for example:
- download the newest alpha-version, run its exe-file
- disable the 'run in background' setting in the options, close this alpha-version
- open its newly created main ini-file
- make this exact change in 'the line ...' to ...
- make this exact change in 'the other line ...' to ...
- close the main ini-file, load this ET-alpha again, open its index journal, ...

but such algorithm will probably have sense only after some improvements (related to the date column) are implemented:
in some further alpha-version

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Thu Apr 01, 2021 5:43 am

Thank you for your feedback adamantine,

The screenshots are helpful, thank you.

I have added index journal date sorting to Everything 1.5.0.1252a.
Does this version miss any changes?
moreover, currently (under such circumstances) there's not much sense in the id column for me (and unfortunately i cannot even disable it)
You can remove the #id: entry from your log format and resize this column to 1 pixel.
I will consider an option to hide the ID column.
in case i'm doing something wrong: is there any relatively simple algorithm that i may try to perform?
  • Completely exit out of Everything (From the File menu, click Exit).
  • I assume you are not using the Everything Service and you are running Everything as an admin -please correct me if I am wrong
  • Replace your Everything.exe with the latest Everything 1.5 Alpha.
  • Launch Everything.
  • Everything should remember your run-in-background setting.
  • Everything should remember your log-format.
    -Please let me know if this is not the case.
You can quickly opening your Everything.ini from Tools -> Options -> Config
Make any changes
Save and close notepad
Accept the prompt in Everything to reload your config.

Some ini settings you can change directly in the Everything search bar, for example:
  • In Everything, type in the following search and press ENTER:

    Code: Select all

    /journal_log_format=#split:#date:<YYYY.MM.DD   hh.mm.ss>   #filename:      #action:
    
Thank you for your suggestions.

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Sat Apr 03, 2021 8:32 pm

Does this version miss any changes?
the 'file modify' action was certainly absent in index journal once (this action was related to a textual cue-file)
i suppose that more or less the same thing happened more than just once
You can remove the #id: entry from your log format and resize this column to 1 pixel
resizing is not remembered (even within the current session of ET-alpha)
I will consider an option to hide the ID column
this option could be useful
I assume you are not using the Everything Service and you are running Everything as an admin
i'm not using the Everything Service, i'm running Everything as an admin (windows-2003 with only 1 user)
Replace your Everything.exe
i didn't understood this part of your message: i have a regular 1.4-ET and also the lastest 1.5-alpha (1252)
so i'm not sure that i know how to comment these 2 items:
"Everything should remember your run-in-background setting"
"Everything should remember your log-format"
You can quickly opening your Everything.ini from Tools -> Options -> Config
Some ini settings you can change directly in the Everything search bar
i do such things in a more simple and intuitive way: i always edit the ini-file when ET is completely unloaded

are there any exact steps that i must do after getting a new alpha-version? (and before checking if its index journal works normally)

are these ini-lines correct?:
journal=1
journal_max_size=1640873984
journal_log=1
journal_log_directory=

are there any other ini-lines that i should edit? (so that index journal could become a substitution for that cenvo-rox program)


(added later):
index journal is sorted by date. but i noticed that content of the file index-journal-2021-04-04-1.5a.txt has wrong order of lines
look at the order of time (15.24.50 after 20.27.03) (15.24.42 after 20.27.00) in the following example:
2465   2021.04.04   20.27.03   C:\Program Files\Cenvo-Rox\data\cr_file_events.cdb-journal      Create
2466   2021.04.04   20.27.03   C:\Program Files\Cenvo-Rox\data\cr_file_events.cdb-journal      Delete
2467   2021.04.04   15.24.50   D:\Images\2021.04.04\2021.04.04   15.24.50.png      Create
2468   2021.04.04   15.25.00   D:\Images\2021.04.04\2021.04.04   15.25.00.png      Create
...
3313   2021.04.04   20.26.50   D:\Images\2021.04.04\2021.04.04   20.26.50.png      Create
3314   2021.04.04   20.27.00   D:\Images\2021.04.04\2021.04.04   20.27.00.png      Create
3315   2021.04.04   15.24.42   F:\New-4\Everything-1.5.0.1252a\Everything-1.5a.db.tmp      Create
3316   2021.04.04   15.24.42   F:\New-4\Everything-1.5.0.1252a\Everything-1.5a.db      Delete
3317   2021.04.04   15.24.42   F:\New-4\Everything-1.5.0.1252a\Everything-1.5a.db.tmp      Rename From
3317   2021.04.04   15.24.42   F:\New-4\Everything-1.5.0.1252a\Everything-1.5a.db      Rename To
3318   2021.04.04   15.24.42   F:\New-4\Everything-1.5.0.1252a\index-journal-2021-04-04-1.5a.txt      Modify

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Wed Apr 07, 2021 6:51 pm

this is the very fresh example of a situation when the 'file modify' action is absent many times (the action is related to a textual file 1.cue):

1) everything is good in cenvo-rox program (all 11 actually happened 'file modify' actions are present):

18:34:00, 18:34:00,   20:22:19, 20:22:19,   20:26:24, 20:26:24,   20:30:39, 20:30:39
20:34:39, 20:34:39,   20:40:25, 20:40:25,   20:45:08, 20:45:08,   20:53:34, 20:53:34
20:57:50, 20:57:50,   21:02:04, 21:02:04,   21:12:43, 21:12:43

i don't know why all these actions are duplicated (but i remember: this is not very unusual behavior for cenvo-rox)
at least these actions are not lost (unlike ET-alpha)


2) while index journal only has this 1 line (same file 1.cue):
18:34:00   (file modify)

void
Site Admin
Posts: 6462
Joined: Fri Oct 16, 2009 11:31 pm

Re: Index Journal Suggestions

Post by void » Fri Apr 09, 2021 6:37 am

Thank you for your reply adamantine,
resizing is not remembered (even within the current session of ET-alpha)
Everything will reset the column width to the default if you set the column width to 0 pixels to avoid columns from being "lost".
Instead of resizing the column to 0-pixels, try resizing the column to 1-pixel.
Does Everything remember the almost invisible column width?
i didn't understood this part of your message: i have a regular 1.4-ET and also the lastest 1.5-alpha (1252)
Are the Everything.exe 1.4 and Everything.exe 1.5 files in the same folder? -If so, please try moving them to separate folders. Otherwise, they might fight over the same Everything.ini.

-Please make sure Store settings and data is disabled for Everything 1.5 under Tools -> Options -> General.
-This will store your settings in Everything-1.5a.ini in the same location as your Everything 1.5 exe.
are there any exact steps that i must do after getting a new alpha-version?
Since you don't have the service installed, all you need to do is make sure Everything is not running (Right click the Everything System tray icon and click Exit) and overwrite your Everything.exe with the latest version.
are these ini-lines correct?:
journal=1
journal_max_size=1640873984
journal_log=1
journal_log_directory=
Yes, these are fine.
The journal_max_size is a little large (1564 MB).
Set journal_max_size to the maximum journal size in bytes.
For example, to set journal_max_size to 100MB:
journal_max_size=104857600
are there any other ini-lines that i should edit? (so that index journal could become a substitution for that cenvo-rox program)
Just the journal_log_format setting at this stage.
I plan to add more in a future release.
index journal is sorted by date. but i noticed that content of the file index-journal-2021-04-04-1.5a.txt has wrong order of lines
The lines written to the log are currently always in ID order.
this is the very fresh example of a situation when the 'file modify' action is absent many times (the action is related to a textual file 1.cue):
Is there a program running that has a handle open to this file?
-If so, Everything will only receive one "modify" notification when the file is first modified and another when all the file handles are closed.

Was Everything running in the background during this time?
If not, Everything may only log the last "modify" action, ignoring the earlier ones. (this is probably not the case here as only the earlier time is logged)

adamantine
Posts: 72
Joined: Mon Jan 09, 2012 10:56 am
Location: kiev

Re: Index Journal Suggestions

Post by adamantine » Fri Apr 09, 2021 11:57 am

try resizing the column to 1-pixel ... Does Everything remember the almost invisible column width?
i tried 1 pixel - it works (but looks ugly)
it's better for me to wait for the implementation of this: "I will consider an option to hide the ID column"

moreover, the current situation (in index journal) leads me to a thought that the 'export' button may become useless for me too
because ET-alpha automatically adds the actions to a file like this: index-journal-2021-04-09-1.5a.txt (such file is created once a date)
which seems to be more or less appropriate behavior (at least at this stage)
i have not got used to it yet. so i will continue to think about it (and experiment with it)

and seemingly, the very important thing right now is to wait for the change in this aspect:
"The lines written to the log are currently always in ID order"
i hope that it's possible to change this approach to a better variant:
the lines written to the log are always in the order which is present in main window of index journal
(but the order by date could be quite enough: when the log starts from the old actions and continues with the newer actions)
Are the Everything.exe 1.4 and Everything.exe 1.5 files in the same folder?
of course no (they are even in the different drives)
make sure Store settings and data is disabled for Everything 1.5 under Tools -> Options -> General
it is always disabled
make sure Everything is not running
i always try to pay attention to things like this
Right click the Everything System tray icon
i don't have the system tray (thanks to the 'group policy'). and it feels great :)

a bit of off-topic: i cannot find a reliable (and visually ideal) way to get rid of the windows start button
currently i use the 'StartKiller' utility which does its job wonderfully (the 3 previously uploaded screen-shots show my task-bar completely)
but StartKiller requires an exe-process to be constantly running
i wish i had a (sophisticated) piece of the ahk-code to achieve the same effect (but without usage of this utility)
journal_max_size=1640873984 ... The journal_max_size is a little large (1564 MB)
after i experimentally entered something like 999999999999, ET-alpha itself gave me this number: 1640873984
so i thought that this is the largest number that's possible
this aspect is not that important for me but anyway: are you sure that 1640873984 is not the largest option?
Is there a program running that has a handle open to this file?
it seems that the answer is 'yes'
the program is notepad2, and that 1.cue file was constantly opened
during several hours the workflow was:
edit something in the file, save it, the file stays opened, (slightly later) edit something in the file, save it...

so i guess that cenvo-rox can handle (understand) such modifications, while ET-alpha can't do it
is it because cenvo-rox has its service installed? is it because cenvo-rox has its process constantly running? (CRFileMonSvr.exe)

but these questions are not that essential to me because it looks like i will have to get used to this:
"Everything will only receive one "modify" notification when the file is first modified"
or to forget about a dream to substitute the cenvo-rox (in case i eventually fail to get used to such numerous new things/changes)


at first, these your words ("and another when all the file handles are closed") were not clear to me
because in that '11 actions' example: there was just 1 action (not 2 actions) related to the 1.cue file (in ET-alpha)
but later i realized that the file is still not closed
so after it will be closed, i may expect the confusing observation:
the ET-alpha displays the just added 'modify' action (for a file which was actually modified several days ago)
it looks like i will have to live with such behavior of ET, or forget about the whole idea of substitution (cenvo-rox > ET 1.5)
since i really don't want to forget, i plan to continue considering all the possibilities to switch to ET 1.5 (when it becomes beta-version)
Was Everything running in the background during this time?
no

Post Reply