everything to store file GUIDs in its database

Have a suggestion for "Everything"? Please post it here.
Post Reply
echo21
Posts: 4
Joined: Sat Feb 09, 2019 12:55 pm

everything to store file GUIDs in its database

Post by echo21 » Sat Feb 09, 2019 3:00 pm

I think conventional hierachical file systems as NTFS work with a file referencing concept that is sub-optimal by design: every file can be referenced only by its path name. So if you make links (e.g. in a word document, a local html file, an *.lnk file on your disk, etc. ) to a NTFS file there is a high risk that the links get invalid if only you move the files from one directory to another. Try to think of this. This is a major misconception of e.g. NTFS. If you don't think so, please don't read further...

Ok, nice, it seems you agree ;) .

So given that, the only way out of this mess is not to link to files' path names anymore but to link to files via their everlasting unique identifier. Then you can move your file around and the links keep all valid and working! I believe that this (new) thinking is not only a nice but unrealistic dream. There are only 2 steps to go. The second step would be to create working links to such unique identifiers once we have them. But let's defer this second step for now (I am very busy at the moment but I will follow up later on).

Step #1, getting a unique identifier for each of the files in a file system seems solved by Microsoft's NTFS specs: read this blog post https://blogs.msdn.microsoft.com/oldnew ... 0/?p=11363 . After this lengthy introduction here comes my feature request for everything: Can you please also grab the 128 bit GUID of every file, store it in the everything database, and to add a column "GUID" to show up in everything?

After implementation of this GUID feature, everything could support a step #2a: the everything API could be extended by a new function that returns a pathname of a file for a given file GUID.

What do you think?

Best regards

echo21

NotNull
Posts: 1115
Joined: Wed May 24, 2017 9:22 pm

Re: everything to store file GUIDs in its database

Post by NotNull » Sat Feb 09, 2019 5:02 pm

echo21 wrote:
Sat Feb 09, 2019 3:00 pm
What do you think?
I think you gave this quite some thought and I liked reading it!


echo21 wrote:
Sat Feb 09, 2019 3:00 pm
Can you please also grab the 128 bit GUID of every file, store it in the everything database, and to add a column "GUID" to show up in everything?
Everything already does so (as far as I know). The hierarchy (= folder structure) is based on that: file A has parent B and B has parent C, etc. Those "GUID's" are used for that.
You can use the frn: function for that. Let's say you have a file c:\Windows\notepad.exe. It's FileID can be found withe the command

Code: Select all

fsutil.exe file queryfileid c:\windows\notepad.exe
( 0x0000000000000000000e0000000237b1 on my system).
Searching for
frn:0x0000000000000000000e0000000237b1
in Everything returns c:\Windows\notepad.exe (or one of the other hardlinks to this inode)


echo21 wrote:
Sat Feb 09, 2019 3:00 pm
So given that, the only way out of this mess is not to link to files' path names anymore but to link to files via their everlasting unique identifier. Then you can move your file around and the links keep all valid and working!
Microsoft already stole your idea! It's called Distributed Link Tracking and works approximately like you describe :) It exists since (at least) Windows XP.

echo21
Posts: 4
Joined: Sat Feb 09, 2019 12:55 pm

Re: everything to store file GUIDs in its database

Post by echo21 » Sat Feb 09, 2019 5:59 pm

Hi @NotNull , thanks a lot for your reply (I haven't found a "thank you" button to click in this forum so far)!
NotNull wrote:
Sat Feb 09, 2019 5:02 pm
Everything already does so (as far as I know). The hierarchy (= folder structure) is based on that: file A has parent B and B has parent C, etc. Those "GUID's" are used for that.
You can use the frn: function for that. Let's say you have a file c:\Windows\notepad.exe. It's FileID can be found withe the command

Code: Select all

fsutil.exe file queryfileid c:\windows\notepad.exe
( 0x0000000000000000000e0000000237b1 on my system).
Searching for
frn:0x0000000000000000000e0000000237b1
in Everything returns c:\Windows\notepad.exe (or one of the other hardlinks to this inode)
Excellent!!! So I'd like to revise my feature request to "everything provides a (hidden but un-hideable) column FileID/GUID"
NotNull wrote:
Sat Feb 09, 2019 5:02 pm
Microsoft already stole your idea! It's called Distributed Link Tracking and works approximately like you describe :) It exists since (at least) Windows XP.
It's more like that you are preventing me from wanting to re-invent the wheel (ok, I didn't know that that wheel exists already :) ) .

void
Site Admin
Posts: 4597
Joined: Fri Oct 16, 2009 11:31 pm

Re: everything to store file GUIDs in its database

Post by void » Sun Feb 10, 2019 1:18 am

Keep in mind the frn: function in Everything 1.4. only shows the first hit.

The next major version of Everything will show all hits.
The next major version of Everything will also have an optional File FRN column.

I already index the FRN for folders, this FRN is needed to look up files in the USN Change Journal.
The FRNs will be gathered as needed, I would prefer not to index file FRNs as that will double the memory usage of Everything.
However, I am consider the option to do so.

echo21
Posts: 4
Joined: Sat Feb 09, 2019 12:55 pm

Re: everything to store file GUIDs in its database

Post by echo21 » Sun Feb 10, 2019 10:48 pm

Hi @void, thank you very much for your reply! And of cause for making the world better by providing everything!
void wrote:
Sun Feb 10, 2019 1:18 am
Keep in mind the frn: function in Everything 1.4. only shows the first hit.

The next major version of Everything will show all hits.
Oops, I am confused, how can 1 FRN reference more than 1 object (file)? This is what a unique identifier shouldn't do, right? So isn't the FRN a unique identifier?
void wrote:
Sun Feb 10, 2019 1:18 am
The next major version of Everything will also have an optional File FRN column.
Great news, thanks in advance!
void wrote:
Sun Feb 10, 2019 1:18 am
I already index the FRN for folders, this FRN is needed to look up files in the USN Change Journal.
The FRNs will be gathered as needed, I would prefer not to index file FRNs as that will double the memory usage of Everything.
However, I am consider the option to do so.
Yes, an everything FRN indexing option (default off) would be nice...

NotNull
Posts: 1115
Joined: Wed May 24, 2017 9:22 pm

Re: everything to store file GUIDs in its database

Post by NotNull » Mon Feb 11, 2019 3:34 pm

echo21 wrote:
Sun Feb 10, 2019 10:48 pm
Oops, I am confused, how can 1 FRN reference more than 1 object (file)? This is what a unique identifier shouldn't do, right? So isn't the FRN a unique identifier?
It is unique for every file on disk. But it can have multiple 'aliases'. Those are called hardlinks. Your Windows folder is full of those.

echo21
Posts: 4
Joined: Sat Feb 09, 2019 12:55 pm

Re: everything to store file GUIDs in its database

Post by echo21 » Mon Feb 11, 2019 3:39 pm

NotNull wrote:
Mon Feb 11, 2019 3:34 pm
It is unique for every file on disk. But it can have multiple 'aliases'. Those are called hardlinks. Your Windows folder is full of those.
Makes sense to me now. Thanks for clarification!

Post Reply