How is Everything so efficient?

Plug-in and third party software discussion.
Post Reply
planeW
Posts: 2
Joined: Fri Mar 19, 2021 6:56 am

How is Everything so efficient?

Post by planeW »

Hello!

I've been using Everything for so many years now and as a software developer it always felt like this is too good to be true and its developer must be cheating somehow ;)

On the one hand Everything is incredibly fast at searching, which indicate that the in-memory data structure for the database is likely a sorted array (or multiple array chunks) for its fast lookup times, cache friendliness and great scaling with multiple threads. Maybe the array items also store the index or memory address of their parent folders, for fast path building and to save memory.

But on the other hand Everything also seems to be highly dynamic, in that there are constantly files and folders being removed or added to the database, which wouldn't work that well with an array, since inserting items sorted or deleting them involves a lot of moving memory around and presumably updating lots of references to items.

I also thought that maybe those changes would be stored in different arrays, like a database diff, but I suppose this would impact the search performance too much.

So does someone know how Everything is so fast and dynamic at the same time? Because I'm always hesitant to use arrays with lots of entries once adding and removing items becomes a common operation. Or maybe I'm just underestimating how fast computers nowadays are.

Thanks!
rollsch
Posts: 8
Joined: Wed Oct 05, 2022 8:08 pm

Re: How is Everything so efficient?

Post by rollsch »

Would love to know more about this. The program is incredibly fast.
Thy Grand Voidinesss
Posts: 567
Joined: Wed Jun 01, 2022 5:01 pm

Re: How is Everything so efficient?

Post by Thy Grand Voidinesss »

Even accessing its Options is instant [which when compare to FreeCommander's Settings makes it look lightspeed fast]
defc
Posts: 14
Joined: Fri May 29, 2020 9:31 pm

Re: How is Everything so efficient?

Post by defc »

I think one of the keys is this is still in c++.

I actually don't even know if it uses an in-memory db or custom data structures, they must be getting flushed to disk pretty frequently too, plus it has to keep updating them with Windows fs events. I bet the current view stores a list a of affected nodes and then if there is an update, it can check if any of those nodes are impacted so they can be refreshed.

This kind of efficient low level programming by a dedicated dev is not very common these days with everyone having moved on to JS/Python/Web. Another example of an efficient app I use daily is xplorer2 (zabkat.com) which is a must have as a file explorer. I believe there is an integration too.
ChrisGreaves
Posts: 602
Joined: Wed Jan 05, 2022 9:29 pm

Re: How is Everything so efficient?

Post by ChrisGreaves »

planeW wrote: Fri Mar 19, 2021 7:23 am... it always felt like this is too good to be true and its developer must be cheating somehow ...
I agree. I wrote myself a content-search of MSWord documents, and wrote it in Word97/VBA. Blew (back then) Windows File Explorer out of the water.

I think that David, the developer is very bright and is a very good programmer.

But I suspect (and do not know) that the main advantage comes from NOT inheriting core code from the days of MSDOS and 1.2MB 5 1/4" floppy disk ideas. This Article is a starting point, but perhaps someone with a better memory than I can dig out the folklore that Bill Gates once said something like "ordinary people won't understand sub-directories".

I have seen enough corporations that carry a basic, biut stale, philosophy for dozens of years.
I suspect that some core ideas have held true right through the development of Windows, and perhaps insistence on backwards compatibility acts as a hobble on progress.

I suspect that David has not been hindered by old ideas from the 1980s; he has had to deal with those old ideas, as hurdles, but they have not been an obstacle to his progress.

A recent example is the introduction of SSDs. Out the door goes defragmentation. In the door comes even faster content searching (for all applications, not just Everything (and as I write that I am shaken by just how weird that sounds!))

I hope that helps us travel along the magic path ...
Cheers, Chris
Last edited by ChrisGreaves on Wed Feb 28, 2024 6:22 pm, edited 1 time in total.
NotNull
Posts: 5167
Joined: Wed May 24, 2017 9:22 pm

Re: How is Everything so efficient?

Post by NotNull »

Parts of Everything are written in assembler.
meteorquake
Posts: 367
Joined: Thu Dec 15, 2016 9:44 pm

Re: How is Everything so efficient?

Post by meteorquake »

Aside from the points mentioned, my impression is that a lot of slowness and bloat in the programming world comes from using 3rd-party libraries and frameworks particularly as regards speed for tasks that are repeated rapidly. In most cases not only is coding such things specifically going to lead to much faster execution, but in many cases it's much quicker to do so than to learn how the library works, and of course more fun. d
rollsch
Posts: 8
Joined: Wed Oct 05, 2022 8:08 pm

Re: How is Everything so efficient?

Post by rollsch »

When you see how convoluted and bloated modern frameworks it's easy to see how some asm and c code could be phenomenally fast especially if directly going to the kernel Apis at the lowest possible level.

I'd love to see some of the coding tricks David is most proud of posted here. I'm sure everyone could learn a thing or two.
void
Developer
Posts: 15096
Joined: Fri Oct 16, 2009 11:31 pm

Re: How is Everything so efficient?

Post by void »

Everything 1.3 and earlier had a lot of assembly.
Everything 1.4 is 100% C
Everything 1.5 is 100% C

A lot of the performance improvements in Everything 1.5 come from tight loops and keeping important things in CPU cache.
Thy Grand Voidinesss
Posts: 567
Joined: Wed Jun 01, 2022 5:01 pm

Re: How is Everything so efficient?

Post by Thy Grand Voidinesss »

ChrisGreaves wrote: Wed Feb 28, 2024 12:12 pm [...]
core code from the days of MSDOS and 1.2MB 5 1/4" floppy disk ideas. This Article
[...]
How archaic and annoyingly constraining this looks like

I must have totally forgot about such limitations from the times of me being a kid - or never had run into those limits; with the latter one being more feasible as hardware existence determines digital consciousness
Last edited by Thy Grand Voidinesss on Sat Mar 02, 2024 11:18 am, edited 1 time in total.
ChrisGreaves
Posts: 602
Joined: Wed Jan 05, 2022 9:29 pm

Re: How is Everything so efficient?

Post by ChrisGreaves »

Thy Grand Voidinesss wrote: Fri Mar 01, 2024 9:24 amI must have totally forgot about such limitations ...
limitations? Limitations ?!!???
Good Grief what on earth are you talking about? :twisted:
In The Good Old Days there were NO folders, just a large leather briefcase that could hold two boxes of punched cards (so about 4,000 cards)
In The Good Old Days, using a column as two six-bit bytes, we could fit 160 bytes on a punched card, so 640,000 bytes on two boxes of cards, so think about ten pounds. A 1.44 MB floppy weighed what? Half an ounce?
"Long filename support" we had, as long as you could hold a 2,000 card deck tight enough to write across the top with a marker pen.
Sigh!
Kids nowadays! :lol: :lol:
Cheers, Chris
NotNull
Posts: 5167
Joined: Wed May 24, 2017 9:22 pm

Re: How is Everything so efficient?

Post by NotNull »

ChrisGreaves wrote: Fri Mar 01, 2024 1:18 pm just a large leather briefcase that could hold two boxes of punched cards
A briefcase. Now that's a good idea. The times someone tripped and dropped the box of punched cards and then had to put the code in the right order again ... Even with markings on the side that took a while.

1.44 MB disks were designed to fit in a chest pocket of a shirt. Problem solved.
Post Reply