Slow "Querying" while compiling something

Found a bug in "Everything"? report it here
Post Reply
dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Slow "Querying" while compiling something

Post by dzenanz » Wed May 15, 2019 2:47 pm

I love Everything and use it as my search too. But if I have something compiling and it uses 100% of the CPU, Everything's search slows down to a crawl. Also scrolling is slow, if not impossible. As I write computer code for a living, this happens somewhat frequently.

Compiler executable (cl.exe) has "Below normal" priority, while Everything has "Above normal". This is enforced by Process Hacker 2. But that does not help much. I am using Version 1.4.1.935 (x64). Is this a known problem? Is there a workaround?

therube
Posts: 2197
Joined: Thu Sep 03, 2009 6:48 pm

Re: Slow "Querying" while compiling something

Post by therube » Wed May 15, 2019 4:32 pm

How many files does the compiler throw around in doing its' work?
Thinking that if it is very many, in a very short period of time, that number could be overwhelming Everything... ?
Probably some settings, unknown to me, that could be adjusted, to try to help compensate... ?

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Wed May 15, 2019 7:56 pm

I didn't think of the number of files compiler touches. I was thinking "why doesn't Everything fight for CPU time better?"

While compiler does throw around many files by human standards (at least dozens, probably hundreds, possibly low thousands of files per second), it should not be too much by computer standards.

How would performance for this case be affected by disabling "Index date modified" and/or "Index recent changes"? Is either significant?

NotNull
Posts: 1179
Joined: Wed May 24, 2017 9:22 pm

Re: Slow "Querying" while compiling something

Post by NotNull » Wed May 15, 2019 8:45 pm

dzenanz wrote:
Wed May 15, 2019 7:56 pm
I didn't think of the number of files compiler touches. I was thinking "why doesn't Everything fight for CPU time better?"
That will be more on how Windows handles that (= not very good). Example: yesterday my system had 100% CPU load due to an update, although I configured the responsible TrustedInstaller routine to run at the lowest priority. Result: unresponsive system.
JUst a thought: does CL.exe start other processes that are *not* running with lowest priority?
While compiler does throw around many files by human standards (at least dozens, probably hundreds, possibly low thousands of files per second), it should not be too much by computer standards.
I don't suspect that is the cause either. But to be sure, you can (temporary) disable "scanning for new files":
In the Everything search bar, type
/monitor_pause
(followed by ENTER)
To resume monitoring, type
/monitor_resume
(followed by ENTER)
(the files that were missed in that period will be picked up automatically)

How would performance for this case be affected by disabling "Index date modified" and/or "Index recent changes"? Is either significant?
From what I know about the inner workings of Everything (= not much), most CPU power is used when you have a complex filter active (or exclusions you defined for your indexes) and for sorting the results.
I doubt if index date/recent changes has much influence on CPU usage.

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Thu May 16, 2019 1:08 pm

NotNull wrote:
Wed May 15, 2019 8:45 pm
does CL.exe start other processes that are *not* running with lowest priority?
No. devenv.exe (Visual Studio process) launches MSBuild.exe, which launches cl.exe. cl is the leaf which does the work and consumes CPU time. But I also gave lower priority to MSBuild.

void
Site Admin
Posts: 4667
Joined: Fri Oct 16, 2009 11:31 pm

Re: Slow "Querying" while compiling something

Post by void » Fri May 17, 2019 8:25 am

Do you observe the same issue with Everything 1.4.1.895?

I wonder if this is related to the newly added Sleep(0) which will make Everything give up its time slice every so often, perhaps too much.

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Fri May 17, 2019 1:28 pm

I disabled "Index recent changes" earlier, it did not seem to have had much of an effect.

Now, I downloaded Everything 1.4.1.895 (it is not listed, I had to type the name), and I will let you know once I had a chance to observe it under load.

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Fri May 17, 2019 2:02 pm

It does not look like version 1.4.1.895 made much of a difference. :(

void
Site Admin
Posts: 4667
Joined: Fri Oct 16, 2009 11:31 pm

Re: Slow "Querying" while compiling something

Post by void » Mon May 20, 2019 3:03 pm

Thanks for checking Everything 1.4.1.895.

Does Everything show any CPU usage in the Task Manager when compiling? -If so how much CPU Usage %?
Are you running into any low memory conditions?
What does Everything's page faults look like in the Task Manager? -It could be a zero soft page fault performance issue.
What type of CPU do you have?

Note:
Everything will show Querying... in status bar after a search takes longer than 250 milliseconds.

therube
Posts: 2197
Joined: Thu Sep 03, 2009 6:48 pm

Re: Slow "Querying" while compiling something

Post by therube » Mon May 20, 2019 7:41 pm

(Process Hacker, double-click on Everything.exe, Performance [tab], will show a running graph of [Everything] CPU history. Similarly, Process Explorer.)

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Tue May 21, 2019 7:31 pm

I have 32GB or RAM and 1GB swap file. If my computer runs out of memory, I get allocation errors not page faults.

During most recent incident, about half of RAM was free, page fault count for Everything barely budged, and CPU use was generally low, with some spikes going to about 50% of CPU time.
Screenshot 2019-05-21 15.25.05.png
You do not have the required permissions to view the files attached to this post.

void
Site Admin
Posts: 4667
Joined: Fri Oct 16, 2009 11:31 pm

Re: Slow "Querying" while compiling something

Post by void » Thu May 23, 2019 3:39 am

Does increasing Everything's base priority help? try setting it to high?

What type of CPU do you have?

dzenanz
Posts: 8
Joined: Sat Jun 10, 2017 3:55 pm

Re: Slow "Querying" while compiling something

Post by dzenanz » Thu May 23, 2019 9:25 pm

AMD Ryzen 7 PRO 1700 Eight-Core Processor (with hyper-threading enabled, so appears as 16 cores).

Giving Everything high CPU and I/O priority did not matter. It still sometimes queries for many seconds, in this instance for more than a minute.
Screenshot 2019-05-23 17.16.35.png
Screenshot 2019-05-23 17.15.03.png
You do not have the required permissions to view the files attached to this post.

NotNull
Posts: 1179
Joined: Wed May 24, 2017 9:22 pm

Re: Slow "Querying" while compiling something

Post by NotNull » Sat May 25, 2019 7:59 pm

Did the /monitor_pause made any difference?

Post Reply