Run indexing process as administrator

[void]

Everything 1.5.0.1319a adds a Run indexing process as administrator option.

When enabled, Everything will launch an additional "Everything" process as an administrator to index and monitor NTFS volumes.

The main UI process will keep running as a standard user.

Installing the Everything Service is still the recommend method for indexing and monitoring NTFS volumes.

Run indexing process as administrator might be useful for the portable version of Everything if you wish to avoid installing the Everything Service or wish to avoid running the main Everything UI process as an administrator.



The Everything Service is not needed if you run the indexing process as administrator.
Run as administrator is not needed if you run the indexing process as administrator.



To enable Run indexing process as administrator:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Uncheck Everything Service.
• Check Run indexing process as administrator.
• Uncheck Run as administrator.
• Click OK.

The indexing process will exit when you exit the main Everything UI process.
Disable the exit_indexing_process_on_exit ini setting to keep the indexing process running when exiting the main Everything UI process.



When Run indexing process as administrator is enabled and you launch Everything for the first time you may see the User Account Control prompt.
Install the Everything Service if you would like to avoid this prompt.

To install the Everything Service:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Everything Service.
• Uncheck Run indexing process as administrator.
• Uncheck Run as administrator.
• Click OK.

[tuska]

2void
Thank you very much for this new option! 

Could you please additionally provide a command line option, e.g. -admin-service?

Examples:

Code: Select all

Path\to\Everything64.exe -admin-service
L:\Tools\Everything\Everything64.exe -admin-service
L:\Tools\Everything_PRIVATE\Everything64.exe -instance "PRIVATE" -admin-service

[void]

Thank you for your feedback tuksa,

I will add a -admin-service command line option.
Thank you for the suggestion.



For more technical details, enabling Run indexing process as administrator (admin service) will:

• Launch Everything.exe with the -svc command line option.
• Stop this process when you exit Everything.

The -svc command line is the service entry.
If run as a normal program, Everything will relaunch as an admin process and host the Everything Service in session 1 (instead of session 0).



You only need to run Everything.exe -svc once for all Everything instances.
Run indexing process as administrator will do nothing if there is already an Everything service running.

[Stamimail]

Run indexing process as administrator might be useful for the portable version of Everything if you wish to avoid installing the Everything Service or wish to avoid running Everything as an administrator.

What is it good for? What's the case of the user?

[therube]

The service itself is running with admin privileges.
But the user GUI is running at a user level.

If you ran the "entire" Everything, As Administrator, both the service AND the user GUI would be running at an admin level.
(Likewise, anything spawned from the GUI, like your media player or text editor or... also would have admin privileges.)

(And generally, running a program at an admin level is not [as] safe as running at a user level.)

[void]

To help clarify, I changed:
"wish to avoid running Everything as an administrator"
to:
"wish to avoid running the main Everything UI process as an administrator"

[NotNull]

I will add a -admin-service command line option.

I find the admin-service option name confusing as there is no service involved in this case.

It's not that I have a much better suggestion (maybe -index-as-admin) ..

[void]

Thanks for the feedback NotNull,

I'll change the name to index as admin.
Index as admin is much clearer to the end user.


I did try to avoid using "service" in the UI.

Technically, Everything is running the Everything Service as a normal admin process in session 1.
So, I shortened this to admin-service internally.

[raccoon]

1. I would also like to see the Everything Service terminate when the main Everything process is exited. This is consistent with other software that uses registered services. It would also make updating alphas/betas save the step of terminating the Service manually. Or is this service necessary to monitor NTFS journal changes while Everything is closed?

2. Make these features use radio buttons instead of check boxes in the settings UI, so the uninitiated expert user doesn't go ticking all these options needlessly. Do allow them to be set together via INI and /command but not UI interface, and continue to display multiple radio buttons ticked if Everything is Run As Admin externally (ie, via Shortcut).

3. Why not call them "Indexing Process" as Service or as Administrator, and then *actually* utilize this process for indexing with signals sent to the main UI to simply refresh database from file stream (shared memory) when ready. This would/could resolve a lot of latency hang issues with the main UI thread while accessing sleeping drives, network paths and deep content scanning -- it's all done by a background process instead.

[void]

Thanks for the feedback raccoon,

1. I would also like to see the Everything Service terminate when the main Everything process is exited. This is consistent with other software that uses registered services. It would also make updating alphas/betas save the step of terminating the Service manually. Or is this service necessary to monitor NTFS journal changes while Everything is closed?

The Everything service can be stopped at any time.
Everything will resume reading the USN Journal the next time the Everything Service is started.
No changes will be missed.

I will consider an option to stop the service on Everything exit and restart the service on Everything startup.
Thanks for the suggestion.



(Consider running a script to start the Everything service, Everything search client, wait for the client to exit and stop the Everything service)

Start Everything Service as a standard user:
https://stackoverflow.com/questions/4436558/start-stop-a-windows-service-from-a-non-administrator-user-account#answer-18856461
(I will look into making the installer do this)



The Everything Service can be accessed with multiple instances.
I would like the default to be to keep the Everything service running.

2. Make these features use radio buttons instead of check boxes in the settings UI, so the uninitiated expert user doesn't go ticking all these options needlessly. Do allow them to be set together via INI and /command but not UI interface, and continue to display multiple radio buttons ticked if Everything is Run As Admin externally (ie, via Shortcut).

In the installer and accessed denied dialog I will use radio buttons.
I would like to avoid radio buttons in the Options window as all 3 options are separate and all 3 options can be enabled / disabled.
Generally you will only want one checked and I am looking into showing a warning dialog when 2 or more are checked. (eg: are you sure you want to install the service and run as admin? -only one is required... YES/NO/CANCEL)

3. Why not call them "Indexing Process" as Service or as Administrator, and then *actually* utilize this process for indexing with signals sent to the main UI to simply refresh database from file stream (shared memory) when ready. This would/could resolve a lot of latency hang issues with the main UI thread while accessing sleeping drives, network paths and deep content scanning -- it's all done by a background process instead.

Yes, I am not sure about calling it just "index as admin".
I would really like to emphasize that it is another process.
"index as admin" sounds like it is a simple switch when it's really launching a whole new process.



I will consider calling them "Indexing Process" as Service or as Administrator.
Thanks for the suggestion.
"Run as administrator" doesn't require either of these.

Fun fact, I almost removed the "Run as administrator" option entirely.
I'm leaving it in as some admin users will find it useful..


I will look into moving more of the property gathering functions to the Everything Service / Indexing process in a future version of Everything.
Thanks again for the suggestions.

[NotNull]

Start Everything Service as a standard user:
https://stackoverflow.com/questions/443 ... r-18856461

Or: Start/stop Everything service as a restricted user
(You likely have to change the line set Service=Everything with set Service="Everything (1.5a)" to make it work on 1.5.)

[void]

Thanks for your post NotNull,

The next alpha update will allow standard users to start and stop the Everything Service:
Everything will automatically start the Everything service on startup. (if it is installed and not running)
Everything will automatically start the Everything service when attempting to connect to the Everything Service. (if it is installed and not running)
There will be a new option to stop the Everything service on exit.


I'll have an update soon.

[therube]

1. I would also like to see the Everything Service terminate when the main Everything process is exited. This is consistent with other software that uses registered services.

Any benefit to or not to do this?
(Other then the fact that the Everything Service wouldn't be running all the time.
But then, who would ever exit the main Everything process .)

It would also make updating alphas/betas save the step of terminating the Service manually.

AFAIK, you don't have to do that.
(I happen to do that - as a matter of course, as I have always done that since day 1.)


Note that without Everything Service, ES (Everything Command-line Interface) won't work.
(Error 8: Everything IPC window not found. Please make sure Everything is running.)

[NotNull]

It would also make updating alphas/betas save the step of terminating the Service manually.

AFAIK, you don't have to do that.

Huh .. just tested this with 1.5 and I can indeed rename the exe when only the service is runnning.
I could have sworn that this was not possible in 1.4 ... Thanks for the eye-opener!

Note that without Everything Service, ES (Everything Command-line Interface) won't work.
(Error 8: Everything IPC window not found. Please make sure Everything is running.)

The Everything service doesn't need to be running for that. As long as the database manager (it's in the system tray) is running, you're good.
(you might have only EFU files in your Everything database; no servce needed in that case)


I don't see a need to stop the Everything Service in-between. It is not doing anything by itself. Only when the 'regular' Everything indexing process orders a "go fetch" command to the service, the service will do something.
(at least: that is how I understand the mechanics)

So when no regular Everything is running, the service is idling. It hardly uses any memory too.

[void]

Everything 1.5.0.1320a renames "admin-service" to "index-as-admin".

admin_service ini setting is replaced with index_as_admin
exit_admin_service_on_exit ini setting is replaced with exit_indexing_process_on_exit

You might see a prompt to enable "Run indexing process as administrator" when running 1320 for the first time.
Even when "Run indexing process as administrator" was previously enabled.



Everything 1.5.0.1320a will experiment with having "index as admin" the default option for the portable version.

If you want to avoid the UAC prompt on each launch, install the Everything Service.

To install the Everything Service:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Everything Service.
• Uncheck Run indexing process as administrator.
• Uncheck Run as administrator.
• Click OK.

The Everything Service can now be started and stopped by a standard user.
Running Everything will automatically start the Everything service if it is installed.
Everything will automatically start the Everything service if it is installed when attempting to connect to the service.



Everything 1.5.0.1320a adds a stop_service_on_exit ini setting.

Enabling will stop the Everything service when you exit the main Everything UI process.

To enable stop_service_on_exit:

• In Everything, copy and paste the following into your Everything search box:
  /stop_service_on_exit=1
• Press ENTER in your Everything search box.
• If successful, stop_service_on_exit=1 is shown in the status bar for a few seconds.

Everything 1.5.0.1320a adds the following command line options:

-enable-index-as-admin
-disable-index-as-admin
Enable or disable "Run indexing process as administrator".
Returns immediately.

-index-as-admin
Run an additional indexing process as administrator.



Everything 1.5.0.1320a adds the following search commands:

about:access-denied
This might be useful to manually change between the Everything service and the "Run indexing process as administrator" option.

[raccoon]

When starting 1321a, the status bar reports "Waiting for Everything Service..." for a pretty long time after startup. This is on first launch after upgrading, as well as subsequent launches when the Everything Service is not started. No other activity is being reported in the status bar during the wait.

The wait time is 59 or 60 seconds before a start service prompt and UAC is displayed.

Also, I have enabled the /exit_indexing_process_on_exit=1 option but it does not seem to terminate the Everything Service automatically. I have waited longer than 60 seconds for it to terminate. Tested with multiple attempts at re-registering the service.

This is on Windows 7 64. Is this why the UAC is always displayed? Is it a feature of Win 10 for standard users to start/stop?

I can type about:access-denied to skip the wait.

[void]

Thank you for your feedback raccoon,

It sounds like the Everything Service is not allowing start/stop from a standard user in your case.
-Could you please confirm from the Task manager -> Services -> Right click Everything 1.5a -> Start/Stop are available/clickable?



Please make sure you install the new 1321a Everything Service at least once.
This will set the service permissions to allow start/stop.

[therube]

(Heh. I'm now a bit confused with these changes .)

With the ZIP (x86)...

General -> Start Everything on system startup
- I've never had that enabled. Now it shows as "indeterminate". I guess that doesn't matter (& will not start on system startup).

General -> Run indexing process as administrator
- Is enabled, though I do get a UAC prompt on each open. (Opens speedily on my end.)

start "" .\everything.exe -instance 15

So now, the indexing process appears to be a child of the GUI? (Which I guess doesn't matter?)

The indexing process shows as High integrity & Full elevation.
Is that more or less "safe" then using the Everything Process (service), or doesn't matter (particularly)?


When using the indexing process, on startup (after UAC prompt), there is a short period of time, oh I suppose it could be a few seconds, where the GUI has drawn, files are listed, but during that time, Everything is actually in a "busy" state (& you might catch file changes turnings up - mouse cursor should show that). I guess that is Everything picking up on changes since its last close?

[tuska]

2void

Thank you for the new command line options for portable use, e.g. "index-as-admin"!

I currently use the following settings for my portable Everything version on system drive C:

• ⏹ Store settings and data in %APPDATA%\Everything
• Start Everything on system startup
• ⏹ Everything Service
• Run indexing process as administrator
• ⏹ Run as administrator
• Indexes -> Database location: Field without content (Everything.ini, Everything.db, etc. in the same location as Everything64.exe)

The only thing that "bothers me" at the moment is that the UAC query has to be confirmed for Everything when the PC is started.

[void]

Thanks for the feedback therube and tuska,

General -> Start Everything on system startup
- I've never had that enabled. Now it shows as "indeterminate". I guess that doesn't matter (& will not start on system startup).

Another Everything instance is set to run on startup.
Please check your registry under:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Run
There should be an entry for Everything
Manually delete it if needed.

General -> Run indexing process as administrator
- Is enabled, though I do get a UAC prompt on each open. (Opens speedily on my end.)
start "" .\everything.exe -instance 15

If you wish to avoid the UAC prompt, use the Everything Service.
"Run indexing process as administrator" and the "Everything Service" do the exact same thing, that is: run Everything.exe with -svc
"Run indexing process as administrator" runs in session 1, while the "Everything Service" runs in session 0.
"Run indexing process as administrator" will show the UAC prompt.
The "Everything Service" will not show the UAC prompt.

I recommend the "Everything Service" if you want to avoid the UAC prompt:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Everything Service.
• Uncheck Run indexing process as administrator.
• Uncheck Run as administrator.
• Click OK.

So now, the indexing process appears to be a child of the GUI? (Which I guess doesn't matter?)

Correct.
The indexing process is launched as a child process from the GUI process.

The indexing process shows as High integrity & Full elevation.
Is that more or less "safe" then using the Everything Process (service), or doesn't matter (particularly)?

A lot safer than running Everything GUI as an admin.
"Run indexing process as administrator" and the "Everything Service" offer the same "safeness".
Both run as High integrity / Full elevation.
Both have very limited functionality and only feature NTFS indexing and monitoring services.

When using the indexing process, on startup (after UAC prompt), there is a short period of time, oh I suppose it could be a few seconds, where the GUI has drawn, files are listed, but during that time, Everything is actually in a "busy" state (& you might catch file changes turnings up - mouse cursor should show that). I guess that is Everything picking up on changes since its last close?

There was a few changes in 1320a with redrawing new results.
I am working on ways to make the redrawing "less busy".
There will be changes in the next alpha update.

The only thing that "bothers me" at the moment is that the UAC query has to be confirmed for Everything when the PC is started.

I recommend the "Everything Service" if you want to avoid the UAC prompt:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Everything Service.
• Uncheck Run indexing process as administrator.
• Uncheck Run as administrator.
• Click OK.

This is on Windows 7 64. Is this why the UAC is always displayed? Is it a feature of Win 10 for standard users to start/stop?

I couldn't get the UAC prompt to show on Windows 7 x64.
I'll keep looking into the issue.

Maybe setting the permissions failed.
Any errors are shown in the debug output when installing the Everything Service. (please try toggling the Everything Service to see any errors)

[raccoon]

It sounds like the Everything Service is not allowing start/stop from a standard user in your case.
-Could you please confirm from the Task manager -> Services -> Right click Everything 1.5a -> Start/Stop are available/clickable?

Please make sure you install the new 1321a Everything Service at least once.
This will set the service permissions to allow start/stop.

Yes, I terminated the 1319 service when I installed 1321 and allowed it to install the new service. Yes, the Stop menu item is available, and then Start becomes available when stopped. I verified the taskmgr process is running as a standard user. Same within services.msc as well.

This is on Windows 7 64. Is this why the UAC is always displayed? Is it a feature of Win 10 for standard users to start/stop?

I couldn't get the UAC prompt to show on Windows 7 x64.
I'll keep looking into the issue.

Maybe setting the permissions failed.
Any errors are shown in the debug output when installing the Everything Service. (please try toggling the Everything Service to see any errors)

Start/stop for the Everything Service as a standard user should work on Vista+ (from my testing)

I'll shoot you two debug logs. One at Everything startup prior to the prompt to start the everything service (with UAC), and one for toggling the service off and on (via options settings and via services.msc).

[void]

Thanks for the logs raccoon,

The new permissions are not being applied if the "Everything Service" is currently installed.
I will post a fix soon.

For now, please uninstall the Everything Service and reinstall the Everything Service:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Uncheck Everything Service.
• Click Apply.
• Check Everything Service.
• Click OK.

[raccoon]

Oh, cool, so the solution was just toggling the [x]Everything Service options setting, which I already did for the debug log I guess I should have tried exiting and relaunching Everything to discover it was already fixed!

However, the /exit_indexing_process_on_exit=1 setting does not seem to work yet. Also, I cannot set /exit_indexing_process_on_exit=0 because it will just report "exit_indexing_process_on_exit=1" in the statusbar when I do.

[void]

exit_indexing_process_on_exit is a "machine" ini setting.

Everything does not allow setting "machine" ini settings from the search box.
/exit_indexing_process_on_exit=1 will not work from the search box (access denied)


Machine ini settings are stored in the Everything.ini in the same location as your Everything.exe.


To enable exit_indexing_process_on_exit:

• In Everything, from the File menu, click Exit.
• From the Start menu, search for: notepad
  Right click Notepad and click run as administrator.
• In Notepad, from the File menu, click Open...
• Select your Everything.ini in the same location as your Everything.exe
  (typically: C:\Program Files\Everything\Everything.ini)
• Change the following line:
  exit_indexing_process_on_exit=0
  to:
  exit_indexing_process_on_exit=1
• Save changes and restart Everything.

I will consider making exit_indexing_process_on_exit a "user" ini setting.
User ini settings can be set from the seacrh box.

[raccoon]

If my install is C:\Portable\Everything15a\ then are all setting stored in the same ini?

Does exit_indexing_process_on_exit default to 1? My setting is 1 in Everything-1.5a.ini but did not edit the ini myself, only used the search box command. So it was the search box command that would have changed it from 0 to 1 if the default was 0. This setting does not exit the indexing process when I exit Everything.

I have just discovered the ini setting stop_service_on_exit=0. I changed this to 1 by editing the ini in Notepad and restarted Everything. Now the indexing process does terminate on Exit.

I guess I am confused why these two settings exist in contrast. I suppose the former is just for the As Admin indexing process, but not the As Service indexing process. I suppose that makes sense. I just didn't read your posts correctly.

[void]

Sorry, I didn't make this clear.

exit_indexing_process_on_exit = exit the "Run indexing process as administrator" process on exit.
stop_service_on_exit = stop the "Everything Service" on exit

exit_indexing_process_on_exit is 1 by default.
stop_service_on_exit is 0 by default.

exit_indexing_process_on_exit
stop_service_on_exit

If my install is C:\Portable\Everything15a\ then are all setting stored in the same ini?

Yes.

If Tools -> Options -> Genreal -> "Store settings and data in %APPDATA%\Everything" is enabled, your user settings are stored in %APPDATA%\Everything\Everything.ini
Otherwise, user settings are stored in the Everything.ini in the same location as your Everything.exe.

I will look into allowing /stop_service_on_exit=1 from the search box when "Store settings and data in %APPDATA%\Everything" is disabled.

[therube]

Another Everything instance is set to run on startup.

While I certainly knew I had multiple Service "instances" running, I didn't realize that any had been set up (under /currentversion/run).
(Odd, as that's something I "never" do.)

"C:\DEV\LOCATE\test\Everything.exe" -startup -instance "SHERRY"

(1.5.0.1302, as it is.
-startup, is "in the background", different from starting "on system startup")

(Autoruns told me that.)

[void]

Machine ini settings can now be set from the search box when "Store settings and data in %APPDATA%\Everything" is disabled in Everything 1.5.0.1322a.

For example:
/exit_indexing_process_on_exit=0

[tvt]

The Everything Service is not an option for me because it exposes all filenames to all users. Does running the indexing process as administrator solve this problem?

[void]

No, the admin indexing process will provide all filenames to all users.

Use folder indexing (Tools -> Options -> Folders) to index only accessible files/folders.

[therube]

The Everything service doesn't need to be running for that. As long as the database manager (it's in the system tray) is running, you're good.

Ah. So ES.exe is keyed to the Everything GUI rather then the Service, & likewise is keyed to the -instance name (if used) AND to the associated .db. (As in ES.exe, the "lookup", reads the .db rather then interfacing with the service.)

[NotNull]

Might be. The source code of Everything is in the vault next to the Coca cola recipe, so we will have to guess ...
My guess is that es.exe talks to the database manager.
The database manager is loaded into memory (does not require the GUI to be visible).
the database manager accepts the ES serach query and search in it's database for matching files and returns these to es.exe, which in turn shows them in the terminal (or exports them to disk)

The service is unrelated to es.exe; ES.exe isn't even aware of the existence of a service.

ES needs to talk to the right database manager and as each Everything db. manager runs in its own instance (either named or unnamed), ES needs to specify which database/instance to use.

(so ES is not reading a .db file from disk)


A bit off-topic:
For more confusion: multiple Everything (1.5) instances can make use of a single service, even when the service is started with the -instance something parameter.
I guess this is done by letting the service check if the request comes from a 'valid' source (like: a program signed by voidtools or ??)

But as said: all speculations (and a bit of analyzing), so this might be way off.

[tvt]

No, the admin indexing process will provide all filenames to all users.

Use folder indexing (Tools -> Options -> Folders) to index only accessible files/folders.

I do not see an option to index all accessible folders; it looks like I would have to add and remove paths manually. Also, if I understand correctly, this will miss changes when Everything is not running.

I actually want administrators to be able to search all files. Could you please add an option to restrict access to the service?

[NotNull]

I actually want administrators to be able to search all files. Could you please add an option to restrict access to the service?

FWIW:
the service is only needed for local NTFS system volumes.
If you are planning to index network drives from the client PC, that will be done using the credentials of the user starting Everything.
And that means only the files/folders this user has access to will be included in Everything.

[horst.epp]

I don't see the real problem here.
The service allows to see all file or dir names
but accessing them will be restricted to the rights of the actual user by Windows.
So you are trying to get security by hiding names, which isn't a real problem.

[tvt]

I don't see the real problem here.
The service allows to see all file or dir names
but accessing them will be restricted to the rights of the actual user by Windows.
So you are trying to get security by hiding names, which isn't a real problem.

Windows has a "list folder" permission for a reason. Filenames and other metadata can reveal sensitive information. Do you work on Everything?

[horst.epp]

...
Windows has a "list folder" permission for a reason. Filenames and other metadata can reveal sensitive information. Do you work on Everything?

No, Everything has only one Developer.

[void]

I do not see an option to index all accessible folders; it looks like I would have to add and remove paths manually. Also, if I understand correctly, this will miss changes when Everything is not running.

To index only accessible folders:

• In Everything, from the Tools menu, click Options.
• Click the Folders tab.
• Click Add....
• Select your base folder (eg: C: drive) and click OK.
• Click OK.

Repeat for multiple paths.
I recommend adding the entire drive, eg: C:

Everything will miss changes to folder indexes when not running.
However, Everything will rescan these folders daily to keep them up-to-date.
You can control how often Everything rescans under Tools -> Options -> Folders.

I actually want administrators to be able to search all files. Could you please add an option to restrict access to the service?

I will consider an option to restrict access to the Everything service.
Thank you for the suggestion.

For now, stick with folder indexing if you have sensitive filenames.

[tvt]

Repeat for multiple paths.

That's the problem; folder indexing doesn't have the options to add and remove volumes automatically.

You can control how often Everything rescans under Tools -> Options -> Folders.

Unfortunately, we need the results to be current.

I will consider an option to restrict access to the Everything service.

Thank you!

[raccoon]

You don't have to use "Folder Indexing." Each NTFS volume has the ability to specify what is indexed, and there is an Exclusion section as well that applies to all volumes (old and new/future).

99.9% of all Administrator-sensitive files are located on the C: drive. Automatically added external volumes (eg, USB keys) are almost always user agnostic.