search command or preprocessor to PROCESS SHELL RETURN VALUES

[gautxori]

hi, I would like to know if there is any way (search command or whatever) to process Windows shell output values (%ERRORLEVEL%, $LASTEXITCODE or any other program return values) from the Everything search bar.
I am interested in being able to “apply a low priority filter” to the search results that would allow, for example, restricting the results of “*.xlsx” for those files that have worksheets with a given name, or a certain number of worksheets, using a shell script, either vbscript or powershell. In other words, the script or program would return a value to Everything, which would process it in order to filter results. Is this possible, or can someone implement it as a search command in Everything 1.5?

[void]

Something like this should be possible with ES.

I will consider support for %ERRORLEVEL%
Technically, %ERRORLEVEL% should work, but the Everything environment block will be different to the executed file.

[gautxori]

Thanks David
I can see perfectly well how to do it with the command line version of everything, but the thing is to do it from the GUI window.

From the command line I am able to process each of the results that everything gives me, but from the GUI window I am not able to send the list of files to an external command and return the result of the command to the same window.
------
One of the drawbacks of using %errorlevel% is that you have to process that output variable for each file in the list. It seems more practical to send the filtered listing, via the "search command" (which could be called "run:"), to the standard input of the external program; and process the standard output of that program once it has processed that listing, to do everything in one step. I'd suggest to impose a strict output format to the external program (e.g. JSON): it would create a dictionary where each key should match entries in the standard input (search results), and would be paired to a value provided by the program.

So, putting in Everything search bar
*.xlsx run:my_script>2 musicworkbooks path:whatever
The my_script script would receive the listing for the result of searching "*.xlsx musicworkbooks path:whatever", would process it, and return the JSON dictionary. Everything would compare the results with the condition ">2" stated in the command to determine the final result in the graphical window.
Doing it with ERRORLEVEL would be MUCH slower. And with that feature Everything would become a really powerful search system.

------
I know you can do something similar from the GUI by following these steps:
- do the search "*.xlsx musicworkbooks path:whatever"
- export the search result to an EFU (or CSV or TXT) file [this part has to be done manually].
- process from my_script that EFU file, and generate ANOTHER EFU file [which means leaving the Everything interface],
- the EFU file could be displayed in a new window, using Everything.exe -f file.EFU.
That workflow can be modified by using other Everything features, such as:
- Everything CLI (as indicated), IPC o ETP. This implies working from outside Everything GUI
- "Custom values for properties" mentioned in Tagging without modifying files and Index Properties. This way of implementing access to properties / tags imposes many restrictions, such as fixed file paths.

my proposal involves automating those processes, controlling them from the search bar, and adding flexibility (by pairing values to search result items).
This proposal adds the feature of exposing Everything search results to other processes through standard input / output, which complements the mechanisms of CLI, IPC, ETP, file lists, or exporting results to files from the GUI.

There are many other proposals / suggestions related to mine, either using the alternate paths mentined above (CLI, IPC, ETP, Properties), or some like Indirect search with external Program/Script(simple plugin/makro). I think everyone of us would be pleased to find such a feature implemented from the search bar.

[void]

Thank you for your reply.

I like the idea of run:my_script>2

I'm not sure about the JSON dictionary and processing a single run command.
This sounds like a niche use case.
I would prefer simplifying it and running my_script for each *.xlsx result and then checking if that returns >2
or do you think this would hurt performance too much? -does my_script need to know all the results prior? does Everything need to make a call to initialize things before any calls to my_script? -maybe a presearch-run:myinitscript command..

I haven't been to keen on adding "run" commands for security reasons.
However, in the next alpha update I will have an advanced "search_command_allow_all" setting which will allow these types of commands.

[gautxori]

Thank you for your reply.

I like the idea of run:my_script>2

Good! That makes it much more likely that you will end up implementing my proposal.

I haven't been to keen on adding "run" commands for security reasons.
However, in the next alpha update I will have an advanced "search_command_allow_all" setting which will allow these types of commands.

As for the security of my proposal, by using stdout and stdin it is guaranteed, both from the perspective of the user using the script and the possibility of third parties trying to manipulate those channels, otherwise no Windows program would use them. Dont you think so?

I'm not sure about the JSON dictionary and processing a single run command.
This sounds like a niche use case.

The use of a dictionary as the return of the execution of the program is a proposal although other solutions can be used. The key is that the script receives all the list of results that it has to receive and that it returns ONLY ONE value for each item of the results. This is so that the behaviour of the run: command is similar to that of any of the "search functions" you have implemented, such as dm:, path:, ...: all of them process ONLY ONE VALUE for each item. As for the most convenient data structure to process a collection of value pairs either an array or a dictionary would be appropriate. So it's not a niche solution, it's the solution that apparently best fits both the intended purpose and the search paradigm you have created with Everything (which is undeniably the best and most popular free file search tool ever created for Windows).

I would prefer simplifying it and running my_script for each *.xlsx result and then checking if that returns >2
or do you think this would hurt performance too much?

The input/output in the command execution for each search result item can take a significant percentage of the script/program execution time, depending obviously on what it does... and the risk of Everything getting stuck due to improper execution of the script/program is much higher.
The responsibility for the successful execution of the script/program lies with the user, it is impossible to guarantee that at runtime everything works perfectly. But you should take appropriate measures to prevent that process from getting stuck (e.g. limit the execution time of the script by means of a configuration variable, otherwise kill the subprocess) and it will be easier and safer to do so if you have a single execution for each search command.

-does my_script need to know all the results prior?

It depends on the case.

There are several facts justifying it is better that the script receives all the results, besides the previous one. The fact that it knows all the results allows the script to make decisions with respect to all of them, that otherwise it would not have the scope to make. Suppose for example that you want the script to return a sorting of those files with respect to a given criterion... (of course: that would allow to use the result of the run command with the sort: modifier).

does Everything need to make a call to initialize things before any calls to my_script? -maybe a presearch-run:myinitscript command..

Yes, I think the best option would be the one you propose, although this would be a niche case . If the user needs to "initialise something", he would execute a first command. And if he wants to do successive operations, he executes successive run: commands.

By the way the name run: is chosen to avoid collision with existing search functions.

[void]

As for the security of my proposal, by using stdout and stdin it is guaranteed, both from the perspective of the user using the script and the possibility of third parties trying to manipulate those channels, otherwise no Windows program would use them. Dont you think so?

stdout or stdin would not be used here.
The current filename would be passed to your script on the command line.
Everything will wait for your script to complete and use the exit code for the results.

An Everything search should not be able to make external calls by default.

The use of a dictionary as the return of the execution of the program is a proposal although other solutions can be used. The key is that the script receives all the list of results that it has to receive and that it returns ONLY ONE value for each item of the results. This is so that the behaviour of the run: command is similar to that of any of the "search functions" you have implemented, such as dm:, path:, ...: all of them process ONLY ONE VALUE for each item. As for the most convenient data structure to process a collection of value pairs either an array or a dictionary would be appropriate. So it's not a niche solution, it's the solution that apparently best fits both the intended purpose and the search paradigm you have created with Everything (which is undeniably the best and most popular free file search tool ever created for Windows).

This is beyond the scope of Everything.
Use tools like ES or other automation tools to do this.



I have on my TODO list to add a "mark system".
Files can be "marked" with certain symbols, eg: blue square, green triangle, yellow star..
Maybe that would be useful combined with ES and your script.
Example usage:
call es to clear marks, eg:

es -clear-marks

Get your xlsx files from ES:

es *.xlsx -size -dm -export-json out.json

run your script on out.json
have your script call es to mark files, eg:

es -mark "example.xlsx" green-triangle

From there you could view the marked files in the Everything GUI or from ES.

es mark:green-triangle

Would you need more than a simple mark? would you want to store values, integers, strings etc...

[gautxori]