Find dupes for all files inside a folder

Discussion related to "Everything" 1.5 Alpha.
Post Reply
err_24546
Posts: 35
Joined: Wed Oct 11, 2023 1:47 am

Find dupes for all files inside a folder

Post by err_24546 »

Let's say I have a folder in c:folder with 3 files: 1.txt, 2.txt and 3.txt. What would be the syntax to find all duplicates for 1.txt, 2.txt and 3.txt?
Thanks in advanced.
void
Developer
Posts: 15811
Joined: Fri Oct 16, 2009 11:31 pm

Re: Find dupes for all files inside a folder

Post by void »

Find similar names:

Select the files of interest in your folder.
Right click the selected files under the Name column.
Click Copy Name.
Paste the copied names into your search box.

Everything will search for files with similar names.



Find duplicated names:

Select the files of interest in your folder.
Right click the selected files under the Name column.
Click Copy Name.
Add the following to your search:
whole:name:
Paste the copied names into your search box after the whole:name: text.

Everything will search for files with the same name.



Find duplicated sizes:

Select the files of interest in your folder.
Right click the selected files under the Size column.
Click Copy Size.
Add the following to your search:
size:
Paste the copied sizes into your search box after the size: text.

Everything will search for files with similar filenames.



Find duplicated hashes:

Right click the result list column header and click Add Column....
Search for:
sha-256
Select SHA-256 and click OK.
Select the files of interest in your folder.
Right click the selected files under the SHA-256 column.
Click Copy SHA-256.
Add the following to your search:
sha256:
Paste the copied SHA-256 hashes into your search box after the sha256: text.

Everything will search for files with similar filenames.

Gathering SHA-256 hashes will be slow.
Combine this one with the size duplicated method above for the best performance.
(SHA-256 information will only be gathered for files with a matching size)



size file list

Select the files of interest in your folder.
Copy the filenames to the clipboard (Ctrl + C)
Change your search to:

sizefilelist1:

Hold down Ctrl and click the sizefilelist1: text in the search box.
Paste your filenames and click OK.

Everything will list files with the same size.
(this one is more useful for larger files as smaller files will show more unwanted duplicates with conflicting sizes)



Folder data and names SHA256
Finding Duplicates
err_24546
Posts: 35
Joined: Wed Oct 11, 2023 1:47 am

Re: Find dupes for all files inside a folder

Post by err_24546 »

Thank you. I was hoping for some syntax so I could use it outside of Ev. (With some tool that has support).
For example, to find duplicates between 2 paths, you can use the syntax: c:/path1|c:/path2 file:dupe:ext:size.
I'm wondering if there is a way to do something similar, but instead of comparing 2 folders, compare with the entire database
void
Developer
Posts: 15811
Joined: Fri Oct 16, 2009 11:31 pm

Re: Find dupes for all files inside a folder

Post by void »

To find duplicates in your entire index, include the following in your search:

dupe:size

And include content:

dupe:size;sha256



Another option to find duplicated filenames between two paths:

<regex:"^c:\\path1\\subfolder A\\(.*)$" exists:"c:\\path2\\subfolder B\\\1"> | <regex:"^c:\\path2\\subfolder B\\(.*)$" exists:"c:\\path1\\subfolder A\\\1">

Since
c:/path1|c:/path2 file:dupe:ext:size
can find duplicates in just c:/path1.



These searches will work from ES or the SDK.
err_24546
Posts: 35
Joined: Wed Oct 11, 2023 1:47 am

Re: Find dupes for all files inside a folder

Post by err_24546 »

Sorry for my lazy responses and to bump this thread up.
I am going to try to present a specific scenario and the commands I am using to know if it is the right way to achieve my goal or not.
With the following items:
File: C:\my file 1\file1.txt ; size = 948 bytes
File: C:\my file 2\file2.png; size = 4548 bytes
Folder: C:\somes files 1
Folder: C:\somes files 2

CASE 1: Find dupes for file1.txt and file2.png based on their size and extension
Command:

Code: Select all

<size:948 ext:txt>|<size:4548 ext:png> file:dupe:size;ext
CASE 2: Find dupes for file1.txt and file2.png based on their size and extension, but only search in "some files 1" and "some files 2" folders
Command:

Code: Select all

<path:<<C:\some files 1>|<C:\some files 2>> <size:948 ext:txt>|<size:4548 ext:png>>|<<C:\my file 1\file 1.txt>|<C:\my file 2\file 2.png>> file:dupe:size;ext
CASE 3: Find existing dupes in "some files 1" and "some files 2" folders only
Command:

Code: Select all

path:<<C:\some files 1>|<C:\some files 2>> file:dupe:size;ext
Suggestions and corrections are more than welcome.
Thanks for your consideration!
err_24546
Posts: 35
Joined: Wed Oct 11, 2023 1:47 am

Re: Find dupes for all files inside a folder

Post by err_24546 »

@void do you have any thoughts /suggestions?
void
Developer
Posts: 15811
Joined: Fri Oct 16, 2009 11:31 pm

Re: Find dupes for all files inside a folder

Post by void »

Your searches look good.


CASE 1: Find dupes for file1.txt and file2.png based on their size and extension

<size:948 ext:txt>|<size:4548 ext:png> file:dupe:size;ext
The following shorter search will do the same:

<size:948 ext:txt>|<size:4548 ext:png>

-dupe: is not required


CASE 2: Find dupes for file1.txt and file2.png based on their size and extension, but only search in "some files 1" and "some files 2" folders

Code: Select all

<path:<<C:\some files 1>|<C:\some files 2>> <size:948 ext:txt>|<size:4548 ext:png>>|<<C:\my file 1\file 1.txt>|<C:\my file 2\file 2.png>> file:dupe:size;ext
"C:\some files 1\" | "C:\some files 2\" <size:948 ext:txt>|<size:4548 ext:png>

-no need to search for the original files or use dupe:
-use quotes around the path to escape spaces
-path: is not required if you use a \ in your path search.


CASE 3: Find existing dupes in "some files 1" and "some files 2" folders only

path:<<C:\some files 1>|<C:\some files 2>> file:dupe:size;ext
"C:\some files 1\"|"C:\some files 2\" file:dupe:size;ext
err_24546
Posts: 35
Joined: Wed Oct 11, 2023 1:47 am

Re: Find dupes for all files inside a folder

Post by err_24546 »

Many thanks @void!
<size:948 ext:txt>|<size:4548 ext:png>
-dupe: is not required
But without dupe, I get the original files even if there are no duplicates, and since my goal is to find possible duplicates of those files...that's why I put it in.
-no need to search for the original files or use dupe:
Something similar as above. Also if I do find duplicates, I also need the original files, even if they are not in the folders to be searched.

Good to know about quotes working in paths, because when you copy and paste paths with spaces in Everything, <> is automatically added instead of "".

Again many thanks!!
Post Reply