Bulk search with md5 value

[sk2107]

Hello,
I have an external hard drive which has a lot of files, and for some reason it was corrupted, but I have many files which are a copy of the files found in that hard drive distributed in other locations.

Luckily, I have a text file that contains all the md5 values of all files in that drive before it was damaged (file names may not be the same).

so is there a way (may be using advanced search or something else) to filter the equivalent MD5 values in the available file list in Everything for other drives (it is a very big list).

Thank you

[NotNull]

There are some options...
What is the syntax of the file? Please post a couple of lines (including header, if available)

[sk2107]

Actually its csv like, here is a sample:

Name,Path,MD5
py_backup.zip,U:\Saif1_1_drv\GD\W\images_ignored,A71C83D6FA499EF335623499DD8EFA0D
www_dmg.zip,U:\web\dmg,48F932B9B949A8807368DDCC78E0381B
16_GB.rar,U:\WD2T_1\zips_col_N\Col_everything\rars\V3\Ammans_2017\00\MedGulf_Col2\mg_13-05-2014,2169A567C4036A9AA16E45F308909113
sk_prog.zip,U:\WD2T_2\zips_col_O\Col_everything\zips\O1\Ammans_2017\Ammans3\To_Amman_12-2016,42827D045762C2A63377560AE5AC1322
kiswani.net15-03-2023.zip,U:\web,2EAEC1297B7131B509A1C5AA82F01EEF
MyDocument_d.zip,U:\WD2T_2\zips_col_O\Col_everything\zips\O1\Ammans_2017\Ammans4\To_amman_12-05-2016,4FD741FD71962D6BB4E926EBCB66F83F
olds.bin,U:\ccc_d\ccc5\R02\ADM\ccc,EEB717147F137BF2ABE6FF2565CB662C
it-applications.zip,U:\WD2T_2\zips_col_O\Col_everything\zips\O1\000000000000000,31C70CB07E0B76CE0B8404DB77BE60D0
disk1.vhd,U:\not__md5\WD2T_1\Col_everything\vhds\H\c_09-04-2017\vhds,63ECB5FEB50C92EB99782776D90E096C

Thank you

[NotNull]

I will try to convert this syntax to something that is usable i Everything (got something working already).

BTW: How many lines/how many files?

Do your regular files -- the ones you want to compare against -- already md5-indexed?

[sk2107]

the number of records in csv file is 641409

Yes md5 is already indexed in Everything.

Thank for the follow-up

Regards

[NotNull]

Success! Will send instructions later (quite a lot of typing and testing to do ...)

2025-02-11 16_57_05-u_ - Everything (1.5a) 1.5.0.1391a (x64).png (48.71 KiB) Viewed 4558 times

[sk2107]

I will be waiting.

Thanks again

Best regards

[NotNull]

Steps:

1. Make a copy of your file and put it in a new folder
2. Extract SSED.exe from the attached zipfile and put it in the same folder
   (I use it since years. No issues, no malware. But please check yourself.)
3. Start CMD.exe and browse to this new folder.
4. Run the following 2 commands:
   (I'm assuming your file is called input.txt and the result will be in output.efu. Change names in the commands to match your environment. Or change your filenames )

   Code: Select all

   echo."Filename","Size","Date Modified","MD5">output.efu
   ssed.exe "1d" input.txt | ssed.exe -r s/^(.*?),(.*?),/\"\2\\\1\",0,,/g >> output.efu
   

5. Check the output. Should look like:

   Code: Select all

   "Filename","Size","Date Modified","MD5"
   "U:\Saif1_1_drv\GD\W\images_ignored\py_backup.zip",0,,A71C83D6FA499EF335623499DD8EFA0D
   "U:\web\dmg\www_dmg.zip",0,,48F932B9B949A8807368DDCC78E0381B
   (etcetera)
   

6. In Everything, go to Menu => Tools => Options => Indexes => File Lists
7. Add output.efu
8. OK
9. go to Menu => Tools => Options => Indexes => Properties
10. Select MD5
11. Make sure your U:-drive is included. If not:include it (very important step!!)
12. OK
13. In Everythings main window, search for
    U:
14. Add the MD5 column to the result list.
15. Check if filenames, paths and MD5 values are shown
    (I configured the size to show 0 bytes and left date modified empty)

That's it. you can now search for md5-duplicates.
Don't forget to remove the File List from the index when done.

[sk2107]

I will try it.

thank a lot

Best regards

[sk2107]

I tested it and it worked fine.

It was a smart solution, and I really appreciate your efforts and giving time to solve our issues.

Thank you and best regards