preparation for copying takes extremely long

Found a bug in "Everything"? report it here
Post Reply
Debugger
Posts: 565
Joined: Thu Jan 26, 2017 11:56 am

preparation for copying takes extremely long

Post by Debugger »

cannot copy files, or preparation for copying takes extremely long for small files, it can take hours for 600 files by several kb.

Folder:500k files
small size = big problem
very small size = very big problem

Find 600 files

and I'm trying to copy to another folder on the SSD

The problem solves only the program "TeraCopy" copies in a few seconds.
But how to solve this problem in Windows without using TeraCopy?

Image
therube
Posts: 4580
Joined: Thu Sep 03, 2009 6:48 pm

Re: preparation for copying takes extremely long

Post by therube »

This is an issue with Everything?
Or does this also occur with Windows Explorer if you attempted similar?

Are you using the 'Copy to Folder...' of Everything?
Or are you using the context-menu to 'Copy' & the 'Pasting' elsewhere?
Or...?

What is the cluster size on your SSD?

I can take a folder with 100K small files (oh, probably < 1,024 bytes each), select 657 of them, & 'Copy to Folder...' - in a moment (HDD).

What is generating your 600 dialogs that you have pictured there?

What OS?
Debugger
Posts: 565
Joined: Thu Jan 26, 2017 11:56 am

Re: preparation for copying takes extremely long

Post by Debugger »

Windows 7
Probably 4K (SSD) (Normal)


It simply carries out the normal, standard copy method. Select -> copy, paste. Tested on many computers, the same problem.
"Calculating the time required to copy the files" takes too much time
Reproduce:
Use Everything
Find: 600 filenames Regex:^(file1|file2|...file600)\.html$
Folder: 500.000 files:
10 KB single file
Menu -> Copy/paste 600 files (normal method Windows) HDD -> SSD
horst.epp
Posts: 1332
Joined: Fri Apr 04, 2014 3:24 pm

Re: preparation for copying takes extremely long

Post by horst.epp »

Debugger wrote: Sat May 30, 2020 4:09 pm Windows 7
Probably 4K (SSD) (Normal)


It simply carries out the normal, standard copy method. Select -> copy, paste. Tested on many computers, the same problem.
"Calculating the time required to copy the files" takes too much time
Reproduce:
Use Everything
Find: 600 filenames Regex:^(file1|file2|...file600)\.html$
Folder: 500.000 files:
10 KB single file
Menu -> Copy/paste 600 files (normal method Windows) HDD -> SSD
Take a file manager, search for your files and then use its normal copy function.
There is no need for copy/paste and no preparation at all.
Debugger
Posts: 565
Joined: Thu Jan 26, 2017 11:56 am

Re: preparation for copying takes extremely long

Post by Debugger »

But in Everything, I don't know how to do it. It's too complicated here.
In another manager it is extremely simple, but uses the normal Explorer when copying.
How to find a 1000.000 character/lenght regex.
How to enter a folder to search for?

Name: Regex: ^(1|2|3|4|600)\.ext$
Folder: "C:\Files\*.*"
horst.epp
Posts: 1332
Joined: Fri Apr 04, 2014 3:24 pm

Re: preparation for copying takes extremely long

Post by horst.epp »

Debugger wrote: Sat May 30, 2020 5:45 pm But in Everything, I don't know how to do it. It's too complicated here.
In another manager it is extremely simple, but uses the normal Explorer when copying.
How to find a 1000.000 character/lenght regex.
How to enter a folder to search for?

Name: Regex: ^(1|2|3|4|600)\.ext$
Folder: "C:\Files\*.*"
The full Everything search syntax is available in Total Commander with its Everything integration.
So your search in TC would be
ev:Regex:^(1|2|3|4|600)\.ext$
The results feed to Listbox and copy or whatever you want.
vsub
Posts: 432
Joined: Sat Nov 12, 2011 11:51 am

Re: preparation for copying takes extremely long

Post by vsub »

Are you copying the folder that contains those files or you select all of the files and try to copy them
I just tried coping a folder that have 11k file of 250kb or smaller and after 3 seconds scanning,windows started copying the files from a slow 5400rpm hdd to a ram drive

It is normal to freeze for a while when selecting many files at once(not a single folder with many files inside)because in doing,windows builds a list of path to all of the files and the more they are,the more time it will take and then takes the same amount of time to send them to the clipboard

From the beginning until I right click on all of the files and copied the,the ram usage went up by 100mb and it took me around a minute just to copy them to the clipboard
therube
Posts: 4580
Joined: Thu Sep 03, 2009 6:48 pm

Re: preparation for copying takes extremely long

Post by therube »

OH, now I see!

That is how Windows works?
That is absurd!


Source: 76,639 items selected

Destination:

took forever before anything happened, simply a busy,
then eventually a "Preparing to copy..." dialog ... Calculating...

All the while, eating 100% of 1 core.


Come on. You've got to be kidding me!~

I'm at, at minimum, 10 minutes in, & not a single file has yet to copy, still "Calculating...".
CANCEL. (I simply can't take it any more.)


I'm on Win7.
As much as I might hope... I can't fathom that Win10 would be any better.

-------

Above, was "accomplished" using Windows Explorer, selecting files in Source, Ctrl+C, & attempting to paste to Destination, context-menu Paste.

The same "results" can be seen through Everything, selecting the files, Copy, then opening a Windows Explorer windows & Paste.
Ditto Everything's 'Copy to folder...'.

So Everything is relying on Windows facilities, & Windows facilities simply are not efficient (to say the least).


In my original testing, I was using Altap Salamander (file manager) & selecting, copy, then paste, from within there, & even though I took the Copy/Paste as a "Windows" operation, it must not be, it must still be within the confines of Salamander & doing it that way, one does not run into the Windows issues.

-------

So just how does one copy large data sets - using Windows facilities, in an efficient manner?

Command line, you can do it. COPY, XCOPY, Robocopy, XXCOPY, all start instantly.
But a Windows (native) GUI?


https://serverfault.com/questions/18125 ... wo-servers
https://www.online-tech-tips.com/softwa ... any-files/
https://www.raymond.cc/blog/12-file-cop ... fer-speed/


Still haven't figured out how to do it using Windows (GUI) facilities?
Post Reply