Combining and calculating folder sizes of the same name, without actually moving files

If you are experiencing problems with "Everything", post here for assistance.
Post Reply
conficker
Posts: 2
Joined: Tue Jan 21, 2020 11:26 pm

Combining and calculating folder sizes of the same name, without actually moving files

Post by conficker » Tue Jan 21, 2020 11:49 pm

Is there a way to do this?

By this, I mean:
I search for folders, Let's call it "Photos" and in the photos folder, I have another folder called "Maryjane". If in my search bar I have these folders specified, c:\photos|d:\photos|e:\photos ~ And the result is
Maryjane - 57.4MB
Maryjane - 102.5MB
Maryjane - 48.03MB

Would there be a way for Everything.exe to soft combine the folders without moving files, and calculate the entire size of folders that share the same name?

So that instead of 3 separate instances, I instead get one listing of "Maryjane" that says it's
Maryjane - 208MB

Even printed out in a text document, just something that lists their combined sizes.

void
Site Admin
Posts: 6075
Joined: Fri Oct 16, 2009 11:31 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by void » Wed Jan 22, 2020 12:40 am

Everything does not combine results.

You would need to search for each unique filename and check the total size.

To show the total size of all results in the status bar:
  • In Everything, from the Tools menu, click Options.
  • Click the View tab on the left.
  • Check Show size in status bar.
  • Click OK.
Note, enabling Show size in status bar reduces search performance.


What about exporting the results to a CSV file and creating a pivotTable in Excel?

To export to CSV:
  • In Everything, from the File menu, click Export.
  • Change Save as type to: CSV Files (*.csv).
  • Choose a filename and click Save.

conficker
Posts: 2
Joined: Tue Jan 21, 2020 11:26 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by conficker » Wed Jan 22, 2020 12:49 am

void wrote:
Wed Jan 22, 2020 12:40 am
Everything does not combine results.

You would need to search for each unique filename and check the total size.

To show the total size of all results in the status bar:
  • In Everything, from the Tools menu, click Options.
  • Click the View tab on the left.
  • Check Show size in status bar.
  • Click OK.
Note, enabling Show size in status bar reduces search performance.


What about exporting the results to a CSV file and creating a pivotTable in Excel?

To export to CSV:
  • In Everything, from the File menu, click Export.
  • Change Save as type to: CSV Files (*.csv).
  • Choose a filename and click Save.

At this point, I'm not sure I should continue my search for answer(s) here then, because I got the answer that matters: Everything cannot do that (yet?)

But if anyone would be willing to help me out more; Basically this is what I am dealing with

My external drives are full. I have 3 root folders, that each contain several branch folders. Individually there are about 4000 different unique filenames, but some of them are split (not duplicated) throughout the different drives

So what I am trying to do is locate which folders are the largest, throughout all the drives, and calculate their total size. And then delete them. because I need my space back.

To answer this before it arises - Yes, I can search by file size, or folder size, but they vary greatly. One folder of "Maryjane" may be 19GB, and the other 300MB. So i I search by folder sizes, then they will not show up next to each other. but together they would be 19.3 MB except I do not know how to combine a listing for it's total size.

and also I have zero experience with excel.

froggie
Posts: 185
Joined: Wed Jun 12, 2013 10:43 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by froggie » Wed Jan 22, 2020 1:19 am

Would something like TreeSize Free help you? It will, as the name says, list the size of some or all of the folders on your drives. (There are many like it, this is the one I use the most)

Code: Select all

https://www.jam-software.com/treesize_free/comparison.shtml

NotNull
Posts: 2691
Joined: Wed May 24, 2017 9:22 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by NotNull » Wed Jan 22, 2020 6:15 pm

There is no "Press this button" solution for your problem.

You could take a look at software that creates one (virtual) drive out of multiple real drives. Something like Drivepool.
Disadvantage of such a solution is that all your external disks should be connected (Everything creates a database so the disks don't need to be 'online')

Furthermore: this is also possible with a little script (and a lot of help from Everything :-))
But as said: not a "Press this button" solution.

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Wed Jan 22, 2020 10:04 pm

Easy enough to see the sizes, though selecting multiple directories does not sum them, only giving you a count.

Image

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Fri Jan 24, 2020 12:17 am

Code: Select all

C:\WLIB\LeechFTP\OLD>es folder: leechtmp  -size
39,565,173,569 C:\COLD\WLIB\LeechFTP\LEECHTMP
        24,576 C:\Local\SANDBOX\CLEAN-0902\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
        23,694 C:\Local\SANDBOX\CLEAN-2019\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
       180,195 C:\Local\SANDBOX\CLEAN\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
13,075,847,020 H:\LeechTMP
       615,724 H:\TMPS\2006\DOS\x\wlib\leechftp\leechtmp
   662,058,294 H:\TMPS\LeechFTP\LEECHTMP
    17,723,406 H:\XP43_C\DOWN\T-OLD\250.C\xxxdone - TMP\OLD-C\WLIB\LeechTMP
   386,644,670 I:\c_out\leechtmp
    25,629,408 H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\xxxdone - LeechFTP\xxxdone - LeechTMP
     8,549,457 H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\ISO\1800\wlib\xxxxdone - LeechTMP

C:\WLIB\LeechFTP\OLD>es.exe  folder: leechtmp  -size | addup

C:\WLIB\LeechFTP\OLD>gawk "{ sum += $1 } END { print sum }"
1992
Press any key to continue . . .
DONE
As it is, my addup breaks on the comma (,), so it is adding up 39+24+23+180+13+615+662+17+386+25+8, which is 1992.
If my (DOS) addup were not so lame...

I had a very good addup - under UNIX.
Getting it to work, correctly in DOS well that hasn't quite happened, yet.

Code: Select all

#
#       @(#) addup.sh 1.0 90/07/19
#
#       Copyright (C) therube, 1990
#       Adds up a column (default=last) of numbers in a file.
#       95/05/16 updated to allow (999) negative style numbers.

case $1 in
-[0-9])
        COLUMN=`echo $1 | tr -d -`
        shift
;;
*)
        COLUMN="NF"
;;
esac
echo "Adding up column .. $COLUMN .. of file(s) .. $*"

nawk  ' OFMT="%.2f"                                       # 1 "%12.2f"
        { x = '$COLUMN'                                   # 2
          neg = index($x, "$")                            # 3
          if (neg > 0) X = gsub("\\$", "", $x)
          neg = index($x, ",")                            # 4
          if (neg > 1) X = gsub(",", "", $x)
          neg = index($x, "(")                            # 8 neg (123 & change
          if (neg > 0) X = gsub("\\(", "", $x)
          if (neg > 0) $x = (-1 * $x)                     # it to "-123.00"
          neg = index($x, "-")                            # 5
          if (neg > 1) $x = (-1 * $x)                     # 6
          t += $x                                         # 7
          print "x is <<<", $x+0, ">>> running balance:", t
        } ' $*

# 1.  set numeric format to eliminate rounding errors
# 1.1 had to reset numeric format from 12.2f to .2f 95/05/16
#     when a computed number is assigned to a variable ( $x = (-1 * $x) )
#     it causes $x to use the OFMT so -1.23 = "________-1.23" vs "-1.23"
#     and that causes my #5 (negative check) to not work correctly because
#     the index returns a number >1 and to the neg neg than becomes a positive
#     this only occurs if the number happened to b a "(" neg number
# 2.  find the field we want to add up (comes from the shell or defaults
#     to the last field "NF") in the file
# 3.  check for a dollar sign ($) in the number - if there get rid of it
#     so we may add it correctly - $12 $1$2 $1$2$ $$1$$2$$ all = 12
# 4.  check for a comma (,) in the number - if there get rid of it so we
#     may add it correctly - 1,2 12, 1,,2 1,,2,, all = 12   (,12=0)
# 5.  check for negative numbers
# 6.  if x is a negative number in the form 999- "make" it a recognized
#     number like -999 - if x is a negative number like -999 already
#     the test fails (y is not >1) and this "true" negative is not made
#     positive
# 7.  accumulate the total
# 8.  if x is a negative number in the form (999) "make it a recognized
#     number like -999
# * Note that a (-9) (neg neg number) returns a positive
# * Mite not work rite with all forms of all numbers using $-,+. etc. *

NotNull
Posts: 2691
Joined: Wed May 24, 2017 9:22 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by NotNull » Fri Jan 24, 2020 11:07 am

Arithmetic in CMD can be done (set /a RESULT=1+1), but is rather limited as it uses 32-bit numbers. Half of those are reserved for the negative ones. So numbers above 2 billion (2147483647 to be precise) are an issue.
IIRC, you are using GNUwin, which is 32-bit, Does that have the same limitations?

Anyway, nice shell-script. Is nawk available under Windows (never heard of nawk before ..)


On topic (finally :)):
In the new ES version 1.1.0.18 - released a couple of days ago - is a new option -get-total-size that makes this *much* easier:

Code: Select all

+ T:\>es wfn:"program files" -size
 4,034,743,124 C:\Program Files
     2,804,194 C:\develop\Everything Installer\WixMSI 1.0\Sources\Program Files

+ T:\>es "\program files\" -get-total-size
4037547318

+ T:\>
Last edited by NotNull on Fri Jan 24, 2020 1:22 pm, edited 1 time in total.

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Fri Jan 24, 2020 11:49 am

nawk.

Basically awk, but with some differences.
(Don't recall what they were.)

I've only ever used the bin.zip packages, never any installer.
Prior to GNU, I used DJGPP, but they are pure DOS, so didn't work so well in x64.
Prior to that, (Altos) Xenix. Xenix was a, yes, you know who-dun-it, Microsoft product ;-).
Prior to that, there was Texas Instruments DX10.


And it was your "size" mentions in the other thread that prompted me to post this.
But the problem is, for this situation, while it gets the size of a particular directory, it does not get the various sizes for multiple named directories & sum the results, so in my case, the size of multiple leechtmp directories.

NotNull
Posts: 2691
Joined: Wed May 24, 2017 9:22 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by NotNull » Fri Jan 24, 2020 1:54 pm

Doesn't this command give you a total size of - roughly - 53 GB ?
(using ES 1.1.0.8)

Code: Select all

es.exe  "\leechtmp\"  -get-total-size

Note that this query will not include the following directories:
- H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\xxxdone - LeechFTP\xxxdone - LeechTMP
- H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\ISO\1800\wlib\xxxxdone - LeechTMP

Use
es.exe "\*leechtmp*\" -get-total-size
to include those folders too.


BTW: You can make your original code work by specifying the -csv option of ES. That will output unformatted numbers:

Code: Select all

es.exe folder: leechtmp  -size -csv | gawk "{ sum += $1 } END { print sum }"
( It appears that Cygwin's 64-bit gawk can return larger numbers )


BTW2:
Prior to GNU, I used DJGPP, but they are pure DOS, so didn't work so well in x64.
Prior to that, (Altos) Xenix. Xenix was a, yes, you know who-dun-it, Microsoft product ;-).
Prior to that, there was Texas Instruments DX10.
:o You witnessed / were part of the dawn of IT !! :shock: Cool!

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Sat Jan 25, 2020 10:10 am

Oh, -csv is just what the doctor ordered.
So yes, my addup(.bat) or your gawk work with that.

But I don't seem to be getting anywhere with -get-total-size.
(I had tried that earlier

Code: Select all

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechtmp -get-total-size
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es "leechtmp" -get-total-size
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es "/*leechtmp*/" -get-total-size
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es.exe folder: leechtmp  -size -csv | gawk "{ sum += $1 } END { print sum }"
53742470013

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es.exe folder: leechtmp  -size -csv | addup

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>gawk "{ sum += $1 } END { print sum }"
53742470013
Press any key to continue . . .
DONE

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Sat Jan 25, 2020 10:18 am

Though something does seem amiss?
es tmp gts returned results.
es leechftp gts returned results - once.

all other leech* gts attempts only returned 0.

Code: Select all

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es tmp -get-total-size
19066971645

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechftp -get-total-size
4872270

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechtmp -get-total-size
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechftp
C:\WLIB\Mozilla\USERS\FF61\storage\default\https+++www.leechftp.de
C:\COLD\WLIB\LeechFTP
C:\Local\SANDBOX\CLEAN-0902\drive\C\COLD\WLIB\LeechFTP
C:\Local\SANDBOX\CLEAN-2019\drive\C\COLD\WLIB\LeechFTP
C:\Local\SANDBOX\CLEAN-2019\drive\C\wlib\LeechFTP
C:\Local\SANDBOX\CLEAN\drive\C\COLD\WLIB\LeechFTP
C:\Local\SANDBOX\CLEAN\drive\C\WLIB\LeechFTP
C:\TMP\SEA\LeechFTP
C:\TMP\SEA\LeechFTP\LeechFTP
C:\WLIB\LeechFTP
H:\TMPS\LeechFTP
H:\TMPS\2006\DOS\x\wlib\leechftp
H:\XP43_C\WLIB\LeechFTP
H:\XP43_C\WLIB\LeechFTP2
H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\xxxdone - LeechFTP

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechftp size

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechftp -size
         6,220 C:\WLIB\Mozilla\USERS\FF61\storage\default\https+++www.leechftp.de
39,565,173,569 C:\COLD\WLIB\LeechFTP
        24,576 C:\Local\SANDBOX\CLEAN-0902\drive\C\COLD\WLIB\LeechFTP
        23,694 C:\Local\SANDBOX\CLEAN-2019\drive\C\COLD\WLIB\LeechFTP
         1,268 C:\Local\SANDBOX\CLEAN-2019\drive\C\wlib\LeechFTP
       180,195 C:\Local\SANDBOX\CLEAN\drive\C\COLD\WLIB\LeechFTP
    42,115,220 C:\Local\SANDBOX\CLEAN\drive\C\WLIB\LeechFTP
11,933,015,259 C:\TMP\SEA\LeechFTP
             0 C:\TMP\SEA\LeechFTP\LeechFTP
28,437,524,562 C:\WLIB\LeechFTP
   663,676,904 H:\TMPS\LeechFTP
       615,724 H:\TMPS\2006\DOS\x\wlib\leechftp
     1,413,767 H:\XP43_C\WLIB\LeechFTP
     1,481,085 H:\XP43_C\WLIB\LeechFTP2
    27,095,947 H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\xxxdone - LeechFTP

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechtmp -size
39,565,173,569 C:\COLD\WLIB\LeechFTP\LEECHTMP
        24,576 C:\Local\SANDBOX\CLEAN-0902\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
        23,694 C:\Local\SANDBOX\CLEAN-2019\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
       180,195 C:\Local\SANDBOX\CLEAN\drive\C\COLD\WLIB\LeechFTP\LEECHTMP
13,075,847,020 H:\LeechTMP
       615,724 H:\TMPS\2006\DOS\x\wlib\leechftp\leechtmp
   662,058,294 H:\TMPS\LeechFTP\LEECHTMP
    17,723,406 H:\XP43_C\DOWN\T-OLD\250.C\xxxdone - TMP\OLD-C\WLIB\LeechTMP
   386,644,670 I:\c_out\leechtmp
    25,629,408 H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\xxxdone - LeechFTP\xxxdone - LeechTMP
     8,549,457 H:\XP43_C\DOWN\T-OLD\250.DUH\_OLD_C\ISO\1800\wlib\xxxxdone - LeechTMP

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechtmp -gettotalsize
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechftp -gettotalsize
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es folder: leechftp -get-total-size
0

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Sat Jan 25, 2020 10:36 am

Code: Select all

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechftp -get-total-size
4872270

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechftp -get-total-size -p
80673284136
These are correct:

Code: Select all

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es c:/wlib/leechftp -get-total-size
28437524562

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es c:/wlib/leechftp -get-total-size -p
28437524562
I've got to think about what the top results are actually returning & just what is getting summed (with -get-total-size)?

therube
Posts: 2855
Joined: Thu Sep 03, 2009 6:48 pm

Re: Combining and calculating folder sizes of the same name, without actually moving files

Post by therube » Sat Jan 25, 2020 10:46 am

Ah!

Code: Select all

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechtmp -get-total-size
0

C:\WLIB\Mozilla\USERS\FF58 - NO Session\cache2>es leechtmp -get-total-size -p
53742470013
All right, got something positive, enough head scratching for now.
53742470013 is accurate (at least given the results given by es.exe folder: leechtmp -size [& not by comparing by other means]).

Post Reply