Can I speed up EBV reading avif files?

Discussion related to "Everything" 1.5.
Post Reply
Herkules97
Posts: 220
Joined: Tue Oct 08, 2019 6:42 am

Can I speed up EBV reading avif files?

Post by Herkules97 »

Original title:Can I speed up EBV readng avif files?

Using 1.5.0.1407a x64

Even with pausing media stuff's instance a bunch, it still seems to have managed to have read at least 100K audio and video files in the same time image stuff instance has read 15K avif files. I have 5mil files to index and of those 33K are known avif files, but I've come across avif files with other extensions so it could be a lot more. I don't pause Image Stuff as much because all the avif reading causes it to never go above 3% CPU usage normally. It still causes a minor lag while running video games but depending on game it's fine to keep on. Media Stuff usually sits around 13%-15% so that one I have to pause more.
Around 22K avif files seems to have taken 10 days.
Is it because of the GPU? RTX 2060 doesn't have hardware support for AV1, I don't think.
I figured it was a CPU task, that EBV never uses the GPU for reading properties.
Is it software support for EBV, as in you have to add that sort of support?

It has been nice because of the low CPU usage I can usually run it when the system is otherwise using most of the CPU like video games. Sometimes it would be nice to run a task with less than 3% CPU usage..But mostly I've found you can only lower in threads which is 8% each in my case. Even if you can like with supposedly https://mion.yosei.fi/BES/, if the system isn't designed to handle it maybe it can break things..or do nothing at all.
I guess one possibility is to limit multiple programs into the same thread. Then they have to fight over it 😈...But that's already the case, I don't limit every program so programs that are free to use any threads, particularly system components, will already be fighting over the first few threads as I understand it Windows goes from thread 0 to max when assigning work.
void
Developer
Posts: 19870
Joined: Fri Oct 16, 2009 11:31 pm

Re: Can I speed up EBV reading avif files?

Post by void »

Everything doesn't have any native avif support.
Reading any properties will go through the Windows Property System.

What properties are you reading?

Third party avif property handlers can be installed.
There might be faster property handlers out there.



Enabling multiple threads for your volume index might help:
  • In Everything 1.5, from the Tools menu, click Options.
  • Click the NTFS, FAT, ReFS or folders tab on the left. (the one that indexes your volume with the avif files)
  • Right click your volume and under Advanced, under the Threads submenu, check Multiple threads.
Note enabling might make performance worse for some hard drives.
Herkules97
Posts: 220
Joined: Tue Oct 08, 2019 6:42 am

Re: Can I speed up EBV reading avif files?

Post by Herkules97 »

void wrote: Tue May 05, 2026 12:58 am Everything doesn't have any native avif support.
Reading any properties will go through the Windows Property System.

What properties are you reading?

Third party avif property handlers can be installed.
There might be faster property handlers out there.



Enabling multiple threads for your volume index might help:
  • In Everything 1.5, from the Tools menu, click Options.
  • Click the NTFS, FAT, ReFS or folders tab on the left. (the one that indexes your volume with the avif files)
  • Right click your volume and under Advanced, under the Threads submenu, check Multiple threads.
Note enabling might make performance worse for some hard drives.
It is only avif that is slow, threads wouldn't matter as it's otherwise much faster and it doesn't go near 4% anyway and each thread is 8%. The one for audio and video is set to 2 threads and regularly sits at 13-16% as it should and so does the image one when it's not touching avif. It's almost done with known avif, so in a week or 2 it might finally work at normal speed again...Presuming there aren't a lot of avif files with other extensions.
Here is the ini's list, essentially all audio, video and stills properties or hopefully all of those.
properties=35mm Focal Length,Album,Album Artist,Altitude,Aperture,Artist,Aspect Ratio,Audio Bit Rate,Audio Bits Per Sample,Audio Channels,Audio Format,Audio Sample Rate,Audio Track Count,Author URL,Authors,Beats Per Minute,Bit Depth,Brightness,Camera Maker,Camera Model,Camera Serial Number,Color Representation,Comment,Composer,Compressed Bits Per Pixel,Compression,Conductor,Content Provider,Contrast,Copyright,Date Acquired,Date Encoded,Date Released,Date Taken,Description,Digital Zoom,Dimensions,Director,Encoded By,Exif Version,Exposure Bias,Exposure Program,Exposure Time,F-Stop,File Signature,Flash Energy,Flash Maker,Flash Mode,Flash Model,Focal Length,Frame Count,Frame Rate,Genre,Group Description,Height,Horizontal Resolution,Image ID,Initial Key,ISO Speed,Latitude,Length,Lens Maker,Lens Model,Light Source,Longitude,Maker Note,Max Aperture,Metering Mode,Mood,Orientation,Parental Rating,Parental Rating Reason,Part of a Compilation,Part of Set,People,Period,Photometric Interpretation,Producer,Promotion URL,Protected,Publisher,QuickTime Metadata,Rating,Related Sound File,Resolution Unit,Saturation,Sharpness,Shutter Speed,Size on Disk,Software,Subject,Subject Distance,Subtitle,Subtitle Track Count,Tags,Title,Total Bit Rate,Track,Transcoded For Sync,Vertical Resolution,Video Bit Rate,Video Format,Video Track Count,Vorbis Comment,White Balance,Width,Writer,Year
I haven't touched property handlers yet, other ones at least. Maybe I should bother looking that sort of solution up. There is one benefit to the slowness and that's the low CPU usage. It works well for most activities, even CPU-heavy games. But it takes maybe 2 minutes for one file. If that was the speed, it would probably take literal years to finish all 5mil. If my math doesn't suck, it would be 19-20 years. It's already done 7.4mil files and that's in less than 2 months because the folder for the instance is less than 2 months old. So avif has caused a proper crawl, hehe.
It would be faster to read files at normal speed and pause whenever I'm playing video games or using heavier Firefox sessions and resume when not. That's what I do for the audio/video instance and it's done at least 100K files in the same time as 15K avif files when the stills instance is rarely paused. I will likely look at property handler changes if known avif are done and it's still crawling at avif files with other extensions as there is no way for me to know how many of those I have..I presume. Reading the properties in the first place with EBV is one way but then I've already read them.
Herkules97
Posts: 220
Joined: Tue Oct 08, 2019 6:42 am

Re: Can I speed up EBV reading avif files?

Post by Herkules97 »

void wrote: Tue May 05, 2026 12:58 am Third party avif property handlers can be installed.
There might be faster property handlers out there.
So I was looking at an avif inside xnview and comparing with EBV and it can't read much. I wanted to index the content type so I can search for avif files, as some may have other extensions but be avif.
Xnview correctly reads this data...So I looked for property handlers for avif on Google Search and I don't find anything. Is it just Google, my search or a combination as some claim Google Search sucks these days..I can't find any handlers when I search for third-party property handlers. All I find are links talking about them, like MicroSoft's own page(https://learn.microsoft.com/en-us/windo ... properties).

I tried to see if there is a thread about property handlers on here, via Google Search again, but the best I got was viewtopic.php?t=9788.
Idk if support for it would be full, like the mime type for example. Windows doesn't have anything in that field and EBV's own system doesn't have "MIME type" at all. Unless it's named differently. In xnview the MIME type is image/avif.
There is also DNG..And probably other formats. Do I post every format I come across that I'd like to be able to read more fully into this thread?
viewtopic.php?f=12&t=9841

I haven't compared what xnview sees and what I can see in EBV properly, so maybe it can already read DNGs fully. It looks at a glance like xnview has a lot more data under the ExifTool tab in file properties, but that may just be because it's spread out across columns in EBV.

The best I have for finding avif files, with .avif extension or not maybe, was document-content-type. Oddly it shows up as image/heic, reading the Google Search AI summary the container is supposedly HEIF for both HEIC and AVIF or at least can be..At least for now when I search for heic, I only get avif files in the one folder that I know have avif files. But it has also taken about 2 weeks to finish with those and now it's onto DNG which is also taking long. Maybe in a few weeks it will have found more avif files after being done with that. Hopefully if there are avif files with the wrong extension they will also show up as image/heic. It's better than nothing.
Wikipedia page on the same results page also lists image/avif in the HEIF page. So it's not AI summary hallucinating it.
void
Developer
Posts: 19870
Joined: Fri Oct 16, 2009 11:31 pm

Re: Can I speed up EBV reading avif files?

Post by void »

Do I post every format I come across that I'd like to be able to read more fully into this thread?
viewtopic.php?f=12&t=9841
Yes, please.

I will try to support the same as what common web-browsers support.
eg: if you drop the file on your web-browser and it shows, I will try to support that file type.

There are limitations though, things like webp, webm, avif have massive libraries.
I cannot add these libraries to Everything as they are larger than Everything. (bloat)
For Everything 1.6 I have plans to add support for property plugins, so mediainfo and other libraries can be supported indirectly.

With that said, I can still support webp, webm, avif, etc.. natively if the headers are simple enough and I can parse this information easily enough..
Herkules97
Posts: 220
Joined: Tue Oct 08, 2019 6:42 am

Re: Can I speed up EBV reading avif files?

Post by Herkules97 »

void wrote: Sat May 09, 2026 12:20 am There are limitations though, things like webp, webm, avif have massive libraries.
I cannot add these libraries to Everything as they are larger than Everything. (bloat)
For Everything 1.6 I have plans to add support for property plugins, so mediainfo and other libraries can be supported indirectly.

With that said, I can still support webp, webm, avif, etc.. natively if the headers are simple enough and I can parse this information easily enough..
As someone that doesn't know what you are talking about, what do you mean avif has limitations due to massive libraries, but you can support if they're simple enough?
AVIF is possibly too big, but also not?
void
Developer
Posts: 19870
Joined: Fri Oct 16, 2009 11:31 pm

Re: Can I speed up EBV reading avif files?

Post by void »

I cannot add native support for AVIF easily.

I either have to:
1). Use existing massive libraries.
2). Write my own avif parser. (we just want a few properties, not the image data)

I just had a look at the AVIF spec and it isn't too complicated.
So I should be able to add support for basic properties, like width and height.

There's other issues too, like Everything will favor the Windows Property System.
So even If I add native support, you'll still be hit with the slower Windows Property System AVIF handler.
From a quick look at the spec, I assume the AVIF property handler is reading the entire file for each property lookup.
Herkules97
Posts: 220
Joined: Tue Oct 08, 2019 6:42 am

Re: Can I speed up EBV reading avif files?

Post by Herkules97 »

void wrote: Sun May 10, 2026 1:19 am I cannot add native support for AVIF easily.

I either have to:
1). Use existing massive libraries.
2). Write my own avif parser. (we just want a few properties, not the image data)

I just had a look at the AVIF spec and it isn't too complicated.
So I should be able to add support for basic properties, like width and height.
Hmm, according to the instance I index avif with, they already have those basic properties.
Dimensions, Width, Height, Bit Depth, Aspect Ratio, Vertical Resolution, Horizontal Resolution, Software, Exif Version and Document Content Type.

I didn't check using the preview for properties in EBV, so I don't know if all of these are unique to XnviewMP between it and EBV but..
Rating
Dimensions (in Megapixels)
Color Model
Print Size
MIME Type
Exif Byte Order
Orientation
Y Cb Cr Positioning
Flashpix Version

Quicktime-
Major Brand
Minor Version
Compatible Brands
Handler Type
Primary Item Reference
AV1 Configuration Version
Chroma Format
Chroma Sample Plosition
Image Spatial Extent
Image Pixel Depth
Media Data Size
Media Data Offset
Idk if they can have other data that a particular file I check doesn't have such as comments.
For example I found another .avif with far more properties. An entire "ICC Profile" section with around 38 property lines.
Also its quicktime section has a few more properties.
Uploaded it - https://pixeldrain.com/u/pBvvjWC3
void wrote: Sun May 10, 2026 1:19 am There's other issues too, like Everything will favor the Windows Property System.
So even If I add native support, you'll still be hit with the slower Windows Property System AVIF handler.
From a quick look at the spec, I assume the AVIF property handler is reading the entire file for each property lookup.
Yeah, about 1-3min per file..Or longer, Idk. 33K or so files took maybe 12 days, with some pausing here and there.
Fortunately AVIF are few relative to the full 16.4mil instance. If I had 16.4mil avif files, it'd probably take a few decades to finish at the same speed ;).

If nothing else I guess wait for 1.6x or figure out where third-party handlers exist. XnviewMP can read the files so maybe asking them about it.
I've seen random YouTubers use EBV so it's possible others relevant to XnviewMP use it too and might have interest in reading the files proper. Yeah that's some stupid arbitrary logic..
Post Reply