Indexing stalled

Discussion related to "Everything" 1.5 Alpha.
Post Reply
ChrisK
Posts: 2
Joined: Fri Oct 08, 2021 12:33 pm

Indexing stalled

Post by ChrisK » Fri Oct 08, 2021 12:54 pm

LOVE the content indexing/searching!

I've indexed 33K PDF & DOCX files, and it works amazingly.

However, I've run into about 6 files that stall/hang the indexer (ie. it shows that it's working on file X for hours/days at time - even after reboots).

I enabled debug logging (file below), but it does not seem to show anything useful. Here's a link to one of the files that gets stuck (it's a PDF e-book on APT and Malware) https://drive.google.com/file/d/1WaRI00 ... sp=sharing

Any guidance would be appreciated.

Thanks!


Debug Log.txt
2021-10-08 08:39:29.722: Everything
2021-10-08 08:39:29.722: Version 1.5.0.1278a (x64)
2021-10-08 08:39:29.722: Windows NT 10.0
2021-10-08 08:39:29.722: Processors 16
2021-10-08 08:39:29.722: IsAdmin 0
2021-10-08 08:39:29.722: AppData 1
2021-10-08 08:39:29.722: Service 0
2021-10-08 08:39:29.722: cmdline "C:\Program Files\Everything\Everything64.exe"
2021-10-08 08:39:31.843: MENUBAR ENTER OPEN
2021-10-08 08:39:31.843: MENUBAR POPUP 7
2021-10-08 08:39:32.551: MENUBAR LEAVE OPEN
2021-10-08 08:39:32.551: MENUBAR ENTER OPEN
2021-10-08 08:39:32.551: MENUBAR POPUP 8
2021-10-08 08:39:33.069: MENUBAR LEAVE OPEN
2021-10-08 08:39:33.069: MENUBAR ENTER OPEN
2021-10-08 08:39:33.070: MENUBAR POPUP 7
2021-10-08 08:39:35.640: #32768
2021-10-08 08:39:35.647: MENUBAR LEAVE OPEN
2021-10-08 08:39:35.647: COMMAND 40075
2021-10-08 08:39:35.784: Everything
2021-10-08 08:39:35.784: Version 1.5.0.1278a (x64)
2021-10-08 08:39:35.785: Windows NT 10.0
2021-10-08 08:39:35.785: Processors 16
2021-10-08 08:39:35.785: IsAdmin 0
2021-10-08 08:39:35.785: AppData 1
2021-10-08 08:39:35.786: Service 0
2021-10-08 08:39:35.786: cmdline "C:\Program Files\Everything\Everything64.exe"
2021-10-08 08:39:35.788: WM_ACTIVATE 0000000000000000 0000000000000000, lastfocus 0000000000120566, current focus 0000000000120566
2021-10-08 08:39:38.013: WM_ACTIVATE 0000000000000002 0000000000000000, lastfocus 0000000000120566, current focus 0000000000000000
2021-10-08 08:39:38.014: FOCUS 0
2021-10-08 08:39:38.014: FOCUS restore
2021-10-08 08:39:38.021: SYSCOMMAND 000000000000f012 0000000000ea050b
2021-10-08 08:39:43.211: COMMAND 41803
2021-10-08 08:39:43.212: Open pipe failed 2
2021-10-08 08:39:43.212: start all monitors (ntfs: 0 refs: 0)
2021-10-08 08:39:43.213: monitor folder setup
2021-10-08 08:39:43.214: RDC START 0000000000000000 1
2021-10-08 08:39:43.216: open volume \\.\E:
2021-10-08 08:39:43.216: opened 000000000000073c 0.000312
2021-10-08 08:39:43.217: RegisterDeviceNotification E: failed! 1066
2021-10-08 08:39:43.217: _db_monitor_process_update_event_available_event_proc
2021-10-08 08:39:43.217: check for folder updates 0000000000000000
2021-10-08 08:39:43.217: _db_indexed_property_request_available_event_proc 0
2021-10-08 08:39:43.217: indexed property request session created 1

void
Site Admin
Posts: 7819
Joined: Fri Oct 16, 2009 11:31 pm

Re: Indexing stalled

Post by void » Mon Oct 11, 2021 7:26 am

Thank you for your feedback ChrisK,

Thank you for the problem PDF.
Thank you for the debug logs.

I am able to reproduce the issue my end.

The PDF iFilter is dead locked when trying to GetOutputDeviceColorContext.
The PDF iFilter does not always dead lock.

Enabling verbose output will show more debug information.
I'm seeing the PDF iFilter getting stuck requesting a new chunk after chunk 9.

I am working on a fix..
I have had some success with setting the thread COM concurrency modal as multithreaded.
However, this will increase the total number of threads Everything uses.


For now, please consider excluding the problem files.
Also consider using Windows indexing with si:.

void
Site Admin
Posts: 7819
Joined: Fri Oct 16, 2009 11:31 pm

Re: Indexing stalled

Post by void » Thu Oct 14, 2021 5:55 am

Everything 1.5.0.1280a fixes an issue with reading content getting stuck.

Everything will now use multithreaded COM concurrency modal when gathering properties/reading content.

ChrisK
Posts: 2
Joined: Fri Oct 08, 2021 12:33 pm

Re: Indexing stalled

Post by ChrisK » Tue Oct 19, 2021 9:22 pm

Wow! You are truly appreciated! Thank you very much!!!!!!!!!!!!!!!!!!

Post Reply