Keep loosing index when disconnecting 8TB drive

[andymbody]

Hi, I'm new to Everything and love it. However, I keep experiencing an issue that I don't know the cause of...

I have (2) 8TB NTFS backup drives that are basically mirrors of each other. Any files copied to these drives is done manually (no auto BKUP sw). No encryption, etc.... just raw data files organized in standard folders.

I have included both of these drives in my NTFS Index section of the options dialog. I have created a CRC-32 index property and I left my computer running (day and night) for a week with these drives attached so that Everything would do this index. I monitor the progress of the index regularly during that week and today it completed. I wanted to make sure that the index would "stick" this time (more on that in a bit), so I...

1. Shut down EverythingApp
2. Shut down the Win7 laptop
3. Disconnected the drives from the laptop (single USB cable)
4. Started up the Win7 laptop
5. EverythingApp is set to load during boot
6. Looked at the contents of all files in EverythingApp (with drives disconnected) - they were all still showing in results-list, just as they were prior to the shutdown (but greyed out of course).
7. Connected both 8TB drives (via single USB from 10-way splitter)
8. Once I did this, Everything tried to update the results-list and no longer showed content from the second drive (S:)
9. I looked in the NTFS volumes listed in the index area of options, and (C:, R:, S:) were all still listed, but the settings for (S:) had "Include in DB" UNCHECKED. I know for a fact that this was checked, because this is the 2nd time that this situation has occurred since I started using Everything app.
10. I don't know what is causing this setting to change on it's own, but when it does, I have to spend another week indexing the drive. I don't plan to do this again (twice is enough) until I have an idea of why this is happening.

(The reason I need to index these drives is so I can validate the mirror status between the (2) 8TB bkup drives)

I decided to pause further troubleshooting until I post here for thoughts from @void and other users.

Any ideas what could be causing this? Could it be because I'm using a USB splitter?
Andy

System:
Win7, plenty of spare drive and ram space (yes, I know its ancient)
USB3 drives and cables to a USB 2.0 connection on laptop
Everything Version 1.5.0.1330a (x64)
Automatically remove offline volumes is UNCHECKED

EverythingApp issue2.png (615.03 KiB) Viewed 7451 times

[void]

Thank you for your post andymbody,

I suspect the volume name/GUID is changing when you unplug and reinsert the drive.

Could you please confirm:

• In Everything, from the Tools menu, click Options.
• Click the NTFS tab.
• Select your S: drive
• Right click this drive and click Edit...
• Please make a note of the Volume name.
  ---
• Exit Everything (File Exit).
• Shutdown your PC.
• Remove the drive.
• Restart your PC.
• Start Everything (if not started automatically)
• Connect your drive.
  ---
• In Everything, from the Tools menu, click Options.
• Click the NTFS tab.
• Select your S: drive
• Right click this drive and click Edit...
• Please compare the Volume name with the first one.

Has the volume name changed?



Is automatically move NTFS enabled:

• Copy and paste the following into your Everything search box:
  /auto_move_ntfs_volumes
• Press ENTER in your Everything search box.
• If successful, auto_move_ntfs_volumes=x is shown in the status bar for a few seconds.
  where x should be 1 for enabled, and 0 for disabled.
• What is x for you?

-If this option was disabled, please try enabling this option:

• Copy and paste the following into your Everything search box:
  /auto_move_ntfs_volumes=1
• Press ENTER in your Everything search box.
• If successful, auto_move_ntfs_volumes=1 is shown in the status bar for a few seconds.

Something is wrong here with Everything, the old drive should not have been removed.
Everything should see the same volume serial number and automatically include this "new" volume.

I will need to see debug logs to figure this out.
Could you please:

• Re-include your S: drive in your index (under Tools -> Options -> NTFS -> S: -> check include in database.)
• Repeat steps 1-6.
• Before step 7 (connecting your drive), please enable debug logging:
• In Everything, from the Tools menu, under the Debug submenu, click Start Debug Logging.
• Connect your drive.
  ---wait for Everything to reindex.
• Are you indexing properties? -If Everything takes a really long time, please give Everything at least 5 minutes before stopping debug logging below.
• In Everything, from the Tools menu, under the Debug submenu, click Stop Debug Logging.
  ---this will open your %TEMP%\Everything Debug Log.txt in Notepad.
• Could you please send this file to support@voidtools.com

[andymbody]

THANK YOU @void for all the details!

Has the volume name changed?

No... it has stayed the same...
\\?\Volume{9e04e243-985a-4efd-864c-17c2df722c52}

What is x for you?

set to 1

I will need to see debug logs to figure this out.
Could you please send this file...

I performed the steps that you asked... will send the log to you...

Line 578 looks very interesting... "VOLUME SERIAL NUMBER MOVE S: => R:" ... not sure if this is significant or not
BTW... your debug logs look amazing... that's a lot of work to code for capturing this much detail... Kudos!

Also, Win7 likes to ask about how to work with the volumes once connected (What would you like to do? popup). I am prompted about (S:) last. I'm not sure whether this affects timing for other apps and services to gain full access to the volumes?

The other thing I can try is to not have R: connected to the USB splitter (bus) and see if things change with R: and/or S: I can also try to connect the volume directly, not using the splitter to see if something changes (this 'bug' occurs every time, so that's always better than an intermittent issue). But I will wait to hear from you until I continue to try other things to narrow the cause.

Thanks again for your help with this!
Andy

[void]

Thanks for the information.

VOLUME SERIAL NUMBER MOVE S: => R:

I am already guessing the two volumes have the exact same serial number.

Serial numbers are stored in your %APPDATA%\Everything\Everything.ini under ntfs_volume_serial_number.

[void]

Disabling auto_move_ntfs_volumes should fix the issue:

• Copy and paste the following into your Everything search box:
  /auto_move_ntfs_volumes=0
• Press ENTER in your Everything search box.
• If successful, auto_move_ntfs_volumes=0 is shown in the status bar for a few seconds.

I will disabling this feature in the next alpha update as it will likely cause future issues.

[andymbody]

I am already guessing the two volumes have the exact same serial number.

They do show the same in the ini, but when I inspect the hardware serial manually, they are different

Disabling auto_move_ntfs_volumes should fix the issue:

Will do this and report back...

Thank you!
Andy

[void]

but when I inspect the hardware serial manually, they are different

Did:
fsutil fsinfo volumeinfo R:
fsutil fsinfo volumeinfo S:
Return different serial numbers?

[andymbody]

Did:
fsutil fsinfo volumeinfo R:
fsutil fsinfo volumeinfo S:
Return different serial numbers?

My mistake... yes, this is the issue... they are the same... why would WesternDigital do this? Dumb...
Will reboot and see if all goes well with new settings...

[void]

It's not WesternDigital fault.

I think this is caused by the mirroring software.
The volume serial number is being copied.

The problem actually occurs because your R: drive comes online before your S: drive, so Everything thinks you have changed the drive letter for your S: drive to the R: drive.
(S: drive is offline -but hey I found this new online R: drive with the same serial number!)

I'll have the issue fixed in the next alpha update and will make another post here once it is ready for testing.

[andymbody]

I think this is caused by the mirroring software.

I don't use any mirror sw, but maybe I did in the past... I can't remember...

Ok... the new setting that you suggested (auto_move_ntfs_volumes=0) fixed the issue with (S:) not wanting to stay part of the database. That setting now remains checked after a reboot... however...

Now Everything is not continuing the CRC-32 index for (S:). (R:) still looks fine, but (S:) needs to be indexed as well. Is there a quick fix for this, or will I need to remove (S:), then add it back to initialize the index of (S:)? Or will I need to start from scratch to fix the DB entirely?

Thanks!
Andy

[void]

That setting now remains checked after a reboot... however...

Which setting? the auto_move_ntfs_volumes setting?
-Please make sure this is set to 0 in your %APPDATA%\Everything\Everything.ini

Now Everything is not continuing the CRC-32 index for (S:)

Please try the following:
Please make sure your S: drive is connected.
In Everything, type in the following search and press ENTER:
/reindex s:
It will take a long time for Everything to gather the crc32 values for all your files on your S: drive.

[andymbody]

Which setting? the auto_move_ntfs_volumes setting?

no... the "include in db" setting which was always UNCHECKED automatically after each reboot. This setting now remains checked as it should.

I removed the (S:) drive, and added it back (prior to your last post) and it began indexing that drive from scratch. I am assuming this had the same effect that your command would have had. But it's nice to know that these types of commands are available when needed.

I'm thinking of looking into changing the SN of the (S:) so they don't match. I've never looked into what is involved (never had a need to), if it's possible with a sw tool? Of course, if I'm going to do that, I need to do so before I spend days re-indexing (S:)

I have to say... the control that you have provided from the UI itself is really amazing. I do a little coding myself and have never written this type of access to the settings thru the UI. Now that I see it it action, I think it's a great practice.

I would call this issue SOLVED! I just need to find out how to mark the thread as such.

Thank you so much for all of your assistance. I will be sending something your way for your time and effort on the app in general and this issue. Although it will be coming from another account.
Andy

[void]

From an admin command prompt, run the following command:

volumeid S: 0123-4567
where 0123-4567 is the new volume serial number



Instead of indexing CRC32, please consider using sfv sidecar files.
(Index the SFV CRC-32 property instead)

voidhash

[andymbody]

volumeid S: 0123-4567

My Win7 (custom) doesn't seem to have this. But I downloaded the SysInternals version (now hosted by MS). ID has been changed... Thanks!

Instead of indexing CRC32...

I will look into this! Thank you very much!!

Andy

[void]

Everything 1.5.0.1331a fixes an issue with moving volumes with the same serial number.

Everything 1.5.0.1331a adds a auto_move_same_serial_number ini setting.

This setting is now off by default.

To enable Everything to automatically move volumes with the same serial number:

• Copy and paste the following into your Everything search box:
  /auto_move_same_serial_number=1
• Press ENTER in your Everything search box.
• If successful, auto_move_same_serial_number=1 is shown in the status bar for a few seconds.

In your case andymbody, I would leave this option disabled (the default setting)

[andymbody]

In your case andymbody, I would leave this option disabled (the default setting)

Yes, will do... thank you for all your help with this issue!

[andymbody]

Just a thought...

To avoid the majority of issues related to this setting... would it be worth prompting the user during load if Everything detects that a drive letter has changed, or that it finds 2 drives with the same volume ID? If Everything prompted in those infrequent occurrences, the user could acknowledge which action should take place. Chances are that most of your users are savvy users and would understand the question (as opposed to users that may not understand the question they were being asked). You could add a setting "prompt if drive letter change is detected", and turn that setting on by default. Then the user would also have the option to disable the prompt if desired if it becomes annoying.

You could provide a timeout for the prompt so that if the user is not at the computer to answer, Everything will make the decision. And maybe somehow make it obvious to user that the change was made?

I guess this idea might be over-thinking a little, but I thought I would throw it out there.

Andy

[void]

I will consider a prompt when there is a possible move volume.

No prompt is needed if the volume name (volume GUID) stays the same.
Everything can assume the volume has definitely moved in this case.

Windows is pretty good at assigning a unique name to the volume and keeping it between removal and reinsertions.

You could provide a timeout for the prompt so that if the user is not at the computer to answer, Everything will make the decision. And maybe somehow make it obvious to user that the change was made?

An information bar at the top of the result list might work..
The bar could show:
S: was possibly moved to R:, would you like to remove the S: volume and add the R: volume to your index? YES/NO/Don't show again..

Thank you for the suggestions.