How to restore database backups?

[abr01]

Since this afternoon, my main harddisk started dropping out. Yes, i have backups, off course, and this hard disk did this time and time again, but generally works fine. I have to unplug the power supply at the disk's end, and reconnect, it usually works.

But i have noticed, that the whole database has started to be rebuild, which takes quote a long time. After i had about 40%, when the disk dropped out again. Now i'm at 6%.

But i also have that 1 GByte Everything-1.5a.db file from wednesday. Maybe i can restore that one? But i'm not sure, how. At least i can't find any setting, except the import function, but i guess that one is for settings and filters only. And it case i can restore that database, will the newer files also be scanned and integrated, without starting from scratch?

Edit, those .db files are present in %LOCALAPPDATA%\Everything, as expected. But i have no idea how to "connect" them to be active.

[void]

Sorry to hear about the failing hard drive.

To open your Everything.db backup:

• In Everything, from the File menu, click Open File List....
• Select your Everything.db and click Open.

I recommend exporting filenames from your failing drive to an EFU file list:

• In Everything, search for your failing drive.
• For example:
  e:
• From the File menu, click Export....
• Change Save as type to EFU Everything File List.
• Choose a filename and click Save.

Everything will remove your failing hard drive from your index by default.
To revert to your Everything.db backup, disable automatic drive removal and replace your db:

• In Everything, from the Tools menu, click Options.
• Click the FAT tab on the left.
• Uncheck Automatically remove offline volumes.
• Click the NTFS tab on the left.
• Uncheck Automatically remove offline volumes.
• Click the ReFS tab on the left.
• Uncheck Automatically remove offline volumes.
• Click OK.
  
• Exit Everything (File -> Exit)
• Replace your %LOCALAPPDATA%\Everything\Everything.db with your Everything.db backup.
• Restart Everything.

and it case i can restore that database, will the newer files also be scanned and integrated, without starting from scratch?

Yes, Everything will update existing volumes.
Some volumes might need to be reindexed if too many changes occurred.
You won't lose any property indexes. They will persist between a reindex. New and modified files will be added to a property update queue.



Alternatively, to include a file list in your Everything index:

• In Everything, from the Tools menu, click Options.
• Click the File List tab on the left.
• Click Add....
• Select your exported EFU file list of your failed hard drive. (see above)
• Click OK.

[abr01]

Well, actually it's not really failing, luckily. I have that 8 TB harddisk for maybe eight years, and it happened a couple of times before, but worked for years after that. It always was something about the power jack, which i had to re-plug to get back to normal. So hopefully it's the same case this time again. It may be a thermic thing.

Since it's now at 53%, i will try and let it run. But in case it fails again, i have your instructions at hand now, which is very much appreciated. I will also make those EFU lists for all four partitions on that disk. And as i said, i have multiple backups of the most important files, and do syncs every few days, so at least i don't have to worry about a critical data loss. But i'm using Everything all the time, on a daily basis, and try to avoid any downtime.
I don't have the backups disks indexed, that's why i'm opting for the quickest way.

So, thanks for your help! Good to have the instructions now.

And cool, that the properties are conserved, they are quite important for my photo tags and coordinates, mainly.

[abr01]

I had several attempts now to rebuild the database, but the harddisk turns itself off again and again. So yes, i have to buy a new one within the next ten days or so, can't manage doing it earlier.

So i tried restoring my 2.7 GByte backup file until then, but somehow it doesn't really work. For example, when i'm searching for a special tag in my photography collection, i get four results, where it should be several hundreds.

Indexing also starts from scratch, whereas i would expect, with a freshly imported older database, to have maybe 98% of my old structure back.


Now i wonder, why the indexing just doesn't continue, from where the failure occured. For example, i had 66% indexed, i would expect Everything to continue from there. What do you think, what is the best workaround, until i can buy my new disk? Can i stop Everything from beginning indexing, and just load my old, slightly incomplete database?

[void]

Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason will show the reason for the reindex.

Now i wonder, why the indexing just doesn't continue, from where the failure occured. For example, i had 66% indexed, i would expect Everything to continue from there. What do you think, what is the best workaround, until i can buy my new disk?

Open your Everything.db backup as a file list (File -> Open File List -> Select Everything.db backup)
Search for your dying drive. for example:

e:

Export results as a file list.
Open this file list whenever you need to access the filenames on your dying drive.
Or, include it in your normal Everything index under Tools -> Options -> File lists.

[abr01]

Ok, i did that now for all four of my partitions on that disk. Although i'm not sure, if the list is complete, since the indexing isn't finished yet (and maybe won't be).

But it looks, like coordinates are also included, as well as maybe other indexed stuff. What i tried in the second image, was obviously wrong, and gave me that garbled data from the third screenshot.

So, what is the benefit in having those EFU files? Can they be written back to my data on my new drive, saving me re-indexing?

image.png

image.png

[void]

But it looks, like coordinates are also included, as well as maybe other indexed stuff. What i tried in the second image, was obviously wrong, and gave me that garbled data from the third screenshot.

Everything will export all indexed information (if those columns are shown).

Please include the .efu file list in your index (Tools -> Options -> File Lists)
Not the Everything.db

So, what is the benefit in having those EFU files?

They are a backup of the filenames on your dying drive.

Can they be written back to my data on my new drive, saving me re-indexing?

No.

[abr01]

But it looks, like coordinates are also included, as well as maybe other indexed stuff. What i tried in the second image, was obviously wrong, and gave me that garbled data from the third screenshot.

Everything will export all indexed information (if those columns are shown).

Please include the .efu file list in your index (Tools -> Options -> File Lists)
Not the Everything.db

Got it.

So, what is the benefit in having those EFU files?

They are a backup of your dying drive.

In what way? I have Terabytes of files, and that backup is just a fraction.

Can they be written back to my data on my new drive, saving me re-indexing?

No.

Sorry, then i don't understand what they are actually good for.

[abr01]

Sorry for not understanding the whole thing. For example, how do i have 1.2 million elements already, but indexing started anyway?

image.png

[void]

In what way? I have Terabytes of files, and that backup is just a fraction.

The EFU file list is just the filenames and indexed properties.
Not the content of the files.



Is the dying drive an NTFS drive?
-If the drive sometimes works, just add it to your index:
In Everything, from the Tools menu, click Options.
Click the NTFS tab.
Uncheck Automatically remove offline volumes.
Select your dying drive and check Include in database.
Click OK.

Let Everything reindex, which might take ages, but the drive will stay indexed, even when its dismounted.

Sorry for not understanding the whole thing. For example, how do i have 1.2 million elements already, but indexing started anyway?

Hover over the progress bar.
What is the drive of the file being indexed?

[abr01]

In what way? I have Terabytes of files, and that backup is just a fraction.

The EFU file list is just the filenames and indexed properties.
Not the content of the files.



Is the dying drive an NTFS drive?
-If the drive sometimes works, just add it to your index:
In Everything, from the Tools menu, click Options.
Click the NTFS tab.
Uncheck Automatically remove offline volumes.
Select your dying drive and check Include in database.
Click OK.

Yes, it's NTFS. And i'm not sure, if the drive will continue failing. I suspect the high temperatures to contribute to the issue, and, like i said, i had it several times over the years, with long periods inbetween, where the drive ran just fine. But just in case, i will buy another large harddisk next week, and transfer my data. At the moment, i have my smaller backup disk freshly indexed, and use that for a while.

Let Everything reindex, which might take ages, but the drive will stay indexed, even when its dismounted.

When i have the new drive soon, i'll transfer the data, and leave Everything off while doing that, to avoid unnecessary indexing action. After that, of course, i will build an index for the new harddrive, and keep my older drive as a backup. It seems to be a connector problem, rather than failing disks, because i didn't see any corrupted and malformed files.

Sorry for not understanding the whole thing. For example, how do i have 1.2 million elements already, but indexing started anyway?

Hover over the progress bar.
What is the drive of the file being indexed?
[/quote]

I don't have it plugged in right now, but i'd sayy, that they were the proper drive names, K, L, M, and R.

Thanks for your help, and i think it will all play out well.

[abr01]

Now i have a brand new harddisk, and transferred my data from the other backup disk (not the broken one), but it seems, that i can't get the new disk
indexed. Everything crashes again and again, just closing. I hardly got past 80%, now i am back to 77% maybe the tenth time of this night.

Maybe i should delete all index files, and start from scratch? In the past, the database of ET was rock solid, once i had a complete indexing run. So i hope, i will get there again, eventually.

[void]

What properties are you indexing ? -shown under Help -> Troubleshooting information -> Properties
What version of Everything are you using ? -shown under Help -> About Everything -> Version

[abr01]

The properties are

Size, Orientation, Description, Rating, Coordinates (both), and Tags. I also have added a part of the debug log, which might be the most interesting one, mentioning the rescan date.

The version used is the latest one, 1.5.0.1416b (x64).

image.png

[void]

I assume Everything is crashing when gathering a property.

Please try running Everything in verbose debug logging mode:
In Everything, from the Tools menu, under the Debug submenu, check Verbose.
From the Tools menu, under the Debug submenu, click Start Debug Logging...
Wait for Everything to crash.
Open your %TEMP%\Everything Debug Log.txt
Please check the last few files for which Everything is gathering properties.
These will show as:

get property <property-type> <filename>

What is shown?

[abr01]

In the meantime i had a crash, and then moved all *.db files to another place, to start from scratch. But strangely, no new .db file has shown up so far,
even though i'm at 50% now (in %LOCALAPPDATA%\Everything)

I had another crash a few minutes ago, but it didn't took long, to get back to where it was before. I also have activated verbose mode now, so let's see. I also noticed a probably bad setting, where i had the sizes/ratios for some video formats, which i actually don't need. I only need them for images. I have removed those video file types. Maybe that helps. If there should be another crash, i will send you the log files. At the moment, there's no debog log yet in %temp%.

image.png

[void]

Everything stores the db in memory.
It is saved to disk on exit. (File -> Exit)
You can periodically exit Everything to save your indexing progress to disk.

At the moment, there's no debog log yet in %temp%.

Please make sure Tools -> Debug -> Start Debug Logging is checked.
Everything will save the debug log to %TEMP%\Everything Debug Log.txt

[abr01]

Aha, stopping ET worked, i have a first database "snapshot" now, that could be helpful, thanks! And i also have a debug file now, which is pretty large already. Should a crash occur, i will send that log file over to you. As for the debug settings, i had to reactivate them after relaunch, both of them. I suppose, they will be appended to the older debug log. So, in case the issue happens again, which for some reason seemed to occur at around 80% (maybe some bad file?), i will submit the debug log to you.

[abr01]

Hi.

I have edited this post, after initial frustration. But eventually the indexing went through this time, after starting from 0% again. All good now!

Thank you!