Editor note: You probably would want to write a rant or downgrade something when a migration wizard fails and you lose your migrated files? Yes, yes, yes! This is what happened to Volli, he was able to partially save his files but read on for the details…
Introduction
My current Lenovo T60 laptop is at least 4 years old (nobody at my company knows anymore how old it is, as even support department doesn’t have any records about it) and is waiting for the replacement. It was running Windows XP pretty fine, if one doesn’t count some sporadic hangs and reboots. Around 1 month ago I started to experiment with different operating systems to see what’s happening in modern operating systems world.
At first I tried Ubuntu with it’s smart installer which installs the system somewhere in the existing NTFS file system which is pretty cool. except that laptop hung every other sleep…
Then I managed to install MacOS X on a separate hard drive, but I couldn’t find any hackintosh drivers for my particular wireless card (Intel PRO/Wireless 3945ABG).
Then I was back to Windows, which has Windows 7 as it’s most modern version. Well, it’s considered to be good practice to reinstall windows time after time so I decided to make a backup and do a clean install. For the first time in my more than 15 year experience with MS software I decided to use it’s own user state migration tool rather than manually copy all needed files and settings from ‘Documents and Settings’ folder. I don’t usually keep there much because of spaces in the folder names and all sorts of problems because of that, but still I got there some assorted important documents.
Fail F.A.S.T.
I happily selected ‘Files and Settings Transfer wizard’ (F.A.S.T.) from the Start menu, clicked next-next-next, let windows show me some progress bars and in ended up with a folder with a magic name USMT2.UNC and 2 files inside it: IMG00000001.DAT (with 1.2Gb size) and file called status (with a size of 16 bytes). Somehow I thought that exporting files from windows xp using it’s migration tool and then importing these settings in windows 7 sounds like a pretty normal usage pattern, but how wrong was I!
I copied all my backups to external hard drive, dropped all partitions and installed windows 7. (In the meantime I managed to fry that hard drive with everything on it, but that’s another story.) Now it was time to restore files from the backup!
So I went to Accessories menu to find something called ‘Easy Transfer’. Sounds promising… Except that it doesn’t know how to handle Windows XP FAST archive, that I had. Hmm. That’s weird. Searching on MS KB gave no pointers except that if I wanted to migrate files from an old computer I should have installed Windows 7 on a new machine, then created some magic disk for the old system, transferred files with it and then would import stuff back on Win7. How sick is that!? How the hell I’m supposed to to a clean upgrade if I must have 2 machines to transfer my stuff?
I googled for the magic folder name: USMT2.UNC. This time I found some command-line app called User State Migration Tool, which can be downloaded from MS, but unfortunately its loadstate.exe still couldn’t do anything with that poor folder. Another bit of information was that I should use exactly the same version of windows xp to restore files from such archive. (What’s the point of such transfer?) Anyhow, I installed Windows XP in a VirtualBox and was hoping to run restore procedure and then just copy files manually as I have done tens of times so far. Yeah… FAST said that I have insufficient space on my disk. How’s that – I had around 4Gigs of free space there and this stupid archive is only 1,2Gb?! So, even with the same Windows XP my ‘backup’ was useless.
Further search revealed a lot of complaining people who found themselves with such useless ‘backup’:
However, this time I found a link to a tool called fastconv.
Here is a page with lengthy description:
http://windowsxp.mvps.org/fast.htm
Ok, let’s give it a try. At the first try fastconv log said to me:
Error Source store is invalid. [ERROR=13 (Dh)]
Along with the fastconv there was one more utility called rmv2opq.exe and it was said that it
converts a removable media store into a normal compressed store
Whatever it means. Anyway, I ran it. The result was ‘astonishing’ – target folder contained the same .dat file and status file, but this time status file was 12, rather than 16 bytes! What a progress! However, this time, running fastconv.exe on these ‘new’ files gave more information – it started listing a lot of .dat files but crashed in the end. Fortunately, fastconv had a switch to keep temporary files. When I used that, it generated 2 folders: 0000/ and TRANS/.
Folder 0000/ contained 2255 files with names from 001.DAT to 8CF.DAT. When I looked inside these files I could see that these were extracted original files but with wrong names. TRANS/ folder had a binary file DB.DAT. However, when I looked inside I could see both .DAT file names mixed with original file names and all that in unicode, meaning that simple grepping wouldn’t help as unicode uses 2 bytes per symbol so all names had 0×00 between characters and everything was again mixed with binary data. Fiddling with the fastconv didn’t give any results as it kept crashing after generating temporary and database files.
Getting hands dirty
Ok, lets explore what we have. DB.DAT file’s header said: ‘m e m d b d a t f i l e v 9 n o d b g‘. Couldn’t find anything reasonable for neither opening it nor format description to write my own parser.
Then I started looked for something to extract unicode text from binary files. And found an application called BinText from McAfee, that was doing exactly what I needed. From the DB.DAT file it generated me a listing like this:
At some point I saw such pattern:
00000002CCB6 00000002CCB6 0 0000002E.DAT 00000002CD3A 00000002CD3A 0 arrow_l.cur 00000002CDB8 00000002CDB8 0 arrow_l.cur 00000002D002 00000002D002 0 arrow_m.cur 00000002D056 00000002D056 0 0000002F.DAT 00000002D0DA 00000002D0DA 0 arrow_m.cur 00000002D158 00000002D158 0 arrow_m.cur 00000002D3A2 00000002D3A2 0 arrow_r.cur 00000002D3F6 00000002D3F6 0 00000030.DAT 00000002D47A 00000002D47A 0 arrow_r.cur 00000002D4F8 00000002D4F8 0 arrow_r.cur
Which could be interpreted like this:
<original file name> <corresponding .dat file> <original file name in 8.3 format> <original file name again in some other format>
Following was simple – clean up the log and leave only original file and .dat file:
$> grep -B 1 DAT trans.txt > files.log
Which resulted in a file that looked like this:
00000002F0A2 00000002F0A2 0 beam_r.cur 00000002F0F6 00000002F0F6 0 00000038.DAT -- 00000002F442 00000002F442 0 beam_rl.cur 00000002F496 00000002F496 0 00000039.DAT -- 00000002F7E2 00000002F7E2 0 beam_rm.cur 00000002F836 00000002F836 0 0000003A.DAT --
Then I wrote a small program to parse this log and rename files (handling of duplicates was important as I couldn’t restore directory structure). Before final renaming I ran it several times with writing logs to spot all problems in the log file.
Ok, I lost the directory structure and some file names are still broken, but at least I got my files back and I can fix these small issues by hand.
Lessons learned? Don’t blindly trust ‘migration’ software and keep good ol’ files handy.
P.S. Post named after the song ‘I fought the law’ by Dead Kennedys.