Monday, January 23, 2017

With Great Deletion






Subtitle: Comes Great Paranoia
I commented yesterday that the creation of our new Webserver requires a 500GB SSD (Solid State Disk) that I would have to cannibalise from our principal working server.

The trouble is that this disk is part of an existing 4TB array of critical data.

Storage Spaces
On its best platforms Microsoft has a new-ish storage platform called Storage spaces.  You can read how I implemented this here

https://majzel.blogspot.com/2016/05/an-introduction-to-microsoft-storage.html

In particular Storage Spaces lets you combine SSD's and Spinning Hard Disks into a Hybrid Entity providing you the user much faster disk access.   Behind the scenes a scheduler moves frequently used files to the SSD and infrequently used files to the Hard Disk.

But, there is no way to simply remove the SSD component, you must destroy the volume, and all your data, and then in my case use another 4TB spinning disk on its own

Critical?
So,  on this Volume I'm destroying is: every Virtual Machine I ever cared about.  I'm talking DOS, OS/2,  My SAP systems,  Linux. Everything. In fact all 3 categories: Development, Production and Template for the last 15 years or so.

I really don't want to loose any of that.


Backup
So I made these backups, because there are few things worse than an IT guy loosing all his precious files

- Total Commander backup to another 4TB disk
- Backup to NAS
- NAS Backup to Amazon Cloud



I'd recommend all three of  QNAP for NAS Storage, the Hybrid Backup Sync QNAP application and Amazon Cloud Storage




Obviously not everybody is going to be crying about the rather slow 34 MB/sec upload speed I am getting to Amazon, but yes, I am winging about it.  Come on Amazon!

SyncBack Errors





Whilst waiting for backups to complete I tried to determine why for some time SyncBack backups have been giving multiple errors.  For the record:  Bloody Apple!

I have a ticket outstanding to SyncBack, but in essence

-Windows Explorer from a GUI can copy the files
- A cmd.exe copy cannot copy the files
- SyncBack SE can't copy the files no matter what the copy method
- ALL the files in error are Apple Computer Programs
- ALL the files start with name ._





The Big Delete



When disks are used by Storage Spaces they are no longer shown in Windows Disk Management.   So after the Storage Space Volume was deleted they now re-appear.  It's time to shutdown the system and remove these disks and replace the delete Spinning disk with the copy I made earlier.



A small test
The restarted server with the copy disk works!  Well I tested a few VM's and they all loaded.   After this event I can of course say "as expected".




Mounting and Copying


So in the data conversion phase I connected a USB 3.0 caddy with the 480 GB SSD to the Webserver I am building.

- Apache webserver is stopped
systemctl stop apache2

- gparted is used to create a new ext4 linux partion on the hard disk

- a plain old mount and copy to transfer the data. 
chmod 777 /mnt
mount /dev/sdc1 /mnt
cd /
cp -pr /data /mnt


300GB at about 60MB/second.  This will take 300,000/60/3600 =  1.4 hours


Summary
Hopefully by the end of the day I'll have successfully migrated more than  4,300,000,000,000  bytes of information without error.  That's about 80 times the size of Wikipedia.  Just saying.