Wednesday, July 24, 2019

Upgrading the NVMe Array Very Carefully

Upgrading the Array

This is a positive story about the upgrade of my NVMe based storage array,  that is the highest storage tier on my workstation.

No Moving Parts
Trying to run a computer with minimum spinning parts has been a passion since before 1980.  Actually in 1980 I already had a swap-less UNIX computer system that ran entirely in RAM memory.

That early UNIX system did have floppy disk storage of course  (no hard drives until about 1983) but the mantra was already set.

Whenever possible: Do it in RAM Memory not disk

In 2019

My computer storage hierarchy looks like this.  

- At the top level are some NVMe disks striped together into a 4GByte/second plus array.  This sits in a Windows Server evironment with Virtual machines running on top of that platform.

- The NVMe array is flushed every 24 hours to a physical disk set

- That Data Disk set is then sent to ...

a) A variety of 3 different cloud providers.  But living in slow (sub Gigabit upload rate) England only the smallest critical fraction is uploaded

b) The Data Disk feeds a NAS

c) NAS backups get exported to a local disk which is updated and kept with a friend

d) The data disk also makes full mirror backups to external disks which are taken out of country.

The consequences are:  SCREW UP your NVMe array and it percolates down and poisons all of your backups.

The NVMe array already contains over 80 years worth of photographs.   Several Terabytes.  If it was lost, or in any way corrupted, well, I would be more than upset

The Careful Upgrade

Existing Position,  NVMe array is 3.64TB usable

 Online I can see the new disk and add it online

 Great the Array is now 4.54TB

 Shall we extend the Array and then everything is ...

 Ah wait, you see once the Virtual disk is created in the Storage Pool it can't be enlarged

So we will have to delete it all first .   Damn!

 So not unreasonably you can't delete the Virtual disk until the underlying (in this case) NTFS volume is first deleted.

 Deleting the cache X: drive

 What is this critical system partition?

It turns out that I had placed a token (never to be used one hopes) Paging file on the drive to be deleted.

 Move the Paging file to the C: drive and reboot, then delete the X: using Disk Management.  It is gone!

The Backups I Made

In addition to making sure all the usual offsite and NAS backups had completed, I made 2 extra full backups using the ancient product Total Commander

TC has the advantage of copying files  (when configured) with the correct creation date and time from the original.  Windows and many other programs lose this information.


 NTFS not ReFS must be selected otherwise Dropbox cannot reside here, and it should because by definition this is the fastest storage tier on the system.

Oops DropBox

On restart of Dropbox it decided that the empty directory on the workstation meant it should therefore cascade delete every file on Dropbox everywhere!

Dropbox does have a 30 day restore, but of course it gets the file and directory dates wrong :-(    So I had to restore these files from a backup

Some Checking

After some hours of manual checking, at random.  I ran the SyncBackPro program in Simulation to see what files it thought were more uptodate on the new NVMe array.  I had made a few photo uploads that day and so I was delighted to see that these few files were more uptodate on the source NVMe array.  

It worked then!

The Array is now 1TB larger and all the original data is restored.

I can rest easily again.

Until the next upgrade :-)

Tuesday, July 23, 2019

Eleanor Oliphant Is Completely Fine

How many of us live a Perfectly Fine Life?  And how would we know?  Many of us realise in later years that our adolescent upbringing was not fine by any normal standard.

More than anything I believe that your environment, the attitudes of your parents towards you and to others, form a solid mould that normally irrevocably sets the course of your future life.

If you were subjected to hatred, self doubt, violence, sexual or racial abuse, this can seem to be impossible to recover from.

But sometimes, just sometimes you can break that mould, either by force of mind, or sometimes even by chance circumstance.

And So

I'd very much encourage readers to listen to

BBC, Book at Bedtime, Eleanor Oliphant Is Completely Fine

10 Episodes,  15 minutes each. 150 minutes of your time.  

I feel that you will be a better person at the end of this tale

And Finally
The British Broadcasting Corporation (BBC) and in particular Radio 4 is a treasure trove of high quality audio broadcasting for grown ups.

It is available worldwide to listen to, either from the Webpage or my favourite the Android App BBC iPlayer Radio

What a wonderful resource. Please use it liberally and enjoy.

Monday, July 22, 2019

24 Hour Thunder Run 2019

Those with long memories may recall that Marcus and Agata participated in July 2011 24 Hour Thunder Run described  here and here.  Hey, we even managed a Podium Place.

This year we were invited to join a Relay team of 8 to participate in the same run, still at Catton Park.  It has now changed sponsorship and is the

Continental 24 hour Thunder Run

We quite fancied doing another team of 2 together, and trying to see if we could beat the 220km that we ran previously. But we also wanted to support our Running Club, namely Run Alcester who setup the relay entry.  After they had 2 cancellations we joined the relay.

Then, at the last minute one of our number withdrew for personal reasons. So the stage was set of a team of 7, for this 24 hour, trail running event, at Catton Park July 20 2019

A sign of the Times

In 2011 we came from Switzerland and managed to park our car relatively close to the course.  But in 2019 the event has become exponentially more popular.

We were told to ideally camp somewhere nearby, outside of the race, at a campsite on Thursday night. Queue for the car park / campsite on Friday my 08.00 and then wait around until Saturday noon 12.00 for the race start.

This is a symptom of how insanely popular these events are becoming. So in fact we prepared our Motorhome during Thursday, got it 100% ready and then got up far too early on Friday to get to the Catton Park Campsite for 08.00. Flup!

The Big Picture

We are a team of 7 people doing a Relay 10km loop.  One runner goes out, does a 10km then hands to another runner.  Every relay team must have a runner participant do at least one lap within the 24 hour period.  So teams with one weak runner can just let them do literally one lap and split the other time between n-1.

In our case, we devised a rota and a timing schedule for everybody based on our expected 10km trail time. We planned to modify this as people got tired, or if the weather got bad (ie. rain) or in the night when you need to slow down to avoid the many roots stickup up out of the ground trip hazards.

I only have a picture of the initial estimates, but the paper was a mass of corrections after 24 hours I can tell you. Marcus ran much faster than I predicted and Tony and Julie ran slower :-(


Richard/Lyn(helper) and Marcus/Agata have motorhomes and the others were all shacked up in tents.   As nice people we all got together to help with tent assembly.  The rule is, there can never be too many people to help assemble a tent!

There was also a Gazebo where we can get together and see competitors running past whilst being protected from the elements namely British wind and rain.


There were a few exhibits from previous years including T shirts from last 10 years of so of races.

There was a pre race Yoga Class

The changeover pen as the name suggests was for new contestants to wait.   People coming in from a lap went down the side and you handed over your wristband to a fellow team member.   The team member on the course needed to have this band on,  all competitors needed their ankle tracking bracelets and their race numbers visible.

Solo competitors did not need to stop  (they are solo!) so they had a special track on the right.  Equally if you are doing more than a single lap before handing over you would use this lane.

There is a water station at the finish and one at the 5km half way around the loop point.  Serves only water.

Bands, Timing, Medals and Dossard

Each team member gets their own timing chip and it works like this: All teams start at Saturday noon and the clock starts ticking. Then when your team member runs over a finish mat at the 10km conclusion their time stops, and the time is then started for the next team member. Actually at this point it is unknown who that is as there is no starting timing mat.

The next team member gets going and when they cross the 10k line then their time is calculated by subtracting their finish time from the n-1 person in that team.

Like a High school disco we needed to wear 2 bracelets. The striped on means part of the Thunder Run event, but you could just be a team helper etc. The other bracelet says that you are a trail runner competitor.

Marcus and Agata's numbers

At the end of the race you hand back your timing chip and get back a medal. Quite effective strategy to encourage you to return them chips!

Some Videos

Thunder Run 2019 Start

Agata Arrives

Julie Runs

Lisa Runs

Tony Runs

The Weather

Trail race difficulty in England is hugely affected by the weather.  Although we have had almost exclusively dry and sunny weather past 20 degrees C for the last 4 weeks at home in Alcester, the forecast for the weekend race was >8mm rain on Saturday and some smaller amount on Sunday.

To spell it out:  In the UK trails are mostly on muddy paths.  There are very few mountain style rocky paths.  If it rains heavily these paths  become mud baths.  And if hundreds, or in this case over 1000 competitors are running around a 10km loop, it just makes it muddier and muddier as it is churned up.

Suffice to say that members of our team including Agata were out in torrential rain.   The above picture is where I had just handed over to Agata, got back to the motorhome and then for most of her lap it was absolutely tipping it down

A good way to check is a visual of the backs of ones legs and the amount of mud on the shoes.


Trail shoes are like road shoes but with a stiffer base.  The idea is that you pass over stones and uneven surfaces and that flexing of only a road shoe would lead to instability.  Additional factors like thicker side walls help to guard against situations where you hit roots or stones on the sides of your foot.  Overall I don't recommend that anybody merely uses road shoes for trail.

Agata took 2 trail shoes variants and Marcus 3. But due to the weather conditions it was only going to be one of the above two that I used. Ones not shown are strictly for dry trails and ones preferably not on grass but on mountain gravel.

For Marcus the Black Salomon Speedcross 4 Gortex and the bullet proof wet weather shoe that can't be beat.  Used on the Arc of Attrition  in 2019.

But for mud specifics then I'd advocate the Inov Mudclaw 300  This is not at all waterproof but is suitable for mud mud and mud. Each shoe is 300 grams, with low drop.

Walking the Course

Yes, if the course is a loop why not walk the course ahead of time.  This gives you opportunity to see the racing line, look out for difficult sections, and in our case remove debris that could trip us and others up.  We spent over 2 hours on Friday doing mostly the latter  (picking up crap) and making sure the route was as pristine as possible for all.

We noted that the Organisers had marked and chopped some stumps on the earlier part of the course, but seem to have given up on the later stages. Ah well!

Pre Race Temptations

We had a whole 24 hours post tent setup before race start and the onsite food facilities were typical of long distance trail events ... unhealthy.   In trail running psyche there is a strong tradition that unhealthy foods like cake, fish and chips, fatty burgers are performance enhancers for Ultra Distance competitions.

Marcus and Agata are 180 degrees out of favour with this opinion. In daily life are bodies are rather sensitive, fat and carbo laden foods, piles on our kilos and reduces are fitness and our ability to compete.

So we brought large portions of Lentils, Yogurts and Marcus's specialities of Rice Pudding and Jelly babies.

Race Report
All but one of us assembled on Friday and had rather too much time to acclimatise, eat unhealthy foods and in Marcus and Agata's case walk the course.   The productive time we spent included

- Assembling the tents
- Getting our race numbers
- Deciding on a running order
- Make an estimated time plan so we knew roughly when we could goto sleep and set and alarm to getup for our next 10km segment
- Selecting our clothes
- Worrying if it really was going to tip it down with rain all Saturday as promised.

The race started on time and Corin was #1 in our team sequence.   We soon established a pattern and but for the fact that Marcus' presumed 60 minute 10k trail estimate proved too long, we had a routine going.

Although it rained hard on Friday night, the course was relatively dry at race start. There were a few quite sticky mud sections, turns out it would get a lot worse!

No sooner than Marcus finished his first lap and handed over to Agata the heavens opened.  This made the course a mud quagmire.   All competitors and our team included then had a much more difficult course.

Marcus' next run was at past 20.00 and we had to carry head torches and the trail had turned to a muddy soup in many places. For Marcus my Inov Mudclaw 300 as below provided a real advantage with the blocky tread pattern being optimised for assisting with traction especially on sections with bad camber.  Camber and heavy mud equals sliding and / or falling over unless your shoes and balance can save you.

We all got to do at least  night lap and the real challenge was to modify the running order times based on the previous running times.  And, iff you are runner n, then alerting runner n+2 that they will need to get up in about 40 minutes since you just handed over to runner n+1 

Apart from the heavy downpour on Saturday there were no other weather extremes, and by Sunday daylight the course had both dried and a visible racing line had developed.  

The forest section was still very tricky as there were literally hundred of exposed roots on the path that you had to be careful to jump over whilst at the same time overtaking other competitors.

All of us took the strategy that after a 10km running loop we would have some food and then set our alarms to wake us up for the next loop with some margin for error.  None of us took rucksacks or food for the loop.

We modified the running order for the last laps so that we could best guarantee 3 of us would get 4 laps in total

And here are our individual results

Our Team

Richard doing a warm up

Agata,  looking good of course



Coming back from a lap, with helpers Lizzie and Mark

Bumped into another Run Alcester member not running for us!

Lisa and Corin

Our Gazebo with Lyn on the left from where we cheered our team and others around

Corin storming round

Tony, Julie, Lisa, Agata, Richard, Marcus


Example handover.  Agata is coming in and takes off her bracelet

Corin puts on the bracelet inside the handover area

And he is off
As fast as he can go

Other People

Lots of partially naked men


Spider Man and woman are here every year we understand.

Last Lap

So you are allowed to finish any lap started before 12 noon Sunday, so we changed the running order for the last team relay.

Marcus who was putting in sub 50 minute circuits including the one in mud, mud and mud replaced Corin as #1 in sequence.  Then Agata, and so Corin was able to start as 3rd position at 11.30 meaning his finish would be about 12.10 ish

We are all waiting 200 metres from the finish

Other teams doing the same thing

Corin is spotted and we join him and then all run in together as a team

There really were so many competitors!

Extended Team

Mark and Lizzie  (visually impaired runner at Run Alcester) came to help cheer us to the finish.   Their good humour and friendliness was appreciated as ever.

The 24 hour Thunder Run is not cheap, but we think it is well organised, and they have coped with the huge increase in popularity, still being able to offer a decent event, with facilities for all competitors.  There are solo, duo and other team categories, so  you can pick which kind of trail entry suits you best.  Whilst the weather is an unknown factor, even in this year with medium rain, competitors like ourselves had a wonderful time. Well worth it!  We have to decide whether to put an entry in for 2020 and in what category.