Wednesday, December 16, 2015

Human Error meets OCD and I Never Noticed That

Synopsis: This is a tale of stupidity but ultimately redemption.

I am currently testing out GrooveMusic application,  which I suppose is Microsoft's answer to Spotify, the Swedish music service that I rather like and pay for.

To me GrooveMusic primarily, or maybe encourages you to play your own music that is either stored locally or in the Onedrive cloud. So  I thought I would Cloud upload some songs as a test.

Imagine my surprise when I saw 1205 errors as shown above.

This is an example of the problem.  See it?  No, neither did I at first.

With hindsight on my side here are the corrected filenames. Any the wiser!

Well it is the fact that the filename STARTS WITH A SPACE character.

I was absolutely shocked and used my UNIX command line under Windows to find instances of this problem

This says print out all the .mp3 files that start with a space, followed by any character sequence, including further spaces or characters and ending in .mp3 extension

I was shocked to find well over a thousand problems.


Cygwin is a very complete way to run UNIX (well GNU) commands on your Windows Operating System.

Simply goto, download the setup executable.  Run it.  It connects to the Internet via one of a set of named mirrors and installs all the necessary programs.  It can occupy over 10GB, my installation is about 5GB.  Once installed you open up a terminal command line (rather like DOS) and enter your UNIX commands into it.  All your drive letters are mapped underneath a /cygdrive filesystem structure, and, as per UNIX the directory separator is a / not a \.

The fixing commands used below are simple UNIX commands run in that korn shell command line interpreter

How did this Happen
After a long period of denial, which included wild and childish accusations that Agata must have sneaked onto our server and deposited garbage files, I eventually concluded that my use of MP3tag was faulty.

I discuss MP3Tag here, it's the best ever way to rename your music filenames and metadata that I have ever seen

This is the bad format string (there is a space before $num)

This is the good format string

I use format strings to name the music files based on the internal mp3tag metadata.  But you can easily do it the other way around filename -> metadata and about a zillion other options.

So all songs should have a uniform file naming scheme and internally consistent mp3tag information.

And it should not, definitely not include a space at the beginning of the filename!

The OCD Answer

 Well since my local VLC media player is quite happy with these files, and since I primarily want to use Groove Music with Streamed Microsoft Internet content (not local or cloud content), this error is sort of a mute point.

But it has been bugging me!

a) how is it possible that I never noticed this, ever!
b) It has been going on for years, oh, the shame.
c) I didn't even know about some programs not liking filenames beginning with a space, more shame.

Time for correction

 This UNIX command finds all the problem files and then executes my custom program renfile on them, and puts the output of renfile into a text file called doit.

This is the renfile program

It expects an argument which is the bad, space prefaced filename.  If checks for this itself and if found generates the UNIX rename command (mv) necessary to create a new file without the leading space

So, the output is channelled into file doit which is simply a set of one line move commands.

So I had a look at the file to see it looks reasonable and is not going to obliterate in some way my music collection!

I can then execute this list of command with

ksh -x ./doit

And so over 1000 files were changed.  Gulp!

We had it tough!
I'd additionally like to point out my original Computer systems used,  pre DOS used the 8.3 character naming scheme.   So there were no long filenames and all you could have was

1-8 letters, case insensitive, no spaced
3 letter extension

Here is an example of my Cromemco CDOS 02.54 operating system from 1981.

When IBM PCDOS and Microsoft MSDOS first came out they copied the 8.3 standard and then sometime later IBM with OS/2 and Microsoft with Windows 95 came up with opposing schemes to enable Long Filenames i.e. those with typically upto 255 characters including upper and lower case alpha and many punctuation and special characters.

So all my early music and media and computer files had cryptic 8.3 names, not including spaces.  You kids these days have it easy, I just want to say it!

So, is all well?
I think so.  The paranoid step of creating a command file before blatting my master system was just to reassure me.

I did execute the doit command files I created.  It changed the source files.   These get staged to a new disk and then pushed to Onedrive cloud for use by PC or Windows Mobile GrooveMusic app

All the errors have gone. :-)