Wednesday, November 02, 2016

SuperMicro Post LED Code 79



Subtitle: When a Plan Comes Together

Whilst the title of this post might seem a tad specific I encourage you to read on.

I'm documenting working past one of a number of heart stopping problems I've been facing in my new Server Build.


Background: On a Budget
One could regard it as a challenge, but in fact not really.  In Switzerland, returning goods that are not faulty is rarely possible without penalty if at all.

So if you buy a camera, or a phone, or indeed a motherboard that may not work how you like it, but is technically not faulty.  Then tough.   No full refund and often not even a partial refund.

When I see people in America buying 4 different phones, doing YouTube reviews on them and then sending back all but one (or even all), just because they can.  Well you can't do that in Switzerland.

So building a new system is easiest therefore

- If you choose your components perfectly
- Perhaps you copy another persons design  (well mine after I document it here perhaps!)
- If everything arrives and is not faulty, because even if genuinely faulty you may have a struggle to return it
- Maybe you work /in the business/ and have access to test equipment, e.g spare CPU's and memory to aid in fault finding and experimentation

Marcus get's none of these breaks. 

Worse still, I am on a limited budget, so I can't afford to buy a new Nvidia 1060 GTX Graphics card, and if it does not work, and since I can't return it, just put it in a drawer at home and "chalk it upto experience"


The Problem Stated

I bought a SuperMicro X10-DRi-T motherboard as part of the masterplan to build my 2016+ Workstation / Personal Server

But when I plugged in the Xeon CPU the machine would not boot. In fact it does not even get out of the BIOS:

It hangs on Post Code 79,  System Initialization  (System Initialisation)




Whooa that Motherboard



The DRi-T motherboard  was carefully chosen because

- SuperMicro are the king (to me) of Server motherboards.  No other manufacturer comes close to their variety
- This board has 16 memory slots for upto 2TB of memory.  I am not kidding.
- In fact >32GB DIMMS are ludicrously expensive, limiting you to only 512GB in practise. Surely enough? 
- Dual Processor: The best way to get a lot of cores/threads at an acceptable cost
- USB3 native on motherboard
- 3 PCIex16 slots.   Whilst you only need 1 for a decent graphics card, 3 are better and some may be obscured due to things in the way like CPU coolers or memory DIMMS etc.   With 3, gen 3.0 slots surely it's going to work
- 10GB Ethernet:  Well its good to have it, even though today the multiport Switches to handle it, still don't exist below 3K USD.
- IPMI: Remote support of the machine and its state  (though see later :-(((  )



All is not Well
When a motherboard starts and put something to the screen it's already a very good sign.  It means
- there is a valid CPU working
- there is valid memory
- The processor started and executed the BIOS ROM

A Lucky Break
The "Big Xeon" processor I bought did not work.  So I contacted the company and bought a second, entry level, cheapo, cheapo Xeon.  NB: If you know the price of Xeon, I will say that cheapo is a sort of relative comment



2 weeks later that arrived.

To my delight putting that in gave full BIOS instantly without error.  So is the original processor faulty?   Not so fast.  In fact since the "Big Xeon" processor was executing BIOS the CPU is certainly operational.  So probably not faulty.  Hmmm

.... Why lucky then?
Well because having a working BIOS means I can start the BIOS and then use the internal menus to upgrade both BIOS code and IPMI code


The Obvious Stuff


Update BIOS



I'm only showing part of the BIOS update.  There is more, a lot more.


Update IPMI

IPMI allows for remote power on/off and manipulation of your server from a webscreen remote from the actual system.






But heartbreak city.  After upgrading both the BIOS and the IPMI firmware the problem was still there.  I was gutted and despondent.


I called Support
SuperMicro Support Europe is in the Netherlands
Phone:   +31-73-640-0390

Hours:   8:30am-5:30pm GMT + 01:00 (Monday - Friday)

The things I learnt from SuperMicro Support

- You can register at the website
- After register you can create a support ticket
- But after creation you can't add to the ticket or see it!
- You have to choose the Geography for the ticket.  I chose Europe
- So then somebody in Europe should be looking at the ticket queue and will eventually respond via email
- You can Google the problem and it vectors to the SuperMicro support published tickets with answer page
- I actually spoke to somebody,  I nearly fainted.
- And doubly fainted, because the nice man, called Paul, was calm, knowledgeable and sensible.  Oh, the joy of actually talking to somebody who knows their stuff


The Workaround


Move the JPB1 from position 1-2  (ON) to 2-3 (OFF)

Doing so fully disables the IPMI web interface and processor completely.

Then instead of hanging at code 79, the Processor completes the boot.





Hell Yeah!

I'll translate this into sounds waves for the non technical





LLCJ: Mama


Lessons Learnt

- The Pleasure of Good Support
I spoke to Paul at SuperMicro.  I was actually thrilled.  He was calm and knew his stuff.  He spoke perfect English. He came up with a logical set of suggestions to follow.  After the initial voice call I could not get thru again and had to maintain an email dialog.  It was still a terse yet fruitful one.

- It's never Instant
I am still inching forward with my server build.  With other commitments and the snail like delivery of some parts, another 2 weeks for a memory card for example, well this could take until Christmas

- It's never Easy
If you don't do this as your job, or regularly, you are at the mercy of the famous Gotcha.  After which is the Doh! and then hopefully the recovery.

Further if you come to an unexpected problem, many times you are at the mercy of Google to come to your assistance.

And so in this end, I hope this post was helpful and of use to Enthusiast Systems Builders everywhere.






See Also AMI Bios Post Codes
For Supermicro X10 DRi-T
Checkpoint Ranges
Status Code Range Description
0x01 – 0x0B SEC execution
0x0C – 0x0F Sec errors
0x10 – 0x2F PEI execution up to and including memory detection
0x30 – 0x4F PEI execution after memory detection
0x50 – 0x5F PEI errors
0x60 – 0x8F DXE execution up to BDS
0x90 – 0xCF BDS execution
0xD0 – 0xDF DXE errors
0xE0 – 0xE8 S3 Resume (PEI)
0xE9 – 0xEF S3 Resume errors (PEI)
0xF0 – 0xF8 Recovery (PEI)
0xF9 – 0xFF Recovery errors (PEI)

SEC Phase
Status Code Description
0x00 Note used
Progress Codes
0x01 Power on. Reset type detection (soft/hard).
0x02 AP initialization before microcode loading
0x03 North Bridge initialization before microcode loading
0x04 South Bridge initialization before microcode loading
0x05 OEM initialization before microcode loading
0x06 Microcode loading
0x07 AP initialization after microcode loading
0x08 North Bridge initialization after microcode loading
0x09 South Bridge initialization after microcode loading
0x0A OEM initialization after microcode loading
0x0B Cache initialization

SEC Error Codes
0x0C – 0x0D Reserved for future AMI SEC error codes
0x0E Microcode not found
0x0F Microcode not found

PEI Phase
Status Code Description 
Progress Codes
0x10 PCI Core is started
0x11 Pre-memory CPU initialization is started
0x12 Pre-memory CPU initialization (CPU module specific)
0x13 Pre-memory CPU initialization (CPU module specific)
0x14 Pre-memory CPU initialization (CPU module specific)
0x15 Pre-memory North Bridge initialization is started
0x16 Pre-Memory North Bridge initialization (North Bridge module specific)
0x17 Pre-memory North Bridge initialization (North Bridge module specific)
0x18 Pre-Memory North Bridge initialization (North Bridge module specific)
0x19 Pre-memory South Bridge initialization is started
0x1A Pre-Memory South Bridge initialization (South Bridge module specific)
0x1B Pre-memory South Bridge initialization (South Bridge module specific)
0x1C Pre-Memory South Bridge initialization (South Bridge module specific)
0x1D – 0x2A OEM pre-memory initialization codes
0x2B Memory initialization. Serial Presence Detect (SPD) data reading
0x2C Memory initialization. Memory presence detection
0x2D Memory initialization. Programming memory timing information
0x2E Memory initialization. Configuring memory
0x2F Memory initialization (other)
0x30 Reserved for ASL (see ASL Status Codes section below)
0x31 Memory Installed
0x32 CPU post-memory initialization is started.
0x33 CPU post-memory initialization. Cache initialization
0x34 CPU post-memory initialization. Application Processor(s) (AP) Initialization
0x35 CPU post-memory initialization. Boot Strap Processor (BSP) selection
0x36 CPU post-memory initialization. System Management Mode (SMM) initialization
0x37 Post-Memory North Bridge initialization is started.
0x38 Post-Memory North Bridge initialization (North Bridge module specific)
0x39 Post-Memory North Bridge initialization (North Bridge module specific)
0x3A Post-Memory North Bridge initialization (North Bridge module specific)
0x3B Post-Memory South Bridge initialization is started
0x3C Post-Memory South Bridge initialization (South Bridge module specific)
0x3D Post-Memory South Bridge initialization (South Bridge module specific)
0x3E Post-Memory South Bridge initialization (South Bridge module specific)
0x3F – 0x4E OEM post memory initialization codes
0x4F DXE PIL is started

PCI Error Codes
0x50 Memory initialization error. Invalid memory type or incompatible
memory speed
0x51 Memory initialization error. SPD reading has failed.
0x52 Memory initialization error. Invalid memory size or memory modules
do not match.
0x53 Memory initialization error. No usable memory detected
0x54 Unspecified memory initialization error
0x55 Memory not installed
0x56 Invalid CPU type or speed
0x57 CPU mismatch
0x58 CPU self test failed or possible CPU cache error
0x59 CPU microcode is not found or microcode update is failed.
0x5A Internal CPU error
0x5B Reset PPI is not available.
0x5C – 0x5F Reserved for future AMI error codes

S3 Resume Progress Codes
0xE0 S3 Resume is started (S3 Resume PPI is called by the DXE IPL).
0xE1 S3 Boot Script execution
0xE2 Video repost
0xE3 OS S3 wake vector call
0xE4 – 0xE7 Reserved for future AMI progress codes
S3 Resume Error Codes
0xE8 S3 Resume failed
0xE9 S3 Resume PPI not found
0xEA S3 Resume Boot Script error
0xEB S3 OS wake error
0xEC – 0xEF Reserved for future AMI error codes

Recovery Progress Codes
0xF0 Recovery condition triggered by firmware (Auto recovery)
0xF1 Recovery condition triggered by user (forced recovery)
0xF2 Recovery process started
0xF3 Recovery firmware image is found.
0xF4 Recovery firmware image is loaded.
0xF5 – 0xF7 Reserved for future AMI progress codes
Recovery Error Codes
0xF8 Recovery PPI is not available.
0xF9 Recovery capsule is not found.
0xFA Invalid recovery capsule
0xFB – 0xFF Reserved for future AMI error codes
DXE Phase
Status Code Description
0x60 DXE Core is started.
0x61 NVRAM initialization
0x62 Installation of the South Bridge Runtime Services 
CPU DXE initialization is started.
0x64 CPU DXE initialization (CPU module specific)
0x65 CPU DXE initialization (CPU module specific)
0x66 CPU DXE initialization (CPU module specific)
0x67 CPU DXE initialization (CPU module specific)
0x68 PCI host bridge initialization
0x69 North Bridge DXE initialization is started.
0x6A North Bridge DXE SMM initialization is started.
0x6B North Bridge DXE initialization (North Bridge module specific)
0x6C North Bridge DXE initialization (North Bridge module specific)
0x6D North Bridge DXE initialization (North Bridge module specific)
0x6E North Bridge DXE initialization (North Bridge module specific)
0x6F North Bridge DXE initialization (North Bridge module specific)
0x70 South Bridge DXE initialization is started.
0x71 South Bridge DXE SMM initialization is started.
0x72 South Bridge devices initialization
0x73 South Bridge DXE initialization (South Bridge module specific)
0x74 South Bridge DXE initialization (South Bridge module specific)
0x75 South Bridge DXE initialization (South Bridge module specific)
0x76 South Bridge DXE initialization (South Bridge module specific)
0x77 South Bridge DXE initialization (South Bridge module specific)
0x78 ACPI module initialization
0x79 CSM initialization
0x7A – 0x7F Reserved for future AMI DXE codes
0x80 – 0x8F OEM DXE initialization codes
0x90 Boot Device Selection (BDS) phase is started
0x91 Driver connecting is started
0x92 PCI Bus initialization is started
0x93 PCI Bus Hot Plug Controller initialization
0x94 PCI Bus Enumeration
0x95 PCI BUS Request Resources
0x96 PCI Bus Assign Resources
0x97 Console output devices connect
0x98 Console Input devices connect
0x99 Super IO initialization
0x9A USB initialization is started. 
0x9B USB Reset
0x9C USB Detect
0x9D USB Enable
0x9E -0x9F Reserved for future AMI codes
0xA0 IDE initialization is started
0xA1 IDE Reset
0xA2 IDE Detect
0xA3 IDE Enable
0xA4 SCSI initialization is started.
0xA5 SCSI Reset
0xA6 SCSI Detect
0xA7 SCSI Enable
0xA8 Setup Verifying Password
0xA9 Start of Setup
0xAA Reserved for ASL (see ASL Status Codes section below)
0xAB Setup Input Wait
0xAC Reserved for ASL (see ASL Status Codes section below)
0xAD Ready To Boot event
0xAE Legacy Boot event
0xAF Exit Boot Services event
0xB0 Runtime Set Virtual Address MAP Begin
0xB1 Runtime Set Virtual Address MAP End
0xB2 Legacy Option ROM initialization
0xB3 System Reset
0xB4 USB hot plug
0xB5 PCI bus hot plug
0xB6 Clean-up of NVRAM
0xB7 Configuration Reset (reset of NVRAM settings)
0xB8 – 0xBF Reserved for future AMI codes
0xC0 – 0xCF OEM BDS initialization codes
DXE Error Codes
0xD0 CPU initialization error
0xD1 North Bridge initialization error
0xD2 South Bridge initialization error
0xD3 Some of the Architectural Protocols are not available
0xD4 PCI resource allocation error. Out of Resources 
0xD5 No Space for Legacy Option ROM
0xD6 No Console Output Devices are found.
0xD7 No Console Input Devices are found.
0xD8 Invalid password
0xD9 Error loading Boot Option (LoadImage returned error)
0xDA Boot Option is failed (StartImage returned error).
0xDB Flash update is failed.

0xDC Reset protocol is not available.