Look Around and Ask Why

Geoff was busy at his desk.  The morning was already filled with moments of regret -- mostly the regret of not calling in sick.  The billing system misfired over the weekend and spit out errant bills for hundreds of customers of Bright Star Power and Gas.  The calls were coming in pretty briskly, and tensions were high as the billing system has been put on hold until a fix can be put in place.

Stop the bleeding then fix the data.  Get the customers whole again.  A routine that is performed all too often.  Geoff didn't realize it yet, but he was the reason for all this.  He was the reason for the last dozen or so of all this.

The billing system has been in the wild for almost a decade now.  It started out as a shining example of a new system built the right way.  The perfect way.  Flexible, forward thinking, the last system that needed to be built.  Perhaps it would have stayed that way if the original, motivated team stayed together.  If requirements didn't evolve and a system that didn't quite evolve in step.  If budget pressures didn't overload that team.

One by one, that team dissolved with the exception of Geoff and a seemingly never ending rotation of developers.  This billing system could not keep talent.  It was not shiny and new any longer.  It now had band aid after band aid holding it together -- at least that is what is seemed.

Things were starting to turn around.  New leadership came in and have provided support for Geoff.  A new team was formed that was motivated to bringing out the best of this system and looking for opportunities to give it a much needed renovation.  The bones of the system was still pretty solid.  A rewrite was not going to happen, but there is hope once they realize some of the simple habits that can quickly stabilize things.

While there was a new team, new vision, and new hope, Geoff was stuck with the old habits.  When an issue came up, a quick, targeted fix was put in to make the problem go away.  An entanglement of conditional statements was his mode of operation and this latest fix was about to be the latest to be piled on.

Geoff, meet John.  John has been here before.  Has fought these battles and learned some good, simple lessons.  Keep it simple.  Before diving into a fix, look around.   Ask yourself why is this code the way it is.  Why does one fix here lead to more problems later requiring just one more fix.

In response to this recent incident, John challenged Geoff to ask himself these questions.  For the short term, those questions produced a very targeted fix that stopped the bleeding.  It also prompted Geoff to take a closer look after the dust settled.  He created tests to confirm what he expected the behavior to be and learned about some behaviors that he didn't quite expect.  This closer inspection and added scrutiny ultimately tamed this nasty bug generator -- freeing up support time for innovation time.

It didn't stop there.  It was a simple lesson learned that really resonated with Geoff.  From this point on he takes a look around.  He asks why.  He tests his assertions and learns how to improve this big system one thorough fix at a time.

Looking ahead, the system is stabilizing.  Modernization efforts are now being taken on by the team.  The system is becoming more nimble.  Problems are now more often found through proactive search and destroy missions and less often when it hurts most.

Look around.  Ask why.