Caché Developers and Change Control

The following should be construed as a rant, and is not to be taken (too) seriously … well, …

We had an interesting “event” the other day, whereas a core display routine was changed and, under certain provider schedules, would get caught in a non-terminating loop.  After a few dozen of these processes got created amongst the other 600+ concurrent users, the quad core server was grinding at around 70 load average, when it typically runs at less than 4.  Although the Red Hat Enterprise Linux server was busting away doing essentially what was nothing, it was still “usable” enough for me to triage the problem and “suspend” those runaway jobs, until clinical development corrected the loop.  While normal system and user access resumed, what ensued within I.S. was something not totally unexpected — at least to me.

Our Change Control Board got its report of the event, and learned that our production clinical system — which is internally developed and is absolutely core to all hospital operations — receives numerous changes per day, in which nearly all changes are not communicated to any other division within Information Services.  It was reported by one trusted source that changes averages >12 per day. One might immediately think that this is an incredulous amount of change, and how does one maintain a true auditing system to prove out what was in effect at any given moment?  But, given its good track record of solid performance and availability, someone else might think that this volume of change — with very few incidents — is a credit to great (or “good enough”) internal process that it does not necessitate any need for external communication.

How does one reconcile this wide variance in speculation?  How about starting with a call for facts?  I, for one, relish the moment ANYONE wants to see or learn more about what it is that I do. Yeah, but try telling that to a bunch of old grumpy developers (or at least to their managers), even if you consider them esteemed colleagues and friends.  NOT!

Well, I am no stranger to this problem, having worked both sides as the application developer and management over application development.  Should an efficient and cost-effective information system and its staff be allowed to continue status quo, because while “the cow is still giving milk, there is no need to butcher it for its meat”?  But it is when that cow occasionally jumps the fence, which in turn requires an elite team of hands to corral and bring it back into the barn, that it has management sit back and ponder on whether there is a need to make better fences… or if that is not feasible, find someone to blame for allowing the cow to get loose in the first place.

Ah-hah!!  So, it is the FEAR that if it is NOT change in process, then it must be change in accountability.

Somehow, someone decided that if you do a bang-up job, then you are “entitled” to continue to pursue whatever course you decide is best.  I was brought up that if you do a bang-up job, you may be allowed increased empowerment (and certainly NOT the pay), but along with it comes the accountability that goes with that entitlement.  I suppose the “freedoms” that is inherit to software development becomes so natural to their thinking process, that associated behavioral responses are somehow skewed into believing the same crap that they write, i.e.,

            ALL LOVES ME;
            ME IS GOD;
            ALL IS STUPID;
            GOTO 10;
99  END

To no surprise, this program is only run by its author.  And so this is where the “fun” always starts within I.S. — a call for process change (followed by accountability).

It is absolutely amazing to me that information technology professionals can be the worst to deal with when asked to re-examine incumbent processes for improvement.  You know how they say doctors make the worst patients?  Well, IT staff make the worst users.  But it’s okay that we in IT continuously disrupt our user community with “infrastructure improvements” and “the latest and greatest software”, even when no one is asking for it, or no one is complaining about what is already in place.  We stand by our claim that THIS IS PROGRESS.

So why do we tend to cry when the spotlight is turned back onto ourselves?  Why do we try to fend off any attempt to examine existing process, even when it is for improving self-process for the better of the whole?  And our defensive chants of that have become so systematic, just like the boring sports athlete interview: “We play a full 60-minutes”, “We take one play at time”, “We don’t look any further than our next opponent”.  IT says, “We’re too busy”, “We don’t have enough staff”, “We’re different, so it needs to be done this way, or bad things that you could not possibly understand will happen”

Shameful, but often true.

One thought on “Caché Developers and Change Control

Comments are closed.