The Evil Banana
- Place five gorillas in a cage.
- Suspend a banana in the cage above a ladder.
- When any gorilla attempts to use the ladder, wet all five gorillas with a fire hose.
- When gorillas no longer attempt to use the ladder, replace one gorilla.
- Note that when the new gorilla attempts to use the ladder, the other gorillas will beat him up. The fire hose is no longer needed.
- Repeat step 4 until all original gorillas have been replaced.
- Note that at this point, no gorillas use the ladder and none of them knows why.
Many people will prevent you from getting a perfectly good banana even though nobody knows why, or maybe they have a very good reason, although nobody knows what that reason is.
Today I ran into that on a mild level. I get paid on the 5th of every month, once a month. Why? I found out because when this company started, they got their credit card receipts submitted on the 1st, usually got credited on the 3rd, and so could do payroll by the 5th. That's no longer necessary, but since their payroll has been geared that way, it would be a pain to change it back.
At AOL, a majority of our sub-processes ran on a command language called TCL (or "tickle"). It started because we bought a company in 1993 that did things that way (some of the code/scripts had 1993 copyright dates in the comments). Hardly anyone uses TCL anymore, but since AOL had built tools on top of tools, that managed other subprocesses in TCL, to change to a much more flexible and faster language like Python or Perl would have required ripping up everything by the roots, and you simply can't do that on a system at that scale. It would be like asking the city of New York to replace all of its sewers, water pipes, and electrical work... all at once... just to replace the material they are made from to a more resilient, superior material. No way. But the sad thing is, just like New York, you have aging systems that can't handle the load, and all your workers are scrambling around doing patch jobs, while managers shift and change projects so frequently, hardly anything can get done. So you just keep adding load to an upside-down pyramid, hoping the top-heavy foundation can take it.
Smaller companies like the one I work in now? Much easier. Right now, my main project is to take a whole bunch of mail/DNS caching systems, and change them from FreeBSD to Red Hat EL 3. All the software we currently use works, if not works better, on Red Hat. In addition, we're doing an entirely different load balance schema on them. Total of 10 boxes replacing 8. At AOL, we were switching from Sun/HP to Red Hat Linux, but so many cooks were in the soup, so to speak, that we ran into hundreds of problems. Developers had compiled programs that didn't run on other platforms, and many of them never actually know how the application worked or was built, since the people who did were long ago laid off or quit. On top of that, we had blanket security rules like default firewalls that did different things. It was like by the time AOL realized they needed to standardize things, it was too late; "how things are done" were all over the map, and many couldn't even remember why.