Monthly Archives: June 2009

I spend a lot of my time thinking about software develoment, keeping up with what works for other people and what works for me. I've recently read Peopleware and Code Complete, and I've formulated some thoughts about software development and technical management that I've been meaning to write about. Hopefully, this copy-and-paste will get me going with that.

I found this printed out on Hacker News, and I think there is a lot of truth in it about the perceptions of software and its development.

The problem with IT, generally, is lack of respect. Because it's something that looks easy and looks like something anyone could do, people don't respect experience and talent.

For some reason this just doesn't work in other fields. While everyone feels they can put a sticking plaster on a cut, they go to a doctor for more serious stuff. Any fool can build a garden wall, but if they want a tower block one hires an architect and a construction company and lets them do decisions.

Whereas, it's quite common in IT for micromanagers to overrule the professionals.

It doesn't happen in building. Managers simply aren't allowed to say things like “Oh, I don't think we should RSJs in those load bearing walls”. If the architect says they're needed, then they go in. We also don't hire doctors by picking the tools they'll use. When you want a doctor to take out an appendix, you want a qualified general surgeon. You don't ask them whether they prefer a #3 or a #4 handle on their scalpels.

And yet it's pretty common to see projects which have picked a technology first (“Oh we're going to use J2EE for this”) and THEN hire the team around that (“Wanted, tech arch, 5 years exp with J2EE, C , C++, JAVA, Perforce, Apache, Perl, Python.”) and only at that point produce the spec. And then if the job isn't one the technology is suited for, that ends up being the developers fault and they're the ones who work overtime to put that square peg in the round hole.

It's scarcely surprising that a) almost nothing works properly, b) there's constant chaos and that therefore c) almost no-one finds their work rewarding in ways other than money.

I freely admit that a lot of my time, I feel like some sort of thief. I sit in projects which are doomed. They're more doomed than a plane that's lost both wings and is on fire. I see the people around the project running about setting more of it on fire as it hurtles towards the ocean and removing more control surfaces and making it worse. And I watch developers switch in and out of seats as we trail wreckage and smoke down. And it's always been like this. The project is a free-fall disaster before I join and after I leave and there's just nothing, nothing, nothing I can do to rescue it at that point in time. And yet I'm being paid to be there.

And I feel bad about that. I feel that I ought to be being paid to achieve something. But usually I'm just being paid to sit in a seat in a vehicle performing a ballistic trajectory straight into its crash site. And even if I quit then the next thing will be the same and whoever takes my place in this seat will be in the same situation.

And this is not how I wanted to work. I wanted to build things that people wanted. Not turn up at an aircrash and fill a seat in it for a while. And while it's financial fulfilling, it's not very emotionally fulfilling.

Most bridges don't fall down. Most patients don't die. Most IT projects are a failure in one way or another. It's like being a surgeon back in the days before anaesthetics or antibiotics. Most of our patients die… and that's got to be bad for morale.

And I can't help but think that this really has its roots in the fact that people think software is simple enough that they can exercise control over it at a level they ought not to be and that they also don't want to pay what it actually costs. Architects generally don't compete on price and you don't get to tell architects that their idea of how strong structural steel is is an underestimate you can feel you can ignore in this project. But people time and time and time again pick the cheapest option for software, design it like they'd design a tin shack and then act surprised when the end results turns out to be flimsy tin shack instead of a tower block.

Originally from http://discuss.joelonsoftware.com/default.asp?joel.3.762361.7, it appears.

I've recently been brushing up on my statistics by reading Principles of Statistics by M.G. Bulmer. I came across a problem and since I wrote some code to check my answer, I figured I'd post it with a short discussion about the answer.

First, the question:

In a certain survey of the work of chemical research workers, it was found, on the basis of extensive data, that on average each man required no fume cupboard for 60 per cent of his time, one cupboard for 30 per cent and two cupboards for 10 per cent; three or more were never required. If a group of four chemists worked independently  of one another, how many fume cupboards should be availabe in order to provode adequate facilities for at least 95 per cent of the time?

My line of thinking to solve this was to find every combination of the 4 chemists needing 0, 1, or 2 cupboards, the probability of each of those combinations happening, and finally summing up the probability of all the hoods needed.

For example, out of the 81 different possible combinations of cupboards required (3 * 3 * 3* 3, with the 3 coming from 0, 1, or 2 hoods needed), there is only one way where 0 hoods are needed in total and this is where all 4 chemists need no cupboards. Following this, there are 4 ways to have 1 hood total be required, with each of the chemists exclusively requiring a cupboard and the other three needing none (1, 0, 0, 0 & 0, 1, 0, 0 & 0, 0, 1, 0 & 0, 0, 0, 1).

So having one cupboard covers the probability of needing no cupbards amongst the 4 PLUS the probability of needing 1 cupboard amongst the 4.

I first did this problem long-handed, figuring out the probability of 0, 1, 2, and so on cupboards until I got to a sum that had a probability > 0.95. I was making a simple arithmetic error (as usual) and my answer was not matching up with what was in the back of the book, so I thought I would write a simple program to calculate the answer since I was confident in what I was trying to do, but was just having trouble multiplying and adding.

Here's the Python script I wrote (note: you need Python >= 2.6 as I use itertools.product to recreate all the combinations of cupboards needed).

The output is the summing of each of the probability of needing 0 cupboards + 1 cupboard + 2 cupboards and so on. The line with the probablity greater than 0.95 is the answer. In this the case, the answer was 4, which would cover the chemists' needs 95.85 percent of the time.

from collections import defaultdict
import itertools

probs = {
    '0': 0.6,
    '1': 0.3,
    '2': 0.1
}

trials = itertools.product('012', repeat=4)

totals = defaultdict(float)

for trial in trials:
    #how many hoods needed in this trial
    trial_sum = sum(map(lambda x: int(x), trial))

    #figure probability of exact trial occurring
    total_prob = 1.0
    for item in trial:
        total_prob *= probs[item]

    #add probability of trial to total prob for this number of hoods needed
    totals[trial_sum] += total_prob

#print out all probabilities
keys = totals.keys()
keys.sort()

running_prob = 0.0

for i in keys:
    running_prob += totals[i]
    print i, running_prob * 100