Friday, December 25, 2009

Work optimally, not long

It has been a long time since I entered anything here. But I'm in the mood. I've cooked, eaten, and even done cleanup after my turkey dinner. If you're uninterested in hearing about cooking turkey, skip the next paragraph.

This time I tried dry brining the turkey. That was surprisingly easy. A few days ago I filled a small bowl with 1 tbsp of salt per 5 pounds of turkey (that's 6.5 ml salt per kg), added some spices for taste (I did sage and a ground up bay leaf), mix, then spread that over my freshly bought turkey, put a plastic bag over it, then let it thaw in the fridge. Today I just roasted it normally. Based on past experience I checked regularly to baste the turkey, but it really wasn't necessary since the juice stayed in the turkey. Tomorrow I'll get supplies, then make turkey soup, which for me is the best part of having turkey dinner.

Enough about food. What I want to write about today is something that we all know in theory, but far too few of us act on. That is that it is better to work at an optimal, sustainable level than it is to work long hours. Certainly this is true in software, and likely in many other fields as well.

More precisely I would give the following rule. If you see a developer working more than 40-45 hours per week, ask yourself two questions:
  1. Will this effort be sustained less than 3 weeks?

  2. When it is over, will this person get corresponding downtime?

If the answer to either question is "no", then I strongly recommend looking at the schedule and figuring out what deadlines to slip immediately to get people back to reasonable schedules. Because you're going to compromise something, and choosing up front is much better than making it by burning people out and getting sub par work.

The reason for this rule is that people have a maximum sustained throughput for creative work when they are working somewhere between 35 and 45 hours/week. (For a reference on that see Rapid Development by Steve McConnell.) You can temporarily raise your hours above that and get improved productivity. But after a few weeks it catches up to you, your productivity drops, and you'll get less done despite working harder. And once you're in that state you've got a vicious cycle. Your lack of productivity pushes you to work harder which makes you less productive until something gives. Quite possibly your sanity.

The one thing that seems odd about this to most people is that people don't realize that their brains aren't working. But the reason for this is simple. When you're impaired, the first thing that gets impaired is your ability to judge how impaired you are. Thus your self-monitoring for impairment is not trustworthy. This effect is seen with many kinds of impairments, including chronic sleep deprivation, alcohol, strokes, and so on.

The army once did a study that showed this in a striking way. They took soldiers and put them on different sleep regimens. Among other things they found that most soldiers could operate indefinitely on 6 hours of sleep per night and would claim to be fine. But their wives said they weren't fine. Comparing results on ability tests before and after the new schedule, the wives were right and the soldiers were wrong. Which shows how easy it is to think you're still functioning when you really aren't.

Now that's the theory. What is the practice? The practice is that most of us don't do this. The reason is that most organizations manage by schedule, putting pressure on us to work harder. We put the effort out, we try to make the deadline, and then we fail to be aware of our declining output. Possibly we make the deadline, but quality suffers, bug counts suffer, and development+maintenance comes out to a lot more than it would have if we had focused on development alone.

What keeps us from doing the smart thing? For many of us, fear. While intellectually we know that we should be able to do more if we keep to a peak level, as a practical matter we're scared of being called slackers for not visibly putting out effort. Particularly if everyone else is working overtime. This fear is particularly reasonable since management usually has no idea how much developers do or why things take us so long. And is doubly so if our direct managers are themselves working too hard and therefore not themselves in the best shape.

Another major thing that keeps us from doing the right thing is that nobody likes to deliver bad news. Nobody wants to tell their manager that the schedule will slip. So we put off delivering the bad news to anyone, and try to keep it from happening. Of course this doesn't keep the schedule from slipping. But there is always the hope that someone else will be forced to admit that the schedule is going to slip. And failing that some people will deliver an incomplete and non-working piece, declare it done, then push the admission of failure off even farther.

Delivering bad news gets harder still because everyone else has also avoided doing it. Everyone delivers their bad news as gently as they can, and the manager inevitably wants to hear it as better than it is. Therefore upper management is usually seriously out of touch. The result is How Shit Happens, which is only funny because it is true.

As an example of how extreme this effect is, The Mythical Man-Month points out that in the rare cases of early projects, the good news starts coming immediately. If the project is late, there is no sign that the project will slip until about 2 weeks before it is due. And this is true no matter how long the project is actually delayed. And the driving force behind this is that nobody wants to deliver bad news, and nobody wants to hear it.

So we have a real organizational problem. How should you solve it?

The easiest problem is to ignore the problem and try to survive. This is suboptimal in virtually every way. However it is what most of us do.

A better option is to try to find an organization without the problem. They do exist. I know because I've been lucky enough to work for some. (Something that I suspect is a key ingredient is working in short iterations. Reality and management dreams tend not to get too far out of sync on a 1 week project.) But those can be hard to find. And in the current economy most people have a hard enough time just finding a job that they can't worry about the quality of the employer.

The final option is to try to make your current organization into one that works better. If not better for everyone, then at least better for you. This is the most difficult path, and there is no simple recipe for how to solve the necessary interpersonal issues that you'll encounter. However I can give you some useful ingredients.

The first is education. Go and get Rapid Development, read it, and learn what it says. It is one thing to confront your manager with the message that you need saner work hours. It is quite another to share research about how to get more done and say that you really want to do it. This works even better if you've identified multiple things you'd like to change at once, so it doesn't come off as your cherry-picking the item that you most wanted to try.

The second ingredient is knowing that delivering negative messages often is not as bad as we think it will be. Admittedly it doesn't always go well. However any manager who has been around the block a couple of times knows that software projects tend to fail, so it won't be as much of a surprise as you might think. Of course your manager is usually under pressure from above to meet deadlines and so is likely to not want to face that bad news. But it won't be a surprise.

The third ingredient is being conscious of the fact that delivering bad news early, no matter how bad it may be immediately, usually works out better in the long run. Yes, it is unpleasant and we all want to procrastinate on unpleasant things. But delivering bad news up front then better news later makes people happier in the long run than letting people think they'll get good news and disappointing them with bad news. (Incidentally if you're going to deliver bad news, take a tip from Machiavelli's The Prince and deliver it all at once, then deliver good news afterwards.)

The fourth ingredient is to understand the person you are talking to, and deliver your message in a way that is likely to be heard. A lot goes into this, but a big factor that lots of people miss is cognitive dissonance. Let me wrap up this topic, then I'll go on to explain it shortly.

The last ingredient is being aware of what the status quo is likely to cost you. If you allow yourself to be aware of that, you'll have lots of motivation to avoid that fate. And in this case motivation is a good thing.

Now let me return to cognitive dissonance. I won't explain it at length. I've done that elsewhere and likely will do it again, but here are the basics.

The principle of cognitive dissonance says that people will go to any length to maintain their positive self-image of themselves. This means that messages which threaten your positive self-image trigger defensive reactions. The more powerfully the message is delivered, the worse the defensive reaction is likely to be. (When you see such a reaction the last thing you want to do is follow up with facts proving your position. That just increases the conflict and therefore increases the defensive reaction.) What this means for any given person will vary from person to person. However if you remain aware of this phenomenon, you can often figure out where the hidden landmines are, and have hope of communicating what would otherwise be impossible.

For example, suppose that your manager thinks he or she is a pretty good manager. (Most of us tend to think we're pretty good at our jobs. No matter how good or bad we may be.) Then at all costs you must avoid threatening that self-image. Therefore you would need to avoid saying anything that even looks like you're delivering the message that there is room to be a better manager. This is true, even if said manager regularly points out things that made them better managers. To avoid the possibility of trouble, it is best to make the case that what you are saying flows from what that manager says about management.

This makes it really bad to say, I was reading this excellent book by Steve McConnell and really liked some of what his advice about managing software teams... Doing that is explaining to your manager how to manage, and anything you say after that which your manager doesn't already know and accept is going to be really difficult to accept.

It is better to say, I've been reading a good book, and it has taught me a lot about myself. It is making me aware that I am not working as effectively as I could... Making the message be about you is a reasonably safe way of side stepping the question of whether this is something the manager should know to tell you.

But it may be possible to do better still. For example consider, You know how you always tell us that we should constantly been trying to improve ourselves? Well I've gotten some ideas from a software classic, and here are some things that I'd like to try... If this is true, then you've put cognitive dissonance in your corner. Your manager will now find that a "Yes" is an opportunity for confirmation of his or her good self-opinion. Better yet, your manager will then be inclined to interpret the outcome as success, giving an even stronger confirmation of your manager's ability as a manager.

Now all that said, there is no guarantee that you can keep a sane work schedule when the going gets tough. However if you can do it, the outcome will be better for everyone. That doesn't mean the outcome will be good, but it is better and that is really worth it.

No comments: