I can't believe that it has been nearly a month since my last post.
If you've been following me on buzz you'll know that I've been doing a joke a day. I started Feb 18, so now I've gone for 2 months, with a new joke each day, without once having to look one up. I'm surprised by this. Today's joke had a musical component that needed to be heard, so it is on youtube.
Stories like this make me sad. Nothing will compensate that man's loss or replace what was stolen, but the county deserves to lose that case.
Moving on, on Hacker News a while ago I tried to explain why quantum mechanics and the general theory of relativity conflict with each other. That may be of interest. I sent that link to a well-respected researcher, John Baez, and he thought the explanation was reasonable in so far as it went. Then he recommended the far more detailed explanation at http://arxiv.org/abs/gr-qc/9310031. (Look for the Download box at the upper right.) Having read it, that explanation is far longer, but gives much more depth than mine. Be warned that my reaction was, "Now that I've read it I feel less educated on the subject than before. Not because I didn't learn anything, but because I was reminded of how much physics I never learned... I was reassured to learn that C.J. Isham has that effect on many physicists as well.
What else did I mean to talk about and didn't? Oh right. I read The Back of the Napkin: Solving Problems and Selling Ideas with Pictures. On the whole it was a very good book, but it had a spectacularly poorly chosen example at the end. The example was a hypothetical firm selling accounting software named SAX Inc. Despite being the market leader its revenue was not growing as the market expanded, and some low end open source platforms were growing rapidly. Economic projections showed that it was poised to lose huge amounts of market share to these new competitors in the near future, and the question is what it should do.
Now if you're ever in a situation that looks vaguely like this, I've got a recommend. Reach for The Innovator's Solution, read it, understand it, get every executive in your company to understand it, and follow its advice as best you can. Because you're on the wrong side of an disruptive innovation, so you want to see what has been tried in that situation, what worked and not (mostly not), and need to understand the organizational reasons for that.
If you don't know what a disruptive innovation is, that is the scenario where an established market with established companies has competition from a new kind of product which is not good enough for the market, but which will become so with projected technological improvement. In this situation what happens is that over time technology improves, the ecosystem of companies that grew up with the (initially) crappy technology take over, and the established companies see their market implode. For a fuller explanation and lots of examples, read The Innovator's Dilemma. That gives the theory, and then the follow-up, The Innovator's Solution, gives a lot more detail on why companies repeatedly make the same bad choices.
Suffice it to say that The Back of the Napkin comes up with a solution, and it is a solution that I guarantee will fail. And as part of the reasoning they manage to come up with a software development project that is much more ambitious than anything that the company has tried before, and gave some suspiciously precise estimates of how much time and money it will take. If you don't know what is wrong with that, go read Software Estimation by Steve McConnell. (If you are responsible for anything to do with scheduling software development, I highly recommend that book on general principle. Steve McConnell's books on software development range from good to classic, and that is on the higher end of the scale.)
None of this is to say that The Back of the Napkin is a bad book. It is not. Indeed if it were then I'd be less pained by the bad example. But it had the opportunity to explain a bunch of very important things to an audience that normally doesn't hear those things, and didn't. Not only did it fail to explain them, it proceeded to actively misinform.
On a happier note I've also read Accelerando by Charles Strauss. I have to say, without hesitation, that it is the strangest book I've read. Let me give an example. Near the beginning some lobsters that got uploaded managed to escape over the Internet, take over a computer network, and turned themselves sentient. They ask the protagonist for help in getting away from human civilization because they don't want to be near us when technology goes critical. He succeeds. Then the book gets weird.
Don't get me wrong. It is a very good book, and I enjoyed it very much. But the whole point of the book is that technology is accelerating, and the result will be continuous future shock. The first few iterations you are lead into trying to understand how the future rapidly got more bizarre. In later iterations you're just following the personal story of humans with unbelievable technology trying to survive in a world where they are obsolete and unequipped to understand the universe around them.
Showing posts with label software development. Show all posts
Showing posts with label software development. Show all posts
Sunday, April 18, 2010
Friday, February 5, 2010
Developing on HEAD scales to Google
I have a simple rule of thumb for what I am and am not allowed to say about Google. If I can find it said in some official-looking place from Google, then I think I'm allowed to say it. Otherwise not.
I was therefore very glad to run across this talk by Guido Van Rossum describing his code review tool named Mondrian, put on youtube by Google Tech Talks, and judging from the knowledge level of the audience, delivered to a non-Google audience. Confirming that this was not, in fact, accidentally open, I found an O'Reilly article confirming that it was public.
I therefore feel comfortable talking about anything and everything said in that video. So you can view this as my spin on the material in that video, and if you want the full version without the editorializing, feel free to take an hour and watch the video for the original source.
These days everyone who is competent has agreed that source control is a Good Thing to use. However opinions on how to use it vary widely. One argument in particular that I've seen in multiple organizations is on the value of using multiple branches versus having everyone develop in one branch, which is often called something like MAIN or HEAD. The issues involved are somewhat complex, and not obvious. So let me give a quick overview.
The primary argument for branching is that people can work on different features on different schedules without getting in each other's way. The primary argument for all developing on the same branch is that you discover conflicts immediately when they are easiest to resolve, rather than months later when people have moved on to different things and have lost context for the pieces of code that conflicts. The primary argument against branching is the pain of merging later. (Granted, distributed version control systems like git have done a lot to reduce this pain. But they have not eliminated it.) The primary argument against developing on HEAD is that it requires a constant level of diligence on the part of developers. When any developer can break all developers, you need to be careful in what you check in.
I've just kept this down to the primary arguments because the secondary back and forth arguments get long, involved, and heated. Also what I've described as two ways of working is really two families of working. There are a lot of ways to organizing multiple branches. And there are quite a few useful uses for branches in a software project even if everyone is developing on the same branch. None of this is made easier by the fact that (as with many religious wars) the people involved have imprinted on one way of doing things. That makes them hyper aware of the problems in other approaches, and they don't even notice potential pain points in their own.
Until I came to Google my personal position was that everyone working on HEAD was the best approach as long as your team was small enough that you could make it work. And I vaguely accepted that the pain of branching was a necessary evil on large software projects. Even when the pain reached the point of craziness, I was mostly willing to accept unsupported claims that it is necessary.
Then I came to Google. There are a lot of groups at Google, and they are free to do their own thing. Many do. However most groups develop out of one giant code base on HEAD. And it works as a development model. Google has made it scale.
Guido's talk describes many elements of what makes it scale. The first piece is having good developers. The second piece is an enforced policy of having every single patch go through a code review process before you check anything in. The third piece is a lot of proprietary infrastructure that Google has built up to make things work. And beyond that you have people paying attention to best practices such as consistent style, good unit testing, so on and so forth. (All of which are reinforced in the code review process.)
My opinion after seeing Google has changed. I freely admit that there are real process and tool challenges to making it possible for large teams to develop everything on HEAD and have it scale smoothly. However it is possible. I've seen it work. And, speaking personally, this is my preferred way to work.
Different organizations are different, have different capabilities, different needs, and different goals. Sight unseen I'm not going to tell anyone else that their organization should try to work like Google does. I simply don't have the facts about your situation. But if, like me not that long ago, you've accepted the claim that large development teams have no choice, you now know better.
Now admittedly most people can't go and see this first hand for themselves at Google. But if you want to watch a large successful project developing code to a high standard in this way, I recommend watching clang. And if you want to know more about what kinds of tool support you need to make this work, go listen to Guido. He's smarter than I am, has been doing it longer, and actually built some of the basic tool support for it.
I was therefore very glad to run across this talk by Guido Van Rossum describing his code review tool named Mondrian, put on youtube by Google Tech Talks, and judging from the knowledge level of the audience, delivered to a non-Google audience. Confirming that this was not, in fact, accidentally open, I found an O'Reilly article confirming that it was public.
I therefore feel comfortable talking about anything and everything said in that video. So you can view this as my spin on the material in that video, and if you want the full version without the editorializing, feel free to take an hour and watch the video for the original source.
These days everyone who is competent has agreed that source control is a Good Thing to use. However opinions on how to use it vary widely. One argument in particular that I've seen in multiple organizations is on the value of using multiple branches versus having everyone develop in one branch, which is often called something like MAIN or HEAD. The issues involved are somewhat complex, and not obvious. So let me give a quick overview.
The primary argument for branching is that people can work on different features on different schedules without getting in each other's way. The primary argument for all developing on the same branch is that you discover conflicts immediately when they are easiest to resolve, rather than months later when people have moved on to different things and have lost context for the pieces of code that conflicts. The primary argument against branching is the pain of merging later. (Granted, distributed version control systems like git have done a lot to reduce this pain. But they have not eliminated it.) The primary argument against developing on HEAD is that it requires a constant level of diligence on the part of developers. When any developer can break all developers, you need to be careful in what you check in.
I've just kept this down to the primary arguments because the secondary back and forth arguments get long, involved, and heated. Also what I've described as two ways of working is really two families of working. There are a lot of ways to organizing multiple branches. And there are quite a few useful uses for branches in a software project even if everyone is developing on the same branch. None of this is made easier by the fact that (as with many religious wars) the people involved have imprinted on one way of doing things. That makes them hyper aware of the problems in other approaches, and they don't even notice potential pain points in their own.
Until I came to Google my personal position was that everyone working on HEAD was the best approach as long as your team was small enough that you could make it work. And I vaguely accepted that the pain of branching was a necessary evil on large software projects. Even when the pain reached the point of craziness, I was mostly willing to accept unsupported claims that it is necessary.
Then I came to Google. There are a lot of groups at Google, and they are free to do their own thing. Many do. However most groups develop out of one giant code base on HEAD. And it works as a development model. Google has made it scale.
Guido's talk describes many elements of what makes it scale. The first piece is having good developers. The second piece is an enforced policy of having every single patch go through a code review process before you check anything in. The third piece is a lot of proprietary infrastructure that Google has built up to make things work. And beyond that you have people paying attention to best practices such as consistent style, good unit testing, so on and so forth. (All of which are reinforced in the code review process.)
My opinion after seeing Google has changed. I freely admit that there are real process and tool challenges to making it possible for large teams to develop everything on HEAD and have it scale smoothly. However it is possible. I've seen it work. And, speaking personally, this is my preferred way to work.
Different organizations are different, have different capabilities, different needs, and different goals. Sight unseen I'm not going to tell anyone else that their organization should try to work like Google does. I simply don't have the facts about your situation. But if, like me not that long ago, you've accepted the claim that large development teams have no choice, you now know better.
Now admittedly most people can't go and see this first hand for themselves at Google. But if you want to watch a large successful project developing code to a high standard in this way, I recommend watching clang. And if you want to know more about what kinds of tool support you need to make this work, go listen to Guido. He's smarter than I am, has been doing it longer, and actually built some of the basic tool support for it.
Sunday, January 31, 2010
Things I've learned at Google so far
Well I've been an employee at Google for about a month. So this seems like as good a place to reflect for a moment.
The first thing that I've learned is that internally Google is incredibly open, but externally there is a lot we can't say. I understand and support a lot of the reasons why it is so, but it can be frustrating. There is a lot of really cool technology at Google that people never hear about. The statistics of what Google deals with are astounding. The technology we use to deal with it is amazing. The way we scale is unbelievable. (I really wish I could go back and have a few discussions on software development methodology raising points about what has proven to scale at Google...) One random fact that I know I can say is that computations happen in our data centers with about half the power drawn for what is industry standard. I'm not allowed to say how we do it, but it is a rather amazing testimony to what smart people can accomplish when we put our minds to it.
Moving on, what about Google's culture? I would describe Google's culture as "creative chaos". There was some confusion about where I was supposed to be when I started. This resulted in the following phone call, "Hello?", "Hello Ben, this is Conner (that's my new manager), where are you?" "Mountain View." "Why are you there?" "Because this is where the recruiter said to go." "Good answer! Nice of them to tell me. Enjoy your week!" This caused me to ask an experienced Googler, "Is it always this chaotic?" The response I got was, "Yes! Isn't it wonderful?" That response sums up a lot about Google's culture. If you're unable to enjoy that kind of environment, then Google isn't the place for you.
Seriously, the corporate culture is based on hiring really smart people, giving them responsibilities, letting them know what problems the company thinks it should focus on, then letting them figure out how to tackle it. What management hierarchy there is is very flat. And people pay little attention to it unless there is a problem. You are expected to be a self-directed person, who solves problems by reaching out to whomever you need to and talking directly. Usually by email. The result is an organization which is in a constant state of flux as things are changing around you, usually for the better. With a permanent level of chaos and very large volumes of email. It is as if an entire company intuitively understood that defect rates are tied to distance on the corporate org-chart, and tried to solve it by eliminating all barriers to people communicating directly with whoever they need to communicate with. (Incidentally the point about defect rates and org charts is actually true, see Facts and Fallacies of Software Engineering for a citation.)
Speaking of email, working at Google you learn really fast how gmail is meant to be used. If you want to deal with a lot of email in gmail, here is what you need to do. Go into settings and turn keyboard shortcuts on. The ones you'll use a lot are j/k to move through email threads, n to skip to the next message, and the space bar to page through text. And m to hide any active thread that you're not interested in (direct emails to you will still show up). There are other shortcuts, but this is enough to let you skim through a lot of email fairly quickly without touching the mouse too much. Next go into labels and choose to show all labels. Your labels are basically what you'd call folders in another email client. (Unfortunately they are not hierarchical, but they do work.) Next as you get email, you need to be aggressive about deciding what you need to see, versus what is context specific. Anything that is context specific you should add a filter for, that adds a label, and skips the inbox. Nothing is lost, you can get to the emails through the list of labels on the left-hand side of your screen in gmail. But now various kinds of automated emails, lower priority mailing lists, and so on won't distract you from your main email until you go looking for them.
When you combine all of these options with gmail's auto-threading features, it is amazing how much more efficiently you can handle email. In fact this is exactly the problem that gmail was invented to handle. Because this was the problem that Paul Buchheit was trying to solve for himself when he started gmail. It is worth pointing out that Paul Buchheit was a software engineer at Google. He didn't need permission to write something like gmail. Corporate culture says that if you need something like that, you just go ahead and do it. In fact this is enshrined as an official corporate policy - engineers get 20% of their time to do with pretty much as they please, and are judged in part on how they use that time. I found a speech claiming that over half of Google's applications started as a 20% project. (I'm surprised that the figure is so low.) To get a sense of how much stuff people just do, visit Google Labs. No corporate decision. No central planning. People just do things like start putting up solar panels in the parking lot, and the next thing you know Google has one of the largest solar panel installations in the world and has decided to go carbon neutral. And the attitude that this is how you should operate is enshrined as official corporate policy!
You've got to love corporate policies like that. Speaking of nice corporate policies, Google has quite a few surprising ones. For instance they have benefits like heavily subsidized massage on site (I've still got to take my free hour massage for joining), free gym membership, and the like. Or take their attitude on dogs. Policy says that if your immediate co-workers don't object, you can bring your dog to work. Cats are different, however. Nothing against cats, but Google is a dog place and cats wouldn't be comfortable. (Yes, there are lots of dogs around the offices, and I've even seen people randomly wander over to find out if they can borrow someone else's dog for a while.) Hmmm. Sick day policy. Don't show up when you're sick and tell people why you're not showing up. Note what's missing. There is no limit to how much sick time you get if you need it. Oh, and food. Official Google policy is that at all times there shall be good, free food within 150 feet of every Googler. OK, admittedly the food quality does vary. That in Mountain View is better than anywhere else (the larger clientele base lets them have a much more varied selection). But you quickly learn why it is common for new Googlers to put on 15-20 pounds in their first year. (I'm trying to avoid that. We'll see if I succeed...)
But, you say, isn't this crazy? Doesn't it cost a fortune? The answer is that of course it does. But it provides value. People bond over food. Even if you're not bonding, having food close by makes short meals easier. And the temptation to continue working until dinnertime is very real. (Particularly if, as with me, you'd like to wait until rush hour is over before going home.) Obviously no normal CFO would crunch numbers and see things that way. But Google stands behind that decision, and the people who work there treasure the company for it.
Speaking of the people who work there, Google has amazing people. It is often said that engineers find working at Google a humbling experience. This is absolutely true. It took me less than a day to realize that the guy sitting next to me is clearly much smarter than I am, and he's nowhere near the top of the range of talent at Google. In fact, as best as I can tell, I'm pretty much average, though I'm trying hard to hold out a ray of hope that I'm slightly better than average.
Let me put that in context. The closest thing that I have to an estimate for my IQ is scoring 2340 on the GRE exam in 1991. Based on conversions that I've seen, that puts me at about the top 0.01% in IQ. Now I was really "on" that day, happen to believe that there are problems with the measurement of intelligence by an IQ test (a subject which I may devote a future blog post to), but without false modesty I wouldn't be surprised to find that I'm as high as being in the top 0.1% in general intelligence (however that could be measured). Which in most organizations means that I get thought of as being very smart.
However software development is a profession that selects for intelligence. By and large only good software developers bother applying to Google. And Google rejects the vast majority of their applicants. Granted the filtering process is far from perfect, but by the time you get through that many filters, someone like me is just average.
This leads to another point of interest. How astoundingly complex the company is. I believe that organizations naturally evolve until they are as complex as the people in them can handle. Well Google is tackling really big, complex problems, and is full of people who can handle a lot of complexity. The result? I've been told that I should expect that after 2 months I'll only be marginally useful. My initial learning curve should start to smooth out after about 6 months. And every year I should expect half of what I've learned to become obsolete. (Remember what I said about Google having a certain level of permanent chaos? If you're like me, it is exhilarating. But sometimes the line between exhilarating and terrifying can be hard to find...)
Oh, and what else did I learn? That we're hiring more people this year. :-)
The first thing that I've learned is that internally Google is incredibly open, but externally there is a lot we can't say. I understand and support a lot of the reasons why it is so, but it can be frustrating. There is a lot of really cool technology at Google that people never hear about. The statistics of what Google deals with are astounding. The technology we use to deal with it is amazing. The way we scale is unbelievable. (I really wish I could go back and have a few discussions on software development methodology raising points about what has proven to scale at Google...) One random fact that I know I can say is that computations happen in our data centers with about half the power drawn for what is industry standard. I'm not allowed to say how we do it, but it is a rather amazing testimony to what smart people can accomplish when we put our minds to it.
Moving on, what about Google's culture? I would describe Google's culture as "creative chaos". There was some confusion about where I was supposed to be when I started. This resulted in the following phone call, "Hello?", "Hello Ben, this is Conner (that's my new manager), where are you?" "Mountain View." "Why are you there?" "Because this is where the recruiter said to go." "Good answer! Nice of them to tell me. Enjoy your week!" This caused me to ask an experienced Googler, "Is it always this chaotic?" The response I got was, "Yes! Isn't it wonderful?" That response sums up a lot about Google's culture. If you're unable to enjoy that kind of environment, then Google isn't the place for you.
Seriously, the corporate culture is based on hiring really smart people, giving them responsibilities, letting them know what problems the company thinks it should focus on, then letting them figure out how to tackle it. What management hierarchy there is is very flat. And people pay little attention to it unless there is a problem. You are expected to be a self-directed person, who solves problems by reaching out to whomever you need to and talking directly. Usually by email. The result is an organization which is in a constant state of flux as things are changing around you, usually for the better. With a permanent level of chaos and very large volumes of email. It is as if an entire company intuitively understood that defect rates are tied to distance on the corporate org-chart, and tried to solve it by eliminating all barriers to people communicating directly with whoever they need to communicate with. (Incidentally the point about defect rates and org charts is actually true, see Facts and Fallacies of Software Engineering for a citation.)
Speaking of email, working at Google you learn really fast how gmail is meant to be used. If you want to deal with a lot of email in gmail, here is what you need to do. Go into settings and turn keyboard shortcuts on. The ones you'll use a lot are j/k to move through email threads, n to skip to the next message, and the space bar to page through text. And m to hide any active thread that you're not interested in (direct emails to you will still show up). There are other shortcuts, but this is enough to let you skim through a lot of email fairly quickly without touching the mouse too much. Next go into labels and choose to show all labels. Your labels are basically what you'd call folders in another email client. (Unfortunately they are not hierarchical, but they do work.) Next as you get email, you need to be aggressive about deciding what you need to see, versus what is context specific. Anything that is context specific you should add a filter for, that adds a label, and skips the inbox. Nothing is lost, you can get to the emails through the list of labels on the left-hand side of your screen in gmail. But now various kinds of automated emails, lower priority mailing lists, and so on won't distract you from your main email until you go looking for them.
When you combine all of these options with gmail's auto-threading features, it is amazing how much more efficiently you can handle email. In fact this is exactly the problem that gmail was invented to handle. Because this was the problem that Paul Buchheit was trying to solve for himself when he started gmail. It is worth pointing out that Paul Buchheit was a software engineer at Google. He didn't need permission to write something like gmail. Corporate culture says that if you need something like that, you just go ahead and do it. In fact this is enshrined as an official corporate policy - engineers get 20% of their time to do with pretty much as they please, and are judged in part on how they use that time. I found a speech claiming that over half of Google's applications started as a 20% project. (I'm surprised that the figure is so low.) To get a sense of how much stuff people just do, visit Google Labs. No corporate decision. No central planning. People just do things like start putting up solar panels in the parking lot, and the next thing you know Google has one of the largest solar panel installations in the world and has decided to go carbon neutral. And the attitude that this is how you should operate is enshrined as official corporate policy!
You've got to love corporate policies like that. Speaking of nice corporate policies, Google has quite a few surprising ones. For instance they have benefits like heavily subsidized massage on site (I've still got to take my free hour massage for joining), free gym membership, and the like. Or take their attitude on dogs. Policy says that if your immediate co-workers don't object, you can bring your dog to work. Cats are different, however. Nothing against cats, but Google is a dog place and cats wouldn't be comfortable. (Yes, there are lots of dogs around the offices, and I've even seen people randomly wander over to find out if they can borrow someone else's dog for a while.) Hmmm. Sick day policy. Don't show up when you're sick and tell people why you're not showing up. Note what's missing. There is no limit to how much sick time you get if you need it. Oh, and food. Official Google policy is that at all times there shall be good, free food within 150 feet of every Googler. OK, admittedly the food quality does vary. That in Mountain View is better than anywhere else (the larger clientele base lets them have a much more varied selection). But you quickly learn why it is common for new Googlers to put on 15-20 pounds in their first year. (I'm trying to avoid that. We'll see if I succeed...)
But, you say, isn't this crazy? Doesn't it cost a fortune? The answer is that of course it does. But it provides value. People bond over food. Even if you're not bonding, having food close by makes short meals easier. And the temptation to continue working until dinnertime is very real. (Particularly if, as with me, you'd like to wait until rush hour is over before going home.) Obviously no normal CFO would crunch numbers and see things that way. But Google stands behind that decision, and the people who work there treasure the company for it.
Speaking of the people who work there, Google has amazing people. It is often said that engineers find working at Google a humbling experience. This is absolutely true. It took me less than a day to realize that the guy sitting next to me is clearly much smarter than I am, and he's nowhere near the top of the range of talent at Google. In fact, as best as I can tell, I'm pretty much average, though I'm trying hard to hold out a ray of hope that I'm slightly better than average.
Let me put that in context. The closest thing that I have to an estimate for my IQ is scoring 2340 on the GRE exam in 1991. Based on conversions that I've seen, that puts me at about the top 0.01% in IQ. Now I was really "on" that day, happen to believe that there are problems with the measurement of intelligence by an IQ test (a subject which I may devote a future blog post to), but without false modesty I wouldn't be surprised to find that I'm as high as being in the top 0.1% in general intelligence (however that could be measured). Which in most organizations means that I get thought of as being very smart.
However software development is a profession that selects for intelligence. By and large only good software developers bother applying to Google. And Google rejects the vast majority of their applicants. Granted the filtering process is far from perfect, but by the time you get through that many filters, someone like me is just average.
This leads to another point of interest. How astoundingly complex the company is. I believe that organizations naturally evolve until they are as complex as the people in them can handle. Well Google is tackling really big, complex problems, and is full of people who can handle a lot of complexity. The result? I've been told that I should expect that after 2 months I'll only be marginally useful. My initial learning curve should start to smooth out after about 6 months. And every year I should expect half of what I've learned to become obsolete. (Remember what I said about Google having a certain level of permanent chaos? If you're like me, it is exhilarating. But sometimes the line between exhilarating and terrifying can be hard to find...)
Oh, and what else did I learn? That we're hiring more people this year. :-)
Monday, November 9, 2009
A proposal on patent reform
In theory the purpose of patents is to increase the public domain by providing temporary incentives to spur innovation. In that spirit patents are supposed to be limited to things that would not be obvious to one versed in the art.
However patents have lead to many problems.
First is the question of which ideas are obvious, and which are not. This is not a simple question at all. The patent office has resolved this question by being fairly lenient about which patents are granted, thinking that the issue can always be resolved in litigation. However people are reluctant to litigate, so invalid patents are acquired in great number. The theory goes that any individual one is unlikely to stand up, but if I threaten you with a dozen patents at once, my odds of having at least one stand up are pretty good. This set of incentives leads to a lot of overly broad patents that are pretty obvious, which causes uncertainty for others.
Second you have the issue of how long patents are good for. When it comes to drugs, a multi-year search and FDA approval can only be recouped from a long patent. When it comes to software a 17 year patent warps the market for many generations of technologies. Thus there can be no simple balance between encouraging innovation and unnecessarily hobbling further advancement. (Many people in software think patents shouldn't apply there. Certainly patents do a lot of damage in the software world, but as Paul Graham points out, if you're against software patents then you're against patents in general.)
Third we have the challenge that areas that change rapidly have a lot of opportunity for being the first to face a particular problem. If you're first to face a problem then it is easy to get a patent. Given that the patent office is lenient on granting them, you're likely to get it. So we tend to see very intense patent activity in areas that change rapidly. However those are exactly the areas where patents do the most to inhibit further progress!
My proposal addresses these issues.
I propose that no patent shall be accepted unless the technologies that make the discovery feasible and commercially viable have been broadly available for at least a decade.
If the problem has been solvable and a solution desirable for at least a decade but nobody has done it, that is evidence that the solution is not obvious to one versed in the art. Conversely if the solution quickly occurs to someone fairly shortly after the problem has become both solvable and commercially interesting, that is evidence that it wasn't really that hard. Furthermore it is exactly advancements in fields that are rapidly changing where patent protection does the most economic harm.
Why did I choose a decade? I chose it as a round number that is fairly close to half the length that a patent lasts. My theory being that if nobody has discovered it in that length of time, then the odds are reasonably high that nobody would have discovered it in the time the patent restricted the use of the invention, and therefore we have evidence that the net economic result caused by granting the patent is positive.
Of course, like any attempt at compromise on a controversial issue, I am sure it will satisfy nobody. But I don't think it is that bad either.
However patents have lead to many problems.
First is the question of which ideas are obvious, and which are not. This is not a simple question at all. The patent office has resolved this question by being fairly lenient about which patents are granted, thinking that the issue can always be resolved in litigation. However people are reluctant to litigate, so invalid patents are acquired in great number. The theory goes that any individual one is unlikely to stand up, but if I threaten you with a dozen patents at once, my odds of having at least one stand up are pretty good. This set of incentives leads to a lot of overly broad patents that are pretty obvious, which causes uncertainty for others.
Second you have the issue of how long patents are good for. When it comes to drugs, a multi-year search and FDA approval can only be recouped from a long patent. When it comes to software a 17 year patent warps the market for many generations of technologies. Thus there can be no simple balance between encouraging innovation and unnecessarily hobbling further advancement. (Many people in software think patents shouldn't apply there. Certainly patents do a lot of damage in the software world, but as Paul Graham points out, if you're against software patents then you're against patents in general.)
Third we have the challenge that areas that change rapidly have a lot of opportunity for being the first to face a particular problem. If you're first to face a problem then it is easy to get a patent. Given that the patent office is lenient on granting them, you're likely to get it. So we tend to see very intense patent activity in areas that change rapidly. However those are exactly the areas where patents do the most to inhibit further progress!
My proposal addresses these issues.
I propose that no patent shall be accepted unless the technologies that make the discovery feasible and commercially viable have been broadly available for at least a decade.
If the problem has been solvable and a solution desirable for at least a decade but nobody has done it, that is evidence that the solution is not obvious to one versed in the art. Conversely if the solution quickly occurs to someone fairly shortly after the problem has become both solvable and commercially interesting, that is evidence that it wasn't really that hard. Furthermore it is exactly advancements in fields that are rapidly changing where patent protection does the most economic harm.
Why did I choose a decade? I chose it as a round number that is fairly close to half the length that a patent lasts. My theory being that if nobody has discovered it in that length of time, then the odds are reasonably high that nobody would have discovered it in the time the patent restricted the use of the invention, and therefore we have evidence that the net economic result caused by granting the patent is positive.
Of course, like any attempt at compromise on a controversial issue, I am sure it will satisfy nobody. But I don't think it is that bad either.
Tuesday, October 20, 2009
What granularity for MVC?
Web developers often try to use an MVC architecture. This means that the business end of a website is divided into models, views, and controllers. Models encapsulate your data, and this is where you are supposed to put business rules like, "You can't take more money out of your account than you have in it." Views are about how to present the data. Ideally the model shouldn't know or care about whether the user is going to look at an edit page or a display page, or look at a page in English or Chinese, those things are presentation logic which belongs in the view. And finally the controller is responsible for the necessary glue between them, for instance to figure out what model and view to use for a given URL for a given person.
There is widespread agreement that this is a good thing in the principle, but things get messy when you go to the details. Why? Well in part because people often break the rules they agree to. Business logic creeps into views, models format text that is just passed through the view, and the controller gets into everyone else's business. After a while you're doing MVC in name only. But also there are honest disagreements about a myriad of issues including where exactly to draw the lines between classes, whether to add in helper classes and if so which ones, how complex views should be, and so on.
I'd like to draw people's attention to a design question that few in the web world think about, namely how granularly we should do MVC.
Traditionally in web development people assume that the right level of granularity is the web page. You get a URL, that goes into a controller, sometimes there can be a cascade of routing decisions, and eventually you get to the code that fetches an appropriate model and view, then puts them together to get a web page. Sounds pretty straightforward and flexible.
But what if we have a complex web page? What happens if we add independent sections on a web page that have little to do with each other but need different dynamic information? Alternately how do we handle displaying many related objects for editing? In my experience the natural tendency is for the controller to become a mess. And the view has to be synchronized with the controller, resulting in fragile dependencies.
For an alternative, let's look back at the history of MVC. Originally MVC arose in the Smalltalk world. And the appropriate granularity for MVC wasn't a whole page, it was a component. You'd write Smalltalk applications and every little dropdown could have its own model, view and controller. Which could all talk to each other, and permitted good decomposition of very complex behavior. This idea has continued in the world of GUI programming. For instance an object-oriented Flash program is likely to be developed on a similar paradigm.
What happens if we take this idea back to serving a web page? Well obviously you need a lot more models, views and controllers. However it isn't as bad as it seems because they also become much simpler. Plus it becomes easier to reuse components on multiple pages through a site. The whole page can still become complex, but MVC is a good way to manage it. After all our design is now closer to the original versions where MVC first proved itself useful.
And as a final thought, ever more complex and dynamic client side interfaces make web pages into something more like any other GUI. Which means that at some point there are likely to be real advantages to using component based MVC within the client interface. But this idea is more natural to try when there is no impedance mismatch between your server-side design and your client-side design. Which suggests that you should start with component-level MVC on the server side, and then "lift" components to dynamic AJAXified components one by one as appropriate. Therefore starting with a component-level MVC gives you more flexibility to later evolve a complex, dynamic client-side interface.
Obviously I wouldn't recommend that every developer rewrite their current websites based on this idea. However I think that the benefits are interesting enough that I'd recommend that developers play with this idea, so that they can make an informed decision about whether to try this approach on future websites.
There is widespread agreement that this is a good thing in the principle, but things get messy when you go to the details. Why? Well in part because people often break the rules they agree to. Business logic creeps into views, models format text that is just passed through the view, and the controller gets into everyone else's business. After a while you're doing MVC in name only. But also there are honest disagreements about a myriad of issues including where exactly to draw the lines between classes, whether to add in helper classes and if so which ones, how complex views should be, and so on.
I'd like to draw people's attention to a design question that few in the web world think about, namely how granularly we should do MVC.
Traditionally in web development people assume that the right level of granularity is the web page. You get a URL, that goes into a controller, sometimes there can be a cascade of routing decisions, and eventually you get to the code that fetches an appropriate model and view, then puts them together to get a web page. Sounds pretty straightforward and flexible.
But what if we have a complex web page? What happens if we add independent sections on a web page that have little to do with each other but need different dynamic information? Alternately how do we handle displaying many related objects for editing? In my experience the natural tendency is for the controller to become a mess. And the view has to be synchronized with the controller, resulting in fragile dependencies.
For an alternative, let's look back at the history of MVC. Originally MVC arose in the Smalltalk world. And the appropriate granularity for MVC wasn't a whole page, it was a component. You'd write Smalltalk applications and every little dropdown could have its own model, view and controller. Which could all talk to each other, and permitted good decomposition of very complex behavior. This idea has continued in the world of GUI programming. For instance an object-oriented Flash program is likely to be developed on a similar paradigm.
What happens if we take this idea back to serving a web page? Well obviously you need a lot more models, views and controllers. However it isn't as bad as it seems because they also become much simpler. Plus it becomes easier to reuse components on multiple pages through a site. The whole page can still become complex, but MVC is a good way to manage it. After all our design is now closer to the original versions where MVC first proved itself useful.
And as a final thought, ever more complex and dynamic client side interfaces make web pages into something more like any other GUI. Which means that at some point there are likely to be real advantages to using component based MVC within the client interface. But this idea is more natural to try when there is no impedance mismatch between your server-side design and your client-side design. Which suggests that you should start with component-level MVC on the server side, and then "lift" components to dynamic AJAXified components one by one as appropriate. Therefore starting with a component-level MVC gives you more flexibility to later evolve a complex, dynamic client-side interface.
Obviously I wouldn't recommend that every developer rewrite their current websites based on this idea. However I think that the benefits are interesting enough that I'd recommend that developers play with this idea, so that they can make an informed decision about whether to try this approach on future websites.
Subscribe to:
Posts (Atom)