Recently, I came across a company’s profile and found their Joel test. Basically, it is for a better quality of software team/company. So I decided to share it here in the simplified version. The great part about it is that it takes about 3-5 minutes compared to approx 6 years to understand SEMA.
The Joel Test
The neat thing about The Joel Test is that it’s easy to get a quick yes or no to each question. You don’t have to figure out lines-of-code-per-day or average-bugs-per-inflection-point. Give your team 1 point for each “yes” answer. The bummer about The Joel Test is that you really shouldn’t use it to make sure that your nuclear power plant software is safe.
A score of 12 is perfect, 11 is tolerable, but 10 or lower and you’ve got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help because companies like Microsoft or Google run at 12 full-time.
Of course, these are not the only factors that determine success or failure. But, all else being equal, if you get these 12 things right, you’ll have a disciplined team that can consistently deliver.
1. Do you use source control?
CVS is fine. But if you don’t have source control, you’re going to stress out trying to get programmers to work together. Programmers have no way to know what other people did. Mistakes rolling back is not easy. The other neat thing about source control systems is that the source code itself is checked out on every programmer’s hard drive — I’ve never heard of a project using source control that lost a lot of code.
2. Can you make a build in one step?
Namely, how many steps does it take to make a shipping build from the latest source snapshot? On good teams, there’s a single script you can run that does a full checkout from scratch, rebuilds every line of code, makes the EXEs, in all their various versions, languages, and #ifdef combinations, creates the installation package and creates the final media — DVDROM layout, download website, whatever.
If the process takes any more than one step, it is prone to errors. And when you get closer to shipping, you want to have a very fast cycle of fixing the “last” bug, making the final EXEs, etc. If it takes 20 steps to compile the code, run the installation builder, etc., you’re going to go crazy and you’re going to make silly mistakes.
3. Do you make daily builds?
When you’re using source control, sometimes one programmer accidentally checks in something that breaks the build. For example, they’ve added a new source file, and everything compiles fine on their machine, but they forgot to add the source file to the code repository. So they lock their machine and go home, oblivious and happy. But nobody else can work, so they have to go home too, unhappy.
Breaking the build is so bad (and so common) that it helps to make daily builds, to insure that no breakage goes unnoticed. On large teams, one good way to insure that they fix breakages right away. Everyone does as many check-ins as possible before lunch. When they come back, the build is ready. If it worked, great! Everybody checks out the latest version of the source and goes on working. If the build failed, you fix it, but everybody can keep on working with the pre-build, unbroken version of the source.
On the Excel team we had a rule that whoever broke the build, as their “punishment”, was responsible for babysitting the builds until someone else broke it. This was a good incentive not to break the build, and a good way to rotate everyone through the build process so that everyone learned how it worked.
4. Do you have a bug database?
If you are developing code, even on a team of one, without an organized database listing all known bugs in the code, you are going to ship low-quality code. Bug databases can be simple. A minimal useful bug database must include the following data for every bug:
- complete steps to reproduce the bug
- expected behaviour
- observed (buggy) behaviour
- who handle it
- whether it has been fixed or not
If the complexity of bug tracking software is the only thing stopping you from tracking your bugs, just make a simple 5 column table with these crucial fields and start using it.
5. Do you fix bugs before writing new code?
In general, the longer you wait before fixing a bug, the costlier (in time and money) it is to fix. For example, when you make a typo or syntax error that the compiler catches, fixing it is basically trivial.
When you have a bug in your code that you see the first time you try to run it, you will be able to fix it in no time at all, because all the code is still fresh in your mind. And if you find a bug in already shipped code, you’re going to incur incredible expense to address it.
Another great thing about keeping the bug count at zero is that you can respond much faster to the competition. Some programmers think of this as keeping the product ready to ship at all times. Then if your competitor introduces a killer new feature that is stealing your customers, you can implement just that feature and ship on the spot, without having to fix a large number of accumulated bugs.
6. Do you have an up-to-date schedule?
Which brings us to schedules. If your code is at all important to the business, there are lots of reasons why it’s important to the business to know when the code is ready. Programmers are notoriously crabby about making schedules. “It will be done when it’s done!” they scream at the business people.
Unfortunately, that just doesn’t cut it. There are too many planning decisions that the business needs to make well in advance of shipping the code: demos, trade shows, advertising, etc. And the only way to do this is to have a schedule, and to keep it up to date.
The other crucial thing about having a schedule is that it forces you to decide what features you are going to do, and then it forces you to pick the least important features and cut them rather than slipping into featuritis (a.k.a. scope creep).
7. Do you have a spec?
Writing specs is like flossing: everybody agrees that it’s a good thing, but nobody does it. It’s probably because most programmers hate writing documents. As a result, when teams consisting solely of programmers attack a problem, they prefer to express their solution in code, rather than in documents. They would much rather dive in and write code than produce a spec first.
At the design stage, when you discover problems, you can fix them easily by editing a few lines of text. Once the code is written, the cost of fixing problems is dramatically higher, both emotionally (people hate to throw away code) and in terms of time, so there’s resistance to actually fixing the problems. Software that wasn’t built from a spec usually winds up badly designed and the schedule gets out of control. This seems to have been the problem at Netscape, where the first four versions grew into such a mess that management stupidly decided to throw out the code and start over. And then they made this mistake all over again with Mozilla, creating a monster that spun out of control and took several years to get to alpha stage.
8. Do programmers have quiet working conditions?
There are extensively documented productivity gains provided by giving knowledge workers space, quiet, and privacy. The classic software management book Peopleware documents these productivity benefits extensively.
Here’s the trouble. We all know that knowledge workers work best by getting into “flow”, aka “in the zone”, where they can fully concentrate on their work and fully tune out of their environment. They lose track of time and produce great stuff through absolute concentration. This is when they get all of their productive work done. Writers, programmers, scientists, and even basketball players will tell you about being in the zone.
The trouble is, getting into “the zone” is not easy. When you try to measure it, it looks like it takes an average of 15 minutes to start working at maximum productivity. Sometimes, if you’re tired or have already done a lot of creative work that day, you just can’t get into the zone and you spend the rest of your work day fiddling around, reading the web, playing Tetris.
With programmers, it’s especially hard. Productivity depends on being able to juggle a lot of little details in short term memory all at once. Any kind of interruption can cause these details to come crashing down. When you resume work, you can’t remember any of the details (like local variable names you were using, or where you were up to in implementing that search algorithm) and you have to keep looking these things up, which slows you down a lot until you get back up to speed.
9. Do you use the best tools money can buy?
Writing code in a compiled language is one of the last things that still can’t be done instantly on a garden variety home computer. If your compilation process takes more than a few seconds, getting the latest and greatest computer is going to save you time. If compiling takes even 15 seconds, programmers will be bored while the compiler runs and switch over to reading The Onion, which will suck them in and kill hours of productivity.
Debugging GUI code with a single monitor system is painful if not impossible. If you’re writing GUI code, two monitors will make things much easier. Most programmers eventually have to manipulate bitmaps for icons or toolbars, and most programmers don’t have a good bitmap editor available. Trying to use Microsoft Paint to manipulate bitmaps is a joke, but that’s what most programmers have to do.
Top notch development teams don’t torture their programmers. Even minor frustrations caused by using underpowered tools add up, making programmers grumpy and unhappy. And a grumpy programmer is an unproductive programmer.
To add to all this… programmers are easily bribed by giving them the coolest, latest stuff. This is a far cheaper way to get them to work for you than actually paying competitive salaries!
10. Do you have testers?
If your team doesn’t have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you’re wasting money by having 
30/hour testers. Skimping on testers is such an outrageous false economy that is a surprise if more people don’t recognize it.
11. Do new candidates write code during their interview?
Would you hire a magician without asking them to show you some magic tricks? Of course not.
Would you hire a caterer for your wedding without tasting their food? I doubt it. (Unless it’s Aunt Marge, and she would hate you forever if you didn’t let her make her “famous” chopped liver cake).
Yet, every day, programmers are hired on the basis of an impressive resumé or because the interviewer enjoyed chatting with them. Or they are asked trivia questions (“what’s the difference between CreateDialog() and DialogBox()?”) whose answers are at the documentation. You don’t care if they have memorized thousands of trivia about programming, you care if they are able to produce code. Or, even worse, they are asked “AHA!” questions: the kind of questions that seem easy when you know the answer, but if you don’t know the answer, they are impossible.
Please, just stop doing this. Do whatever you want during interviews, but make the candidate write some code.
12. Do you do hallway usability testing?
A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability problems in your code.
Good user interface design is not as hard as you would think, and it’s crucial if you want customers to love and buy your product. But the most important thing about user interfaces is that if you show your program to a handful of people, (in fact, five or six is enough) you will quickly discover the biggest problems people are having. Even if your UI design skills are lacking, as long as you force yourself to do hallway usability tests, which cost nothing, your UI will be much, much better.
References:
Joel Test. https://www.joelonsoftware.com/
Go here for more software engineering articles.
love it!
Cheers 🙂