Way back in the year 2000, Joel Spolsky devised a 12-question test to determine the health of software development teams. If you haven’t heard of Joel Spolsky: he’s a big-deal blogger from the 2000s who has since co-founded Stack Overflow and Trello – if you’re at all interested in software, go read his Top 10.
Software developers still use these 12 simple questions to assess if they should accept a job at a specific company, or if they need to improve the practices of their team.
The 12 ‘Joel Test’ questions are simple:
- Do you use source control?
- Can you make a build in one step?
- Do you make daily builds?
- Do you have a bug database?
- Do you fix bugs before writing new code?
- Do you have an up-to-date schedule?
- Do you have a spec?
- Do programmers have quiet working conditions?
- Do you use the best tools money can buy?
- Do you have testers?
- Do new candidates write code during their interview?
- Do you do hallway usability testing?
For each question, answer a simple ‘yes’ or ‘no’. You get one point for each ‘yes’. Which score should you be shooting for? In Joel’s own words:
A score of 12 is perfect, 11 is tolerable, but 10 or lower and you’ve got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time.
Most of these assessment questions shouldn’t come as a surprise to anyone who’s taken a professional software engineering class, worked with other half-decent developers for more than a few minutes, or even just taken a logical look at how software is produced.
I personally like these questions, and use them to assess and improve my own software development practices, but lately I’ve been thinking… how can we create a Joel Test for Radio Technology Teams?
Let’s give it a shot:
The ‘Joel Test’ For Radio Technology Teams
- Do you have a process for versioning and storing configuration changes?
- Do you have an issue tracker?
- Do you fix issues before working on projects?
- Do you have an up-to-date project schedule?
- Do your projects have a spec?
- Do technicians have quiet working conditions?
- Do you use the best tools money can buy?
- Do new candidates solve technical problems during their interview?
- Do you do hallway usability testing?
- Do you have a process for testing changes before making them on live systems?
- Do you have 100% monitoring coverage?
- Do you have a well-defined process for responding to monitoring alerts and after-hours issues?
If you compare my ‘Joel Test’ to the original ‘Joel Test’, you’ll see the original points 1, 3 to 9, and 12 to 12 have direct equivalents in my new test. I’ve then gone and added three two new questions at the bottom which are more relevant to good broadcast engineering practice.
Let’s break these down:
1. Do you have a process for versioning and storing configuration changes?
A lot of a broadcast engineer’s day-to-day involves making configuration changes to systems. These changes need to be organised and tracked, which typically means taking regular backups or exports of the configuration, and storing these in a well-organised shared folder or version control system.
Without these practices, you have no way of tracking changes or rolling back bad changes.
2. Do you have an issue tracker?
Life’s busy, and the human brain can’t really remember more than a few issues and their current status. You need an issue tracker, even if it’s something simple like Trello. This will help you track all reported issues, make notes for future reference, and prioritise the issue.
3. Do you fix issues before working on new projects?
We all want to work on new projects, and roll out the latest equipment or software. But if you don’t prioritise the resolution of issues with existing systems, you will accumulate vast and troubling technical debt. By not prioritising issue resolution, you are creating an unhealthy culture where it’s okay to neglect your end-users.
4. Do you have an up-to-date project schedule?
A proper project schedule will reflect the business needs of your organisation, and will help the whole team stay on track and monitor their progress. Projects will slip without well-defined goals and timelines.
5. Do your projects have a spec?
For each project you work on, have you documented the desired outcomes? Have you formalised the problem solving approach? Having a proper spec will also help you avoid scope creep, or never-ending-project-syndrome. Technical people generally hate writing documentation, but it’s important to get this sorted at the beginning of any project.
6. Do your technicians have quiet working conditions?
Here’s the trouble. We all know that knowledge workers work best by getting into “flow”, also known as being “in the zone”, where they are fully concentrated on their work and fully tuned out of their environment. They lose track of time and produce great stuff through absolute concentration. This is when they get all of their productive work done. Writers, programmers, scientists, and even basketball players will tell you about being in the zone.
The trouble is, getting into “the zone” is not easy. When you try to measure it, it looks like it takes an average of 15 minutes to start working at maximum productivity. Sometimes, if you’re tired or have already done a lot of creative work that day, you just can’t get into the zone and you spend the rest of your work day fiddling around, reading the web, playing Tetris.
The other trouble is that it’s so easy to get knocked out of the zone. Noise, phone calls, going out for lunch, having to drive 5 minutes to Starbucks for coffee, and interruptions by coworkers — especiallyinterruptions by coworkers — all knock you out of the zone. If a coworker asks you a question, causing a 1 minute interruption, but this knocks you out of the zone badly enough that it takes you half an hour to get productive again, your overall productivity is in serious trouble. If you’re in a noisy bullpen environment like the type that caffeinated dotcoms love to create, with marketing guys screaming on the phone next to programmers, your productivity will plunge as knowledge workers get interrupted time after time and never get into the zone.
7. Do you use the best tools money can buy?
If you’re going to be the best engineer you can be, you need the right tools to support you. This means a fast computer, appropriate power and hand tools, test equipment, software utilities, etc. Investing in the correct tooling will ultimately save time and improve job satisfaction.
8. Do candidates solve technical problems during interviews?
When adding someone to the team, you can’t hire purely on an impressive resume or a good personality. You also can’t hire based on technical trivia questions.
But what you can do is pose realistic, hypothetical scenarios to a candidate, and assess their problem solving skills. Look out for their ability to solve the problem, but more importantly their ability to rationally think through a problem and use all the available information.
9. Do you do hallway usability tests?
Hallway tests are about testing your systems on your co-workers. Grab someone as they walk past your desk, and get them to try out the thing you’re working on. Systems will better through objective, honest feedback.
10. Do you have a process for testing changes before making them on live systems?
While Joel’s original test is specifically for developers, where you often build a system and then hand it over to someone else to deploy, working in broadcast engineering has a lot more to do with maintaining and supporting live systems.
Whenever you make a change, do you have a way to test this change for unintended consequences before doing it for real?
This is where N+1 redundancy comes in handy – test on the ‘+1’ before you do it for real on the ‘N’.
11. Do you have 100% monitoring coverage?
Good engineering practice means you have everything monitored. Signal paths, equipment, codecs, power, computers, networking gear, transmitters – everything.
12. Do you have a well-defined process for responding to monitoring alerts and after-hours issues?
Good engineering practice also means having a process for responding to these monitoring alerts, even if they’re after-hours. If you are a one man ‘team’, you need clear boundaries to make sure all non-urgent issues get deferred to business hours. If you’re in a larger team, you need on-call rosters and escalation strategies.