January 15th, 2010
You know how sometimes you read something and it niggles away at you and you can’t quite get it out of your system? This happened to me with a blog post by Ivan Walsh from November last year. In his post (How do you measure technical documents? What metrics do you use?) Ivan describes how a tech writer got fired from the docs team with the justification by management that “technical documents provide no value”.
This is something we’ve probably all come up against at some point in our careers. Fortunately I work for a company where common sense tends to prevail and a well-reasoned argument, from any source, can be influential. But I know of lots of companies where things are very different. Documentation is often seen as just a cost centre. And when the spreadsheet guys are looking at what’s costing money and what’s winning new business it often looks like documentation is something you could do without. Because – think about it – when did good documentation, by itself, ever win one dollar of new business?
The referee was fantastic!
So, how does a documentation manager prove to the numbers guys that they need to spend money on documentation? It’s not easy. And the reason it’s not easy is partly down to the referee scenario. Think of a great football game (and I’m not talking US-type football here, I’m talking about the football the rest of the world plays). With a great game of football you’re never left talking about the referee at the end of the game. In fact after any game of football you never find yourself talking about the referee unless he did a bad job. You rarely notice the referee if he makes good decisions, lets the game flow, nips trouble in the bud, acts impartially … does his job well. You only notice him when he disallows a perfectly good goal, books the wrong person, consistently favours one team, awards a last-minute penalty when it was a blatant dive, and so on.
It’s the same with documentation. People only start talking about the documentation when it’s incorrect, badly written or absent. With documentation: no news is good news. And this is dangerous. It’s easy to cut the documentation effort because initially no one will notice the difference. The quantity and/or quality of output from the docs team will drop off, but because much of the documentation effort is for new product that’s not going to be released for a while – and even once it is released it won’t be the documentation that customers focus on first – and because the docs team will still endeavour to make sure the high priority work still gets done, it’s going to be a while before this change affects customers. In other words you can get away with it for a while before shit starts to hit fan.
The reverse is, of course, true. It takes a while for cuts in documentation resource to work through and start hurting your customers, but it also takes a while to recover from that situation once the powers that be have decided something needs to be done. In fact, after the problem has been recognised things will continue to nose dive, even after you’ve recruited replacement staff, and even supposing the replacement staff don’t need any training and can magically walk in and start being productive from week one. During this time more and more customers will be hit by the documentation-related problems. A head of steam will build up. Customers will talk amongst themselves about how difficult the software is to use. News might even spread to prospective customers. But by this time the damage that was done was done months before – and there’s no quick fix.
It’s back to the referee scenario. Not only do customers never talk about the documentation unless they’ve got something to complain about, but it’s also true to say that while documentation never won a sale, it is sometimes a contributory factor in a lost sale.
This is the point in proceedings – when you’re doing your best to recover from resource-related documentation problems, but customers are still complaining – that necks are on chopping blocks and axes are poised. And you can bet those necks aren’t the necks of the budget police who wouldn’t let you replace a colleague who left, or wouldn’t grow the docs team in line with the growth in the development team, or who “restructured” the docs team to allow budget to be reallocated to more profitable areas of the business.
Docs are important? Go on then … prove it!
This all sounds very grim. So how do you avoid getting into that mess? You might be convinced that what you do is important – but how do you convince others? How do you prove the value of what you’re paid to produce? How do you answer the question Ivan Walsh got asked: “What metrics do you use to tell if the documents are successful?”
Well I’ve been thinking about this and reading around and there’s surprisingly little out there to help me answer this question. There’s been quite a bit written about what metrics you might use to judge individual technical writers – and maybe measure one technical writer against another – or how you can assess the quality of particular documentation deliverables – for example, the number of comments that were raised during pre-publication review, or the number of documentation bugs raised after publication. But the metrics we’re talking about here are measurements of the value of the documentation effort within your company.
I came across a LinkedIn discussion that mainly concerns the micro-metrics of assessing particular documents or writers (rather than the macro-metrics of assessing the documentation effort itself) but these two quotes, I think, touch upon the bigger picture:
in the final analysis any document is only valuable if it answers the questions the user has so they are not required to make some sort of call to a support person
Keith Oxenrider
A better measure is the readers' view of the documentation. Do they like it? Can they use it to solve their problems? Does the documentation reduce the number of calls to the tech support center? Does the documentation aid the tech support center in quickly resolving client problems?
Dave Gardner
The references to support point to a possible solution.
Donald S Le Vie Jr wrote an interesting and useful article in the December 2000 edition of Intercom (“Documentation Metrics: What Do You Really Want to Measure?”), although he mainly discusses writer-specific metrics that he believes you really shouldn’t bother trying to use – for example, hours per page, pages per hour, documents released per month or per writer, errors per page, percentage of time schedule dates were met, etc. However, he ends the article with some metrics he believes are useful:
- Web-based feedback – e.g. find out what customers want more of, or don’t need, in future versions of documentation
- Customer usability of beta or final versions of documentation. Get them to rate the documentation.
- Team up with Customer Support to make sure it’s recorded when support calls are documentation related. “Track this information over time to show the decrease in the number of documentation-related calls so you can assign a dollar figure (a quantitative measure) to any metric.”
Le Vie stresses the importance of regularly broadcasting these metrics throughout management.
Conclusion
In the LinkedIn discussion Padric O'Rouark says:
Sell “help” separate and you would find very few buyers.
That’s true, but it’s also misleading. The fact that you wouldn’t pay for something does not mean it has no value. When I buy a bed from Ikea I expect to find instructions on how to assemble it when I get it home. I wouldn’t pay for those instructions separately though. If I was expected to do this I’d just shop elsewhere. As an aside, the poor quality of Ikea’s assembly instructions is one of the things that makes me dislike the company as a whole (that and the fact they try and steer you round the whole store before you get to the checkout, rather than letting you get in, get what you want and get out).
If customer satisfaction is important to a company then the philosophy of doing only as much as you need to do to make a sale, and not one jot more, probably only works if you’re a make-a-quick-buck-and-disappear business, or if you’re competing solely on price and you’re confident customers will accept poor quality products and bad customer service just as long as the price is right.
Customer-facing documentation has no place in a cut-price or fly-by-night business. But in the software business where companies need time to grow and therefore customer retention is important, user assistance is valuable. If you’re trying to sell corporate software then the people with the purchasing power, the decision makers, aren’t the people who are going to be using the software day in, day out. But if you’re in the business for a few years then some of those people who have to use the software in their daily jobs get promoted and end up being decision makers, or at least decision influencers. If you’ve had to struggle for several years using a badly designed bit of software from Company X, that had little or poorly written user assistance, then, later in life, when you get the chance, you’re going to make damned sure you don’t let Company X inflict any more such pain on any of your staff!
It’s hard to come up with metrics to justify the current size of the docs team, beyond all argument. Bean counters are interested in having more beans to count, so winning new deals and satisfying contracts in order to be paid quickly are things that tend to impress them. In this regard, you could look at the number of documentation requirements within Request For Proposal documents from prospective customers, or you could find out whether documentation was scored as part of customer acceptance tests?
But other than that, what can you come up with to use as proof of your value to the company that pays your wages? Well, the source of your wages may be the key, because really it’s your customers who pay your wages and they are the only ones who can provide metrics that will convince management of the value of what you’re doing.
I started this piece by suggesting that if you sack tech writers, or reduce the proportion of writers to developers, sooner or later customers will start to suffer. If that’s true then it should be measurable. Keep a record of the number of documentation-related support calls or bad customer feedback mentioned in engineer site visit reports. Have numbers gone up or down over the past six months?
You might never be able to prove that customers are happier because of the work you do, but if you’re able to prove that customers are less annoyed with the products and increasingly able to get on with their business without calling support, then you might just be able to convince people that documentation is valuable.
Potentially similar posts
February 4th, 2010 at 2:44 pm (#)
Hi Alistair,
‘Pride’ keeps many projects alive, which should really be pulled or binned.
Internal politics can also be a minefield.
What’s he got against lizards, by the way?
Ivan
February 4th, 2010 at 9:30 pm (#)
You're right. I also think people not directly involved in a project that starts going wrong often react by putting their fingers in their ears and going "La la la la." People often don't want to hear bad news so they go out of their way not to hear bad news. Eventually it can't be ignored any more, the plug gets pulled and the sacrificial sacking takes place.
On the other hand I've experienced a project that got into bad trouble, but when the plug was pulled it looked like the project might be salvageable. That's a tough decision to make. Lots of time/money/effort has been spent. The temptation is just to give it a little more time. But the decision to amputate the rotting limb, while difficult to make, is always the right decision.
February 6th, 2010 at 11:47 am (#)
Thanks for the pointer to McConnell and his concept of thrashing!
It's a great illustration of documentation projects gone awry in later stages: All visible progress gets replaced by unplanned rewriting and editing, project management and meetings.
And it underscores the importance of getting your documentation design and structure approved, so everybody agrees on what the deliverable format and structure will be.
February 11th, 2010 at 9:12 am (#)
[...] I first heard about McConnell and how he’s relevant for tech writers in ITauthor’s blog post “It’s not about writing … it’s about shipping“. [...]