Click here to monitor SSC
  • Av rating:
  • Total votes: 17
  • Total comments: 2
David Poole

Technical Debt and the Cultural Gap

13 January 2014

Sometimes, technical jargon is often so readily understandable by the technical community that they forget that it may be interpreted quite differently by the rest of the business. 'Technical Debt' is an example of a metaphor that is considered very differently by others. By failing to adopt a common language, you could be giving a message about your IT project that is quite different to the one you intended.

The Cultural Gap

Technologists often have considerable difficulty in communicating with the business that they work with, and this can result in strains in the relationship between business and IT. As a result, technologists often regard the business as impulsive and irrational in the decisions they take, and the business assume that technologists will be inflexible, engrossed by technology, and incapable of understanding the urgency of business demands.

Let’s take the simple example of Technical Debt. It is a metaphor that has now joined the mainstream of technological jargon. Ward Cunningham’s metaphor encapsulates neatly the idea that, by doing software development the quick and dirty way, we incur in a sort of debt that must at some point be repaid, just like financial debt. The interest on the ‘technical’ debt is the extra effort that we have to do in future software development in consequence. We can postpone repaying the debt by continuing paying the interest, or we can re-engineer the software properly and avoid any further interest on the loan. We all feel sure that technical debt represents an obligation to redo the job properly in the future. Why doesn’t the business always buy the idea?

Technical Debt viewed from a different angle

I had often wondered at this peculiar inertia as it hampers attempts to fix technical debt. It is easy to explain this inertia as being due to balking at the technical challenges of the task: Although that is certainly a factor, there remains a more general unwillingness to address technical debt that can’t be explained by such ‘refactorphobia’.

A chance conversation with an IT director on the board of a PLC gave me an answer. There is an inherent conflict at the heart of every company listed on the stock exchange in determining the point of compromise between innovation and efficiency, revenue generation and cost saving. Projects to resolve technical debt, address data quality issues and refactoring code are mere pawns in this broader conflict.

Revenue Growth vs Efficiency: The Dreadful Algebra of Necessity

"What would you sooner have, a £1million cost saving or a £1million increase in revenue"? This was the question I posed and I got the opposite answer to the one I was expecting.

In a company quoted on the stock exchange profit generated by revenue growth is far more attractive than profit generated from cost savings. This is true even if the profit from revenue generating activity is less than that from cost saving. The reason for this is simply as follows: -

  • A city analyst will take an increase in revenue to mean that the company has further growth potential and therefore likely to be worth investing in. This drives up the share price and pleases investors.
  • Growth generated by cost cutting and efficiency drives, on the other hand, especially if accompanied with modest or diminishing revenue, is seen as a sign of a company that has run out of steam and is struggling.

Because of this, any opportunity for revenue growth has to be seized with both hands. A cost-cutting programme or efficiency drive can be initiated at any time and success is comparatively easy to achieve. If the emphasis of the enterprise is on cost cutting and efficiency then this is a race to the bottom. The size of the opportunity is finite.

Now ask yourself "does addressing technical debt sit on the revenue generating side of the equation or the cost/efficiency side"?

Every business cares about efficiency, and so is unlikely to want to accumulate technical debt: On the other hand, it faces the dreadful algebra of necessity.

So how can we rebalance the scales in our favour?

Find a business term for technical debt

So many problems in life have their root cause in a breakdown in communication. Do business people truly understand what "technical debt" actually means?

Take as an example a discussion between a non-technical product owner and a technical team leader during which the team leader is heard to say "we could do it in two weeks but it would incur significant technical debt".

What the product owner will hear is

  • It can be done with some technical debt. I am not sure what technical debt is exactly but…..
  • Technical debt is something that an IT department will pay on my behalf. So that means….
  • I can have it my way at no cost to myself. This also means that….
  • I can promise delivery in 2 weeks, but I will say 10 days because IT always pad their estimates.

For this reason I do not like the term technical debt. I would try to describe it in terms that are familiar to the business person and directly relevant to their own interests. Call it a blocker to business agility, or illustrate how it can pose a risk to the objectives of the business person: Talk instead about the added uncertainty of timescales for delivering subsequent enhancements to applications. If possible you should illustrate your point by using the impact of technical debt on those enhancements that are already eagerly anticipated by the business community.

If possible, educate the IT community to describe technical debt in terms that the business can understand; to communicate technical issues more effectively.

Remember what technical debt is for!

Remember that software is only earning its keep when it is in use. Until such time it has simply incurred cost.

Technical debt is accepted because the organisation believes that the cost of the technical debt is outweighed by the benefit of having the software. This belief must be based on a clear understanding of what the costs actually are. This is why it is so important to describe technical debt in business language.

Beware of accepting or allowing technical debt unless everyone involved has a clear understanding of the benefits and costs.

Bear in mind the conflicting influences before accepting technical debt

I learnt a great deal from my conversation with the IT director.

  • If you accept technical debt in iteration 'n' but have a clear plan to address this in iteration 'n+1' then there is a chance you will succeed.
  • If you have a plan to fix the debt in iteration n+2, n+3 and so on then the likelihood of NOT fixing technical debt increases exponentially.
  • Unless there is an explicit plan to fix the technical debt that you accept today, then it probably won't be fixed…..ever!

With these points in mind, ask yourself how you can design your system in a way that increases the likelihood of an item of technical debt being repaid.

A possible tactic is to push for a reduction in user features over a reduction in functional completeness of a delivered iteration.

If the features requested by the users are worth having, then the user community will continue to prioritise them in future iterations. It may be that, on reflection, the user community decides that certain features are not as important as they initially thought!

Be accountable for technical debt

If you present a list of options to someone, then they are very likely to pick the option that suits them best, not the one that suits you. If you have been daft enough to put forward an option that, if chosen, will cause you real problems then you have put your area of responsibility at risk and thus contributed to organisational risk.

Therefore technical people should not abdicate technical decisions to nontechnical colleagues. This does not give either party the right to dictate solutions as all relevant considerations have to be taken into account.

A better approach would be to take forward a list of options that are acceptable compromises and be clear on the course of action you recommend. Depending on your organisational culture it may be wise not to mention those options that would cause larger problems. If you must present them, then I would set out my case as follows:-

  • We recommend this for the following reasons….
  • The following also have merit….
  • We rejected the following for these reasons….

This is not dictating the course of action but it is putting forward a marker for the desired direction of travel. This puts you in a strong position from which to enter into dialogue.

Consider the wider impact for the organisation

Earlier in this article I mentioned that technical debt is accepted because the organisation (not an individual) believes the benefits outweigh the costs of doing so. For an organisation to make this decision then they have to know who will be affected and in what way.

I would recommend putting together a RACI (Responsible, Accountable, Consulted, Informed) matrix for the system and act on it.

A non-technical product owner is within their rights to accept that an item of technical debt may make their own life more difficult. They may choose to bear that cross in order to start using the software earlier.

They are not entitled to accept a debt item that puts one of their peers at risk until they have agreement from them to do so. Department A cannot accept something that will harm Department B unless Department B agrees.

Nothing is achieved if a consequence of accepting technical debt is that one area of the business gains but another loses, possibly to a greater extent. A robust RACI (Responsibility Assignment) matrix will help ensure that the various parties can determine whether the benefits outweigh the costs.

It is here that oversight and governance come into play because it is probable that both Department A & B will cheerfully accept whatever will allow them to achieve their objectives. They are likely to consider their own needs rather than consider the overall results.

When accepting technical debt will cause a problem, say so!

Every project should have a RAID log (Risks, Actions, Issues and Decisions). If you know something will have a negative effect on a business role then at the very least you should put an entry in either "Risks" or "Issues".

This should be escalated to the project or iteration manager and thus on to the affected party.

If your organisation is large enough to have different departments then the ambiguity at the boundary of their responsibilities can lead to things being missed. Don't assume that someone else has spotted it. Take personal responsibility for raising it.

By not raising the matter you are allowing the project to proceed at risk. It could even be that the negative affect of technical debt on one department would actually render the project invalid!

It is best to find this out early rather than face a painful bloodletting during a project post mortem.

When a decision is made, commit yourself to that decision

For a given item of technical debt you have done the following:-

  • Identified all parties that will be affected by the item. One of these will be technical.
  • Highlighted the risks and costs to them all, in language that they understand
  • Put forward your recommendations, considerations and options

With all points being considered, they will decide on the best course of action. The decision may not be the one you hoped for but it should at least be a workable compromise that all parties can sign up to.

At this point you have had your say. Unless new risks, issues or information comes to light, it is time to accept the decision, support it and move on.

It is counter-productive to both you and your organisation to continue to argue your particular view afterwards.

Build your personal credibility

The recommendations I have made this far are for actions that build trust, credibility and respect.

Builds trust

Destroys trust

Clear simple description of the issues in terms the audience will understand

Lengthy prose stuffed with technical jargon.

Putting forward recommendations and options

Being supine gives the impression that you need constant management.  You cannot be relied on to act; you lack initiative.

The RACI matrix shows that you think beyond your own personal area of interest.  If done correctly, it also provides a useful tool.

By focussing on a single product owner, you are not providing them with the support they deserve.

Your inaction may put them at risk and in conflict with their peers.

Supporting decisions made after all relevant facts have been clearly communicated.

Challenging only when substantial and relevant new facts come to light

Continually questioning or revisiting decisions

Pragmatism.  Sometimes technical debt is a necessary evil.

Absolutism and purism.  Inflexibility in the face of genuine business need.

Being proactive.  Putting forward a plan to address technical debt.

Again, being supine.

It is very important to talk to the audience in their own language and idioms.

I had a colleague who used to build a logical and meticulously researched case for a particular course of action. As far as he could see he had covered every angle, thought of every possible consequence and outlined the appropriate mitigation.

He would go into a meeting and his case would be rejected; not on any factual grounds but simply because someone would say "I don't think we should do that"! A single opinion would negate weeks of research into the facts.

Eventually he worked out that his audience were not interested in reams of data. They expected him to know the relevant facts but what they were looking for was for him to say "I believe this is the right course of action and I base this on considerable research". The complexity of his original method destroyed trust and simply created fear and doubt. This was leaving his case vulnerable to a clearly and confidently stated (and often uneducated) opinion.

Now when he talks, people listen!

Concluding thoughts

The suggestions I have put forward in this article are all ‘soft’ skills. Technical practises and competence are obviously necessary but product owners hold the whip hand when it comes prioritising work. Unless you can influence them you will not be able to bring your technical skills to bear.

My personal experience is that business problems are fractals. If you identify problems with a discipline you will see the same problem at departmental level, at division level and probably at organisational level. Learning to think and communicate in the terms of the next tier up in the management hierarchy is the key to solving those problems.

David Poole

Author profile:

David first became interested in databases while working for a B2B direct mail company. The idea that prospect/suspect/customer behaviour could be predicted and influenced through data was and remains an enduring fascination. Working for McCann-Erikson and seeing those ideas applied to a wide range of B2C and B2B clients nurtured this interest but also lead towards the DBA path as database systems at that time simply were not powerful enough to allow the required analysis without significant technical involvement. Moving on to a company specialising in information delivery introduced the concept that data can only be useful if it can be presented in the correct context and targeted at the correct audience Work with the Tote and Moneysupermarket have introduced the rigours of high transaction loads and 24/7 support capabilities and also the design disciplines to ensure efficient data flow. So far his career journey has exposed him to a rich DBA experience which some of which includes SQL Server versions 6.5 through to 2012. He is now the data architect for Moneysupermarket.com

Search for other articles by David Poole

Rate this article:   Avg rating: from a total of 17 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.


Subject: I liked this
Posted by: jh72i (not signed in)
Posted on: Tuesday, January 21, 2014 at 12:39 AM
Message: I like this article very much and agree with most of it.

One thing I have found, though, is that very (very!) often technical representatives don't work hard enough to actually understand the business. This results in any attempt to ‘translate’ technical language to business language failing the credibility test – i.e. the business folks just don’t believe the techie ‘gets’ their world. This has a knock-on effect of the business starting to wonder if this techie is actually equally incompetent in his/her own world.

A breeding ground for culture of mistrust that is just so destructive to any business.

Subject: This topic/article adeptly hits at the heart of IT failures
Posted by: Anonymous (not signed in)
Posted on: Saturday, February 01, 2014 at 8:31 AM
Message: The two themes of communication and management are equally vital to removing the cultural gap and mistrust. The role that often has the autonomy to address these themes is the project manager or project portfolio manager. As David notes, someone could "push for a reduction in user features over a reduction in functional completeness." Moreover, all condoned PM methodologies are built around some level of quality, and mandate some level of "functional completeness." Therefore, even if there is noone with the title of PM, someone has to put on the PM hat and say, "We are NOT doing software development the quick and dirty way."
--Bruce
See my blog, The Business Analyst Side at bbweblog.wordpress.com

 

Top Rated

We don't need Source Control: we're Database Developers
 As part of our long-running series of articles where we ask working database developers how database... Read more...

The Proposals Conundrum
 When you work for a small software development (or any services) company, one of the major challenges... Read more...

David Heinemeier Hansson: Geek of the Week
 Ruby on Rails, the open-source web application framework, grew out of David Heinemeier Hansson's work... Read more...

Alex Payne: Big in the IT Business
 Alex Payne worked on developing Twitter for three years. When he started, it was a small side-project:... Read more...

Seth Godin: Big in the IT Business
 Seth Godin has transformed our understanding of marketing in IT. He invented the concept of 'permission... Read more...

Most Viewed

The Future of Reflector
 Simple Talk asked freelance writer Bob Cramblitt to sit down with the two people behind the agreement... Read more...

Linus Torvalds, Geek of the Week
 Linus Torvalds is remarkable, not only for being the technical genius who wrote Linux, but for then... Read more...

Bad CaRMa
 From hope and euphoria, to desperation, firings and the ultimate demise of a company. Tim Gorman charts... Read more...

Driving up software quality - the role of the tester
 Have you ever wondered what a software tester does? Helen Joyce, test engineer at Red Gate software... Read more...

Don Knuth and the Art of Computer Programming: The Interview
 Fifty years after starting the 'Art of Computer Programming', (TAOCP), Don Knuth is still working hard ... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.