Click here to monitor SSC

What Counts for a DBA: Realism

Published 14 February 2013 4:49 pm

“I need a database designed, which will not only clear up our current order processing issues, but will make our processing 1000% more efficient …”

The manager had that charismatic gleam in his eye that makes you just about believe anything he says.

 ”… and I know you guys can do it…”. Just as our data architect and the rest of the data team begin to sit up in grateful anticipation, the manager finishes his sentence “…by Tuesday…this Tuesday”.

And why not, he probably thought as he airily waved us out of his office. After all, it’s just a case of filling out a few tables in the design tool, drawing some lines between them and hitting “generate”, right? In that light, the Tuesday deadline seems overly generous!

This is, of course, the stereotype of the quixotic nature of managers and their unrealistic dreams which has been immortalized by Scott Adams so well that it is hard not believe that it is reality: but is isn’t entirely fair. While managers sometimes eschew common sense to push their underlings to exhaustion, we DBA/programmers, even the most practical ones, are equally to blame for having unrealistic expectations of what we can achieve in a given time.

The problem is that many DBAs estimate a project in terms of their capabilities alone and the speed with they could complete their part of it. Many of us are passionate about what we do, to the extent of being hobbyist DBA/programmers in our spare time. Yes, working alone, I could have that database designed and coded in the next seven days! What many a DBA fails to do is pause, and to identify carefully every step required to complete the project, and then estimate, realistically, how long each will take.

How long will it take to discover the real business requirements that underpin the boss’s overriding requirement to “get it done”? How long will it take to asses and refactor any existing code? How long for testing? And so on. Inevitably, it turns out that other people will be involved with the project too. For example, what about a usable interface for the customer that has to be created after the database is designed and built?

Realism includes the ability to make a proper estimate of how long a project will take, or indeed, will cost. Every DBA would like to implement their systems to 99.9999% uptime, but we have to be realistic in terms of what that means for costs, maintenance windows (or lack of, since 99.9999% uptime means less than an hour of downtime a year!), manageability, and staffing to be on alert 24 hours of every day of the year. Is it realistic? It can be, but depends on the need and the budget, and nothing else.

As with most things, realism improves with experience and becomes instinctual. Your idealistic inner hero lives in (dis)harmony with your curmudgeonly inner bureaucrat, exhorting caution and realism. Some companies still exercise a kind of “enforced realism”; once you suggest a time you stick to it, even if it means 18-hour days plus weekends. The more progressive employers reach a better understanding of a project’s timeline using techniques such as planning poker.

Regardless, in a company that manages productivity realistically, our manager’s request would have ended with laughter and “seriously, though, how much do you think we can finish by this Tuesday or do I need to ask for more time?” Either that or he would have started with “Once upon a time…

On March 13, I will present a webinar on why being observant really counts for a DBA. You can sign up here: http://www.red-gate.com/products/dba/sql-monitor/entrypage/sql-monitor-what-counts-webinar

12 Responses to “What Counts for a DBA: Realism”

  1. grenac says:

    I agree with the sentimaent. But – no, the Scott Adams version of reality is toned down! I have regularly come across many projects, managers and others far more surreal, bizzare and just plain stupid than the cartoons. To be fair, IT project management has grown up a bit since I left the profession 12 years ago. But, of course now we have everyone else, from Jo public on the street up to Sales and Marketing (or S&M for short), joining in and telling us how the technology works. And, that is from a general public, 99.9% of whom can’t explain how the phases of the moon work. Yet, when you can’t provide “the moon on a stick” – they get upset with your negativity. A healthy cynicism, colloboration and project managers learning to say “NO” first, are how you start to work towards the 1000% improvements.

  2. Keith Rowley says:

    Where does the DBA’s native paranoia go when faced with a development project? It’s like we think we are the only component in the system that isn’t bound to fail on us at the worst possible moment.

    We are data geeks (most of us) so why don’t we as an industry have better methods of applying some of that data to ourselves when it comes to project planning and management?

    I don’t have answers to these questions, but whenever someone brings up this subject of DBAs and unrealistic development expectations this is what I think about.

  3. ralph s says:

    Excellent comments, Louis. Certainly in the right direction. Still, I hear you somewhat accepting the ground rules of those who do not discern reality. You said,

    ” What many a DBA fails to do is pause, and to identify carefully every step required to complete the project, and then estimate, realistically, how long each will take.”

    The acceptance of an implied false ground rule is that the work is in declaring or coding and executing what needs to be done. In fact, the work is most often in the carefully identifying every step. That problem space is really what is hard to estimate. Then, in order to prove out a solution one often needs to apply a hueristic approach, an estimate and narrow in reitterative learning approach to come up with a workable solution. This is not the same as estimating framework construction of a house but is more akin to architecting on the fly.

    Still, I do think think this is an excellent article leading the discussion in a productive direction.

  4. Louis Davidson says:

    Grenac: You mean “regularly” in the past tense naturally. I know that no one I work with has unrealistic expectations.. ahem..

    But I can’t agree with you more. And as a database control freak, it drives me nuts that regularly I end up doing things in a substandard sort of way to finish my part of the build quickly only to have tons of delays for various issues (some of which have to do with the substandard sort of way it got done to “save” time.) If we were realistic about time, skills and abilities, but a bit more reasonable about quality being important… life would be better for sure.

  5. Louis Davidson says:

    @keith rowley:

    I will admit I am completely terrible at estimation….unless I have solid requirements to work from.

    “We are data geeks (most of us) so why don’t we as an industry have better methods of applying some of that data to ourselves when it comes to project planning and management?”

    That is a really good question. For me, the real answer lies in the actual ability to do “planning”. Often, when I am in a room with procedural programmers, relational programmers, and users in a room, all of us will seem to speak vaguely the same language, but our babel fishes are set to different frequencies.

    Most of my substandard work (and timing) has to do with me thinking I needed to build an X, but really need an X + Y, where Y is really a complex equation that someone else didn’t know how to get out of their brain. Even worse, simplification of business processes is not really a “typical” thing. We have computers, so they ought to be able to do “Anything” right. There is that computer that plays chess…

    Sadly, one is rarely praised for saying… Whoa, before we start building, let’s get this right! And if you could stop doing Y, then we could get done quicker…

  6. grenac says:

    Why do think I left project management ? Here are some clues. I was a fully trained project planner estimator and had over 10 years of experience in planning projects. I was the only member of staff in a very big IT organisation that was actually a certified project manager and the SME for several aspects of PM. But, day after day would get newbies who’d just been dropped into the role trying to tell me how things should be done or how they would turn out. Not only by staff with virtually no experience, but no data to justify anything. I had plenty of data, but of course no-one wanted to believe what it or I was telling them.
    As a DBA, I do not beat myself up about projects that overrun or have to be cut short. And, if I say a restore “will take 6 hours”, that’s what it will take – no arguments and whatever the S&M director says.
    After all, as my Software Engineering was always drumming home – “the majority of IT project disasters are down to 2 things: poor planning and poor requirements”. The former can be improved (usually – if you have enough control over it), but the latter, in my experience, is still generally an intractable problem. In summary – don’t beat yourself up when a project goes wrong, most of them do and most of it is really not going to be your fault.

  7. Lee Dise says:

    Ugh! Brings back memories. My own closest call with the Dilbert Singularity — best envisioned as a black hole of stupidity that sucks in all intelligent thought within its event horizon — took place about twenty years ago, when I maintained databases for the bureaucrats of nuclear Armageddon at USSTRATCOM. My boss summoned me to his office, brooded in his thoughts for a moment, pensively glanced up at the ceiling, and then finally spoke: “Lee, I want you to design a new database. The first requirement is that the database must be flexible enough to accept any input the users desire.”

    I’m taking notes, so I wrote, uh huh, lots of VARCHAR, allow NULLs, loose tolerances, gotcha…

    And then my boss said, “And the second requirement is that it must not allow them to enter in any wrong value.”

    I stopped writing. I looked him square in the eye and said, “You want me to design a database around a paradox? Obviously, if any input is valid, there is no such thing as an invalid input…er, right?” I still had hope that he was pulling my leg.

    My boss waved his hand around dismissively and said, “You’re the engineer; make it work.”

    As one might imagine, with requirements like that, the project was doomed to fail. And proceeded to do so. My modest contribution was to ensure that it failed on schedule.

  8. Louis Davidson says:

    @lee dise

    Awesome. The funny thing is, I feel really dumb trying to make up examples for writing, but it seems that someone always has something even worse than I can come up with :)

  9. Sunil gowda says:

    How to develop a skill towards efficient database design? Like basic steps needed

  10. Robert Young says:

    I had held out hope/longing that the combination of fast, high core count machines with SSD as primary storage for RDBMS would put an end to “de-normalize for speed” meme. And thus usher in the Emerald City of organic normal form databases. Such databases are, by definition, more flexible than the hierarchical types favoured by Suits and Kiddie Koders.

    Alas, we’ve made few steps along the Yellow Brick Road. More (most?) pundits still deride SSD as unsuitable for RDBMS. We’re still ignoring the RM, and thus easily plinked by knuckleheads.

  11. paschott says:

    Louis, if you’re looking for examples, remember thedailywtf.com. For my part, I’ve been on the side of some overly optimistic programmers who insisted that the project could be done in a shorter time than was actually needed. That led to everyone working extra hours (not because they were needed all of the time, but to be part of the team). We actually got it done “on time”, but the realistic goal should have given us another couple of months. On the plus side, we got in some necessary refactoring in that project that led to much better DB behavior for storing some of our dates/comments/completion requirements. :)

    Of course, I’ll admit to sometimes just throwing out a shell of a proc or mocking up something I know is horrible so we at least have something, then coming around later to fix it. Usually I manage to do that in the next day or two. I also work with a pretty good group of people who are realistic about estimates. If anything we might go a little too far the other way in over-estimating the time, but we’re right more often than not.

  12. jerryhung says:

    Speaking as a DBA with a MBA

    Realism is different with various sizes of companies too
    Small company – one-man IT show possibly
    Big company – you have dedicated PM or PMO just to tell you what to do, and you do it, only to be delayed by something else

    Funny part I find is there isn’t much “punishment per se” even if you don’t make it by the deadline. Sure you’ll get endless email or meetings or managers dropping by, but unless you really suck, you aren’t really gonna be fired. BUT…you may be the first to be laid off :)

    Either way, I’m proud to say I never missed a deadline and often have to slow down to not set too high of an expectations

Leave a Reply