Click here to monitor SSC

Tony Davis

Simple-Talk Editor
News, views and good brews

Beg, Borrow, Steal before Build

Published Monday, January 26, 2009 4:36 PM

The art of developing an application, or maintaining a database server, really consists of finding ways of postponing or avoiding programming. As Phil Factor points out, most good DBAs and developers are marked out by a propensity for "bone-headed stubbornness and practice beyond the patience of an ordinary mortal". It's only when this attitude blinds a person to an easier way to do things that it becomes an issue.

 

The problem is that the whole Microsoft culture is one that positively encourages immediate coding. It all seems so easy and satisfying to cut down on the tedium of routine administrative tasks by scripting them. It is hardly surprising really, since by leaping naturally to the use of scripting or, god forbid, programming, you never need to stand back and consider those difficult initial questions like "Should I really be doing this?" or "Is there a more effective approach?" One thing leads to another and before you can wake up to the futility of what you are doing, you have thousands of DTS scripts, or directories full of un-maintainable Perl scripts, written in some bizarre form of shorthand by someone who left the company before documenting it all. And then you find out that there was an existing tool that did the same job. Only better.

 

In any enterprise there is, or should be, an exercise which must be gone through before you even consider building an application. You have to find answers to the following questions before you get asked them:

  • Should we be doing this at all?
  • Have we already got the means of doing this?
  • Can we modify an existing component to do this?
  • Can we buy-in an existing product or tool to do this?
  • Should we get someone else to produce it for us?

Only when the answer to all of the above is "no", when all other ways of developing the application have proved futile, should you consider "rolling your own".

 

Even when the decision is made to build the application, you should reuse whatever components you can, and code only when you can't avoid it. When you are refining a prototype, you'll use scripting just to allow you to make rapid changes, before finally casting the application in concrete with a compiled language. Coding in C#, Java or the like should, in almost every case, be just about the last thing you should do.

 

As always, we'd love to hear what you think. And as usual, the best contribution to the debate, added as a comment to this blog, will receive a $50 Amazon voucher.

 

Cheers!

 

Tony.

Comments

 

timothyawiseman@gmail.com said:

I fully and wholeheartedly concur with almost everything that was said. There are two points to be made though:

1.  When there is an existing application that does almost, but not precisely, what is desired, the benefit of the customization of rolling your own must be weighed carefully.  Sometimes getting exactly what is wanted is necessary, and sometimes the time and expense do not just the added feature.  And sometimes you can coordinate with the developper to have the desired changes put in the next version, especially if you are making large orders.

2. Sometimes after you have made the initial scripted version, there is no need to rewrite it in a compiled language such as Java or C#.  There are some rare things that they can do that scripting languages cannot and at times they run faster, but often if the solution is working it is perfectly acceptable to leave it in Python/Perl/shell script and use it as is.  This is especially true if your compiled language compiles to byte code the way java and C# do by default instead of to true machine language the way C and C++ do.
January 26, 2009 3:27 PM
 

Knowtu » links for 2009-01-26 said:

January 26, 2009 7:03 PM
 

J_E said:

Hi Tony,
I see your point about the importance to not reinvent the wheel. Consider my points for doing the opposite:
- If you build it the way users need it, then you are potentially developing a higher quality product then a shrink wrapped product (they can get 'exactly' what they need)
- Licensing  can be more expensive then dev costs
- Equipment may have to improve to support the 3rd party product
- The developers get to learn how the technology works and may be able to work on other projects, saving even more money. It's also fun.
J.
January 26, 2009 8:57 PM
 

stic said:

Hello Tony,

I think that a lot of us have a in-depth need to solve, pine-point, create, innovate - name it. Therefore, in a foggy (UK) morning like this we will immediately turn on fog lights, even almost no one else does. Probably, all others driving with them are in IT too ;-)

People are lazy, they don't do things if they don't have to. Wheel was invented by someone tried, fire, steam engine, electricity, telephone, are out there because someone wanted to avoid doing something (walking, collecting wood, visiting distant family, etc).

So when we script and we have good intention of saving ourself a few clicks I will challenge you by saying - we are doing a good job, because we are doing it for ourself!
However, when we are trying to make others the happiest users on the whole world that is when we are failing to deliver quite to often... Because? They would like us to deliver a system / script / application that will allow them to do less... All to often we deliver something that limits business users and we deliver such solution because we are lazy and we don't want to cover all bases, or even if we could the domain is too big, too  dynamic, too hard to grasp, too %^$%"£.

And here I will definitely agree with you, we shouldn't write, re-write, script, we should step back and think for a while and ask our users the most crucial question: 'How I can make your work easier?'

So even before asking all questions in area of 'buying 3rd party solution / tool, paying someone else to build this for us, etc..' - first we should be really sure that we need it, that we can't life without it... Then, probably, it is worth building IT solution around that...

That are mine two devaluated pennies ;-)

Regards,
S.
January 27, 2009 2:47 AM
 

mknee said:

I couldn't agree more with your first 3 points - the industry is still full of people who have considerably more technical ability than common sense (myself included).

With regard to your 2nd & 3rd points organizations must take internal scripting & automation seriously - someone must have overall responsibility for controlling automation scripting providing one stop to check if an app exists/partially exists.

Ideas should be collected in one place (Database application anyone?), approving or rejecting proposed works & integrating automation projects into the "bigger picture" as well as enforcing standards & some (perhaps minimal) documentation.

I disagree with you on pont 4 (Buy vs. Build) just because an existing product does a job doesn't mean that is by any means the correct solution.

The costs in time of evaluating multiple products, testing, training & integrating into your environment can often out weigh the costs of a well developed internal application.

I'm not saying build is always the answer but it is not a simple case of only build if an existing application does not already exist.

Mike
January 27, 2009 3:15 AM
 

Daniel Penrod said:

I think a lot of programmers in general lose sight of the real purpose of why they are creating something in the first place - "THE END USER!"  It never should be about them and many make it so personal that even the mere suggestion of them doing something another way results in an input failure in their brains.  These same programmers are not willing to use other languages, even if a little study would result in hours of saved time in the long run.
I find this ideology more so in the anti-Microsoft world than anywhere else.  Although the diehard MS coders are no better if all they do is eat, drink and sleep Microsoft.  
To me a true programmer is someone who has a mindset that says – “I don’t know it all and I will do whatever to get the job done, even it requires me to do something I am not altogether familiar with yet.  I AM WILLING TO LEARN AND EXPLORE NEW THINGS!”
January 27, 2009 6:43 AM
 

Necrit said:

I must point out that although determining the points of purpose and necessity that were listed enable efficient and proactive utilization for an individual most of the time.  There are, however, those many circumstances where another factor comes into play, time.  
As an example, you have a report generated weekly and your department head comes scrambling into your office saying, "We need this report platform modified to handle both Lotus Notes AND Outlook."  Naturally, one begins reviewing all the steps above and logically ends up one hour later searching for the best interop assembly component that is commercially available.  An hour later your boss returns frantic and almost blind with panic asking “Has this been completed?”.  Generally an answer along the lines of "I am conducting research into the proper method to resolve this" is given.  Suffice it to say after a lengthy search, one finds out the chosen assembly is not thread safe and by implementing this causes an application wide crash one month later. Personal adventure of mine that taught me sometimes "Rolling your Own" is more sensible.  While preexisting tools exist, not all of them are ready for prime time production operation even when they have the big company’s names on their design.
I propose a sixth question to this ensemble.  "Will readily available solutions meet my needs?"  I for one whole-heartedly believe reinventing the wheel hurts absolutely everybody.  
• The individual who designed the solution the first time
• The programmer who is struggling to produce it again
• The manager who has all but foaming at the mouth to generate enterprise level solutions in a single day
• The customer who is waiting for their product
• You the person talking to himself asking multiple questions aloud and getting strange looks from your coworkers.

Love your site and always enjoy the articles found here. Have a good day :D
January 27, 2009 7:36 AM
 

raibeart said:

It reminds me of the old cartoon. One man is sitting at a computer, furiously typing. Another is grabbing his coat and going out the door. The man going out the door is shouting back at the other guy. "You start coding, I'll go get the specifications."

All too often, we start coding before we know the "whole story." For example, I had a working methodology to consolidate the 3 databases our customers use into one so we could also consolidate multiple instances to a single one. I used SSIS to move all of the data and had scripts that I kept updated to generate all of the SQL objects. They hired a new guy and told him not to use the proven method, come up with another one. He ended up creating scripts for everything. The consolidation went from taking about an hour to running over night. It also took about 2-1/2 months to re do it.

Isn't progress wonderful?
January 27, 2009 7:39 AM
 

BuggyFunBunny said:

>> Even when the decision is made to build the application, you should reuse whatever components you can, and code only when you can't avoid it

This approach is why many (most???) of us keep accreting more crap into 30 and 40 year old codebases; well maybe not that old in milieus non-mainframe but getting there.  From Econ 101, usually the first day: "the prime directive -- sunk costs are irrelevant to decision making".  Read Spolsky on that, he had a very intelligent (for a coder) piece about that.

I find it kind of surprising to see this from a database geek; I hear it all the time from outright coders, who will always seek to build the latest LipStick onto some old pig.  Superficial gets the $$$.  Kind of like building a skyscraper on quicksand.

As we represent the Pig, we should always endeavor to build the most robust, ACID-ic datastores; thus XML need not apply.  Robust, ACID-ic datastores will always scale and morph with integrity (the data kind).  File based datastores cannot.

Here endeth the epistle.
January 27, 2009 8:39 AM
 

Jason Haley said:

January 27, 2009 9:15 AM
 

Daman said:

Hi Tony,

I have a few rules that allow me to get though my day and my next 5 years.

Is this change good for the company?  Users will often make requests that are good for them because they are being lazy, don't want to have to think or they want to control the whole process and create a bottleneck in the process.  If it's good for the company, I will work on it.

Should I build or buy?  
If the problem affects the business rules of the company, I will build it if I can.  The business rules will eventually change and a bought product might not be capable of adjusting.  You will end up building your own.  Then you have the added problem of getting your users un-addicted to the cute bells and whistles that the bought product had and have no bearing on the company.  Your only control with bought products is whether or not to upgrade.  Upgrading may implement new restrictions on your business rules you might not want.

Are there other areas in the company that can use this functionality?  This helps me evauluate time and scope for  the project.  If it will be used in other areas, I have to go visit those areas and see if they have any interest and/or reguirements.

I do believe the less code you develop the better.  The less code you write, the less chance of it breaking.  Whenever I am building something and the code starts to grow, I start figuring I am doing it the wrong way.  I am not performing rocket science, just business.  I think this helps me develop as efficient and tight code as I can.

I am a newbie to your site and am enjoying my visits.

Thank You

stan
January 30, 2009 7:34 AM
You need to sign in to comment on this blog
<January 2009>
SuMoTuWeThFrSa
28293031123
45678910
11121314151617
18192021222324
25262728293031
1234567
How to Kill a Company in One Step or Save it in Three
 The majority of companies that suffer a major data loss subsequently go out of business. David Wesley... Read more...

Migrating from OCS 2007 R2 to Lync: Part 4
 Having migrated the rest of our users and legacy resources across, and start getting ready to... Read more...

Automated Script-generation with Powershell and SMO
 In the first of a series of articles on automating the process of building, modifying and copying SQL... Read more...

Seth Godin: Big in the IT Business
 Seth Godin has transformed our understanding of marketing in IT. He invented the concept of 'permission... Read more...

Using SQL Test Database Unit Testing with TeamCity Continuous Integration
 With database applications, the process of test and integration can be frustratingly slow because so... Read more...