Click here to monitor SSC
  • Av rating:
  • Total votes: 150
  • Total comments: 25
Mike Mooney

The Framework Myth

07 April 2011

If ever there was an irresistible programmer’s platitude, it is that code-reuse is invariably a good thing. Unfortunately, even the best of advice can be taken to a ludicrous extreme, the construction of generic frameworks within organisations.  Mike Mooney gives a warning, based on painful experiences.

I was working for a company a few years ago: The company had been around for a while, and had a bunch of genuinely intelligent senior developers working for them.

These developers noticed that a set of common problems arose repeatedly over the course of several years, and there were being solved repeatedly in different ways, at great cost, without reference to the previous solutions. This seemed wasteful since writing data-access code was particularly repetitive and error-prone. Many teams were sharing the same database but using different code, so there were inconsistencies in how similar business data was treated. Each application’s log files were all over the place, and each had their own approach for error handling. Some teams would go off on their own and unnecessarily reinvent existing algorithms and solutions. Moving a developer from one team to another required as much ramp-up time as a new hire. Each team used a different build and versioning strategy, with the most common strategy being “none.” Setting up a test environment with multiple applications took days. Recreating the production environment was virtually impossible. Chaos: Dogs and cats living together: Mass hysteria.

To address these issues, some of the more senior architects took it up themselves to build a framework that would greatly simplify everyone’s life. By putting in a little design upfront, they could build a framework layer that would solve many of the problems that the developers had been muddling through with over the years, while at the same time homogenizing the code base.

Of course, the company had attempted this before. In fact, there were several previous frameworks over the years. But those previous frameworks were not as good as they could be, either because of design flaws, or changes in the way that the company’s applications work, because they used out-dated technology, or because the previous no-longer-with-the-company designers were now generally considered to be idiots. Anyhow, the new architects had learned from these mistakes, and were designing a new framework that would do a much better job of solving the problems. Due to the scope of such a project, it had been a work-in-progress for about a year. Sure, they were still working out some kinks, and it was not completely finalized yet, but this is a technology investment, and some growing pains were to be expected.

Déjà vu

Well, OK, I lied. This wasn’t one company I worked for. It was several different companies, all with the same story. In fact, it’s a little eerie how exactly the same scenario is played out at companies all over the industry.

Some senior developers will have identified some recurring pain points faced by the developers, and they are resolved to do something about it. As the company has grown, more and more developers have come on board, each with less and less experience, and things need to be brought back under control. By providing a framework, you can lay out the boundaries for developers to operating in, which will encourage consistency, will encourage code reuse, and in the end will allow the company to produce higher-quality software in less time with fewer developers: The applications will require less maintenance-cost over time. The company will wish to pursue the single ultimate goal that should be at the centre of every tactical decision, its overall long-term profitability.

It sounds like a brilliant idea. And if it were to be accomplished, it would be great. But the unfortunate truth is that it doesn’t work. Without exception, I have only seen this result in more work for the developers, longer development cycles, more bugs, poorly compromised designs, and (worst of all) excessive, unhealthily conflict between the development teams.

Admit it, you’ve seen it too. Maybe you were even involved.

So What’s The Problem?

So why does this go wrong?

The problem occurs despite the best of intentions. I don’t want to make it sound like the people involved are stupid or malicious; just misguided. In fact, usually they are excellent developers who are trying to do the best they can to solve a problem.

And the problem is not caused by the foot-soldiers of the IT industry down in the trenches, pushing back on every change the framework team wants to introduce. No, these people are trying to get a job done. Their marching orders are not to solve the whole company’s crosscutting problems, but to ship their product on time and in budget, and many of them believe, perhaps rightfully so, that the framework keeps them from doing that as efficiently as they could.

Again, the problem is the approach.

The Challenge Of Frameworks

So what is a framework? Generally, people think of a framework as something that helps you get your job done by providing access to new functionality that you didn’t have before

For example, the Slightly-Almighty, Moderately-Informative, Usually-Reliable Wikipedia says:

A software framework, in computer programming, is an abstraction in which common code providing generic functionality can be selectively overridden or specialized by user code providing specific functionality.

Software frameworks have these distinguishing features that separate them from libraries or normal user applications:

  1. inversion of control – In a framework, unlike in libraries or normal user applications, the overall program’s flow of control is not dictated by the caller, but by the framework.[1]
  2. default behavior – A framework has a default behavior. This default behavior must actually be some useful behavior and not a series of no-ops.
  3. extensibility – A framework can be extended by the user usually by selective overriding or specialized by user code providing specific functionality.
  4. non-modifiable framework code – The framework code, in general, is not allowed to be modified. Users can extend the framework, but not modify its code.

One of the key points here is that the framework is dictating the application flow, rather than the developer who is using it. This is what the Martin Fowler (who literally wrote the book on refactoring) would describe as a Foundation Framework:

A Foundation Framework is … built prior to any application that are built on top of it. The idea is that you analyze the needs of the various applications that need the framework, then you build the framework. Once the framework is complete you then build applications on top of it. The point is that the framework really needs to have a stable API before you start work on the applications, otherwise changes to the framework will be hard to manage due to their knock-on effects with the applications.

While this sounds reasonable in theory, I’ve always seen this work badly in practice. The problem is that it’s very hard to understand the real needs of the framework. As a result the framework ends up with far more capabilities that are really needed. Often its capabilities don’t really match what that the applications really need.

He recommends instead a Harvested Framework:

To build a framework by harvesting, you start by not trying to build a framework, but by building an application. While you build the application you don’t try to develop generic code, but you do work hard to build a well-factored and well designed application.

With one application built you then build another application which has at least some similar needs to the first one. While you do this you pay attention to any duplication between the second and first application. As you find duplication you factor out into a common area, this common area is the proto-framework.

As you develop further applications each one further refines the framework area of the code. During the first couple of applications you’d keep everything in a single code base. After a few rounds of this the framework should begin to stabilize and you can separate out the code bases.

While this sounds harder and less efficient than FoundationFramework it seems to work better in practice.

I’m not sure I would even call this a framework, because all of the things that make it work best are the parts that take it further and further from being a conventional “framework”.

So Are All Frameworks Bad?

Sweet suffering succotash, no. In my humble opinion, the .NET Framework is a thing of beauty. Back in my Win32 C++ days, MFC was not perfect, but worked serviceably well for what it was intended for, namely abstracting away the Win32 API. CMS frameworks like DotNetNuke and Drupal and Joomla have been become very popular. Apparently there is a subset of people who don’t hate Flash with a passion, and apparently those people love it. MVC frameworks like Rails and Django have caught on like wildfire, with ASP.NET MVC picking up a lot of momentum as well. Microsoft Azure and Google AppEngine are in the process of changing how we will build scalable cloud-based applications into the next decade.

Have you noticed a pattern here? None of them were built by you or anyone you know. They were not built to solve a business need, they were built to reinvent a platform. They were not built to get everyone using a “Customer” object the same way, they were build to make it easier for you to do whatever you want with whatever data you need. They were not built by 3 architects for 20 developers, they were built by 30 or 300 architects for 20,000 or 200,000 developers or more. They were not designed and built and delivered and completed in a few months, they were talked design and dog-fooded and tested and tweaked and redesigned over years by some of the smartest computer science experts in the business. Any yet, despite all that, most of them still sucked, and the ones we use today are the select few that survived.

The thing is this: you and your internal development team of architects are not going to build the next great framework. You’re not going to build a good one. You’re not even going to build an acceptable one.

And the other thing is this: If a framework is not great, it is awful, counterproductive, and destructive. There is no middle road.

Get In Line

By definition, most frameworks try to define a new way for your developers to develop software. The keep you from doing certain things that have been seen as problematic, and require you do things the “right way”, assuming of course that the architects have actually thought through the right way to do things.

The problem is that there are plenty of good ways already, ways that those developers are already trained in and have spend years mastering, and you are not really as clever as you think you are. You can’t think of everything. Even if you could, you can’t design for everything. And even if you could, you shouldn’t. Trying to shoehorn them into an incomplete, shoddy, and unnecessarily restrictive framework will only breed resentment, at which point you are bleeding money that you’ll never see on an expense report. The productivity difference between a happy developer and a disgruntled developer is enormous, and constantly underestimated. You will also alienate and drive away all of your good developers, leaving you only with the not-so-great developers that really don’t have any better options

Atlas Shrugged

In order to create a framework, you are taking on a massive responsibility. It’s not a case of adding a few features. You are building an entirely new layer of abstraction. In doing so, it is your responsibility to ensure that your framework provides the developer with every possible thing he will need, otherwise he will be stuck. If you create a data access framework, but never quite could figure out how to get BLOBs working, you’re really leaving the developer up a creek when he needs to store BLOBs. Sure, it’s a growth process, and there are always bugs to be fixed and features to be added, but when you are forcing a development team to use this framework, and in the 11th hour they realize it doesn’t have a critical feature that they need, you are introducing more obstacles then you are removing.

But We Have To Create Reusable Code!

In my early years, I was dogmatically taught never to write the same code twice. Anytime you see duplicated code, or even similar code, refactor it into a function, right? What if the code was slightly different? Create some parameters. What if you need one extra field in some cases? Add a little logic around that to decide what to return. What if you run into a case where you can’t fix one caller without breaking another caller? Add another layer of abstraction and logical branching to your function. At this point, you started out trying to improve your code-hygiene a little bit, and all you’ve done is spend the whole day creating a monstrosity that is almost impossible to understand. Well sure, you don’t think so, but hey, nobody likes to admit their kids are ugly.

Always, always, always, stop and ask yourself, “Am I creating more problems them I am solving?”

Keep It Simple!

The fact of the matter is, when you try to create reusable code, you are, instead, making things a lot more complicated. It’s rarely a case of just taking what you have already written and exposing it to another component or team.

If you are using the code for yourself, you just have to make sure it works for you based on how you are going to use it, and you should be able to clearly visualize all of the possible scenarios. However, in order to make the code reusable within a framework, you are creating a black box to be called by another developer; as such you need to make it work no matter what, with any sort of crazy input, and you will need to make sure that it is clearly reporting back to the caller when something goes wrong. You need to make sure that the interface makes sense to everyone, not just you. You may even need to write up some documentation (ugh). And when you are all done and have covered every base, someone will call your component in a way you never expected, and of course it won’t work because all of the code we write is rife with bugs, and everyone will argue about how this component “should” be used, and someone will demand to see the unit test coverage stats, and someone else will schedule a meeting to define the scope, and someone else ramble on about how we need a framework, and the manager rolls his eyes and swears never to let these damn developers try to build something fancy again, and nobody is actually getting any real work done.

Worser Is More Gooder

Of course, some will argue that you should always try to make your components as reusable as possible, because it requires you to work through all of those quality issues no matter what, and will produce better software. This would be great if your sole goal was to create the best software in the world, but nobody is paying you to do that. No, you are being paid to write software of sufficient quality within the time and budget allotted. If you spend unnecessarily time gold-plating your code, it may make you feel cool, but it is outright wasteful. I’m not saying to cut corners when you should not, but I what I AM saying is that there definitely are times when it actually does make sense to cut comers. You need to draw a line in the sand of exactly how good this product really needs to be and stick to that, otherwise you’ll never finish. You can software that is really cool, or you can build software that ships. Only one of those options is actually useful to society.

But, But, But who is going to test this monster...

OK, so don’t make something reusable if you done need to. However, what if you think you are going to need it to be reusable? I have a secret to tell you: YAGNI.

I can’t tell you how many times I’ve been working on a very specific implementation to solve a very specific problem, and somebody (who usually has initials in his job title) chimes in saying “well, make this and that and the other thing configurable, so when we sell some other product to some other company, we can include it just by flipping a few switches.” This person is figuring that “it’s just an ‘if’ statement, what’s the big deal?”, but that person is not taking into account the increased code complexity and the exponential increase in testing requirements. Often times, this is even a former developer, who actually knows how code works, but has been out of the trenches for long enough that it has impaired their judgment about code complexity and its day-to-day impacts.

The easiest code
 to refactor is the
 code that doesn’t
 yet exist.

But then again, math solves everything. Say your QA department originally had 143 test scripts to run through before release, and now you’ve added some new configurable switch into the system. If you really want to be thorough, you should now have 286 test scripts, because you now have to retest everything with the switch turned on, and then again with the switch turned off. Say you had actually added to different switches? 572 test scripts. That’s a lot of work for a feature that will never get used. And come on, between you and me and the desk, we all know that Mr. Executive SVP COO is never going to close those magical deals with XYZ Bank and ABC Manufacturing, partly because he has no idea what he’s selling, but also partly because you guys can’t ship anything because you now have 1144 test scripts to run through to get a small release out the door.

So How Do I Know What To Reuse?

If you aren’t going stand around and guess what to reuse, how we will know? Easy, stop trying to prematurely reuse anything. Instead, refactor your existing code to reuse that which is already duplicated, where you can reasonably say will actually benefit from reuse and reduce overall maintenance and complexity.

How do you know when that is? Use that fancy experience and judgment of yours, it’s the only thing that separates you from the robots that will be doing your job some day.

Never build something to
be reused until you actually
have two systems that
actually need to reuse it
right now.

Never build something to be reused until you actually have two systems that actually need to reuse it right now. If you only have one thing that will use it right now, and that other mythical feature will also use it some day, deal with it when that feature comes along (hint: never). Even if that feature comes along, the changes that you actually predicted and designed for it correctly is approximately zero, so you’re still going to have to make design changes anyway. However, if you kept it lean and simple, it’s simply a matter of adding the new code you need to a small lightweight base. But if you tried to build something big and fancy and configurable, you’ll now have to spend twice as much time disassembling all the unnecessarily complicated junk you build around it to support the thing that you were unsuccessfully trying to support in the first place. The easiest code to refactor is the code that doesn’t yet exist.

So What?

So do we give up? Maybe. It depends.

So how do you boil the ocean? There are two answers:

  1. You don’t. It’s too big of a problem.
  2. One pot at a time.

It all depends on your goal. Is it critically important that you actually boil the entire ocean, or do you benefit from every little bit?

Ask yourself, do you REALLY need a framework? Do you REALLY have a problem here that needs to be solved? Do you REALLY think you will save your company time and money by pursuing this? Be honest. Try to be objective. If you find yourself getting the slightest bit excited about the idea of building a framework, then back off; you are not thinking clearly.

Sure, a well-design framework may save time and money once it is complete, but it may never be complete, and it may never be any good, and the small improvement may not save your company enough to justify the huge expense. As awful as it may seem, the honest answer may be that it is in your company’s best interest to plough ahead with the same busted up junk you’ve had all along. It may not be the most rewarding thing in the world, but you are probably not getting paid to fulfil your dreams, you are getting paid to write the damn code.

Keep it simple.
Keep it low-impact.
Keep it clean.
Put the big guns away.

So what do we do? Are we doomed to mediocrity? Not necessarily. The other option is to get your head out of the clouds and solve a few small problems at a time. Keep your eye out for redundancies and duplicated code, make note of them, but don’t do anything right away. When you are building a new component, don’t pull your hair out if it slightly resembles some other code, you don’t have to reuse use everything. Once you’ve identified a few low-hanging redundancies, go back and build some small libraries to consolidate that code. Don’t think big picture. Keep it simple. Keep it low-impact. Keep it clean. Put the big guns away. Keep constantly refactoring to make it a little better every day, and before you know it you’ll have system that doesn’t completely suck.

Too much legacy code never gets cleaned up because everyone thinks it is too hard to throw it all out and rewrite it, and no project manager will allow the developers to waste time refactoring code that already works. They are all probably right. A huge refactoring project is probably a waste of money, and it will almost certainly fail. But small, steady, incremental improvements will almost certainly make your world a better place.

Mike Mooney

Author profile:

Mike Mooney has been a software developer for 15 years, and lives in Chester Springs, PA with his wife and three sons, and co-founder and President of MMDB Solutions(http://www.mmdbsolutions.com) and his blog is located at http://mooneyblog.mmdbsolutions.com

Search for other articles by Mike Mooney

Rate this article:   Avg rating: from a total of 150 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.


Subject: An Excellent Post!
Posted by: madjack (view profile)
Posted on: Tuesday, April 12, 2011 at 12:02 PM
Message: Am immersed in a big project now that's tied in knots by an overbearing framework.

Perhaps it's worst offense is, in trying to "save" any developer from having to think or learn about the 3rd party APIs being used, developers who have been on the project for years (yes, years!) know little more today about how to code with these APIs than they did on day one.

(This is another demonstration of the "Mushroom approach" to managing people - keep 'em in the dark and feed 'em fecal matter.)

Command-and-control software cultures don't work any better than command-and-control economies. Yet, we never seem to learn our lesson.

Thank you for sharing this.

jack

Subject: THANKS
Posted by: Elliott O'Hara (not signed in)
Posted on: Tuesday, April 12, 2011 at 2:45 PM
Message: Brilliant Article, thank you!

Subject: Love this article.
Posted by: Dan Szabo (not signed in)
Posted on: Tuesday, April 12, 2011 at 6:45 PM
Message: Long have I said that the principles of good software design are rarely in alignment with good business practices. I have seen developers that routinely blow deadlines in order to write more elegant code. I have seen projects get crippled by premature optimization. I have seen code-reuse break entire branches of multi-million dollar systems because someone thought consolidating 2 "almost similar" functions into a single function was a good idea.

This article should be the first thing college grads read when they enter the workforce. Theory is good, but action is what sells. Good now is better than perfect never. What do you call a "less-than-optimized" shipped product? --A shipped product.

Thanks for providing this to the community! Great read!


Subject: Platform reuse
Posted by: Sarat Atluri (not signed in)
Posted on: Tuesday, April 12, 2011 at 6:57 PM
Message: Can't agree more.

Half a decade ago i would have thought about writing my own enterprise framework. But these days, with the increased functionality of the application servers/containers and their rapid release cycles, I would rather reuse the default platform framework.

Spending lot of time on reusable framework should be dealt with caution, as the next shipment of the product that we use may already address the issues. Another possible issue is that the assumptions we make while designing the framework may not be valid anymore with the upgrades we make to the products we use.


Subject: Great article overall
Posted by: Mohammad (view profile)
Posted on: Wednesday, April 13, 2011 at 12:55 PM
Message: best line:

"And the other thing is this: If a framework is not great, it is awful, counterproductive, and destructive. There is no middle road."

Often people get married to and idea that they have invested awful amount of time and money on, not realizing how it slows them down constantly because it is not a GREAT idea.

I Know my custom coded javascript libraries are now in my file archives because JQuery and other libraries out there are doing the same thing, only 10 times better.




Subject: Sour Grapes
Posted by: Derek (not signed in)
Posted on: Wednesday, April 13, 2011 at 10:53 PM
Message: Wow... You really come off sounding like an old curmudgeon: "I know how to do this on my own... Don't get in my way you dang nabbed kids!"

Also, this article is extremely schizophrenic. On the one hand you rail against the scourge of the framework and then you have the gaul to hold up MFC (MFC!) as an example of something good!?!?

You go on to say that frameworks are terrible (except when they're awesome!).

It sounds to me like you ran into a few overgrown frameworks projects that didn't work out and you needed to rant about that. Really sorry that's happened to you... but don't start the revolution just yet.

The reality is that systems are always getting more complex... and as that complexity grows we invent new ways to shield ourselves from that complexity so we can continue to innovate. This has happened in all area of computing (and each time there have been old-timers standing around saying "bah your new layers just ADD to the complexity!") and will continue to happen until the end of time.

I'm sure no one would argue that device driver layers in OS's are a godsend these days... but they were once controversial ("just give me a raw serial interface... I'll write to it directly!")... Or that GUI frameworks (like your beloved .Net or even Qt) are necessary and save time.

Indeed, even the web browser I'm typing this in is a framework... you honor the interface the browser wants (it asks you for something like HTML) and it will display it for you. MANY different things can be accomplished by conforming this "framework"... and only a madman would want to go back to the client-server world we lived in before where everyone wrote their own GUI front end and talked to a specialized back end to retrieve and display information.

The list goes on and on.

I personally work in massively parallel scientific computing... where, at this point, there is simply no way to efficiently develop a new application from top to bottom. The machines are too complex; one man (or even team) can't hope to understand all the math, computer science and engineering to do it all themselves; and by working together and and on common frameworks we can not only advance the state of the art: we can also achieve new never before possible things by having interoperable codes.

Just as in other areas of computing there are good and bad frameworks... and new ones spring up all the time. But that does NOT mean that the idea of a framework is flawed... only that progress continues! The good frameworks go on to be core pieces upon which later frameworks stand upon and as things move on we are able to achieve things never thought possible by the previous generation (not only not possible... but even TRYING would be a waste of time!).

Anyway... there is a lot more I could say on the subject (for instance I haven't even mentioned that the quality of every developer is not equal... and sometimes some "guidance" and sandboxing is a good idea)... But for now I will just say: lighten up on the framework! Today's framework is tomorrow's core functionality that will drive us into the next round of innovation...

Subject: Umm. may be not
Posted by: BuggyFunBunny (view profile)
Posted on: Thursday, April 14, 2011 at 11:40 AM
Message: @Derek:

I read the article as saying: "unless you've got oodles of time and talent, making a bespoke framework is waste of both".

I once worked with a guy who "built" a framework by stripping out a few bits of Struts, and putting his name on what was left. He should probably go to jail, but his colleagues were copying java source files and doing the same thing, so there's not much likelihood of anyone dropping a dime on anyone else.

I've been noting since PaperClips and Tea (and that's quite a while ago), that, at least in the java/web world, no one writes java anymore. They write "framework". Whether that's a good or bad thing is certainly up for debate.

Should we all return to C/Assembler? Likely not. Good frameworks, and most aren't, allow one to drop out of the strait jacket of the framework without much punishment. RoR is notoriously bad in that respect.

As this is a database site, frameworks that leverage the RM (and there aren't many, unfortunately) are progress. Language driven frameworks, not so much.

Subject: Reuse can work and provide value
Posted by: Nick (view profile)
Posted on: Thursday, April 14, 2011 at 1:35 PM
Message: There are some babies being thrown out with bathwater here. The developers at my company successfully figured out how to reuse code, and some of that shared code adds up to what you could call "frameworks". One of them provides a single implementation of data access.

The value proposition is undeniable: There are certain types of critical code that need to be implemented once and shared across products, where variation in design and quality would otherwise be detrimental.

It's true that 99% of such attempts fail in software, mainly because they're driven by developer annoyance or ego. But manufacturing industries do it every day and for good reason. You wouldn't set foot in a building where all the load-bearing members were hand-crafted by different people.

Our engineers came from various companies that had tried and failed at this, so they learned some useful lessons that made our effort work, such as:

- Don't try it if you're a startup. You need to be an organization of at least several teams with several related products for the effort to pay off.
- Don't automatically share every line of code that happens to appear twice. Think of shared code as functional components, and only share the important components. (If you step back and look at the product set as a whole, it's easy to spot them.)
- Learn (organizationally) how to share small components before trying bigger stuff. Maven was helpful here.
- Drive the effort through product management rather than grass-roots from developers, so it has measured business value and future products are designed to rely on the shared components.
- Don't let architects design and build stuff in isolation (in fact, don't even hire ivory-tower types). Everyone who works on a shared component is still accountable to product deadlines and their mods are informed by real experience.
- For small components, don't bottleneck on a single team. Let any development team create or modify a component, like internal open source.
- For framework-like stuff - which is really a set of shared components that provide a larger function - _do_ bottleneck on a single team and have a product owner control it. Otherwise, quality and design consistency will suffer to the point where the framework doesn't provide value in reuse.
- It takes strong and pragmatic engineering management to make all of the above happen.

Subject: open source makes good frameworks too,
Posted by: AlexK (view profile)
Posted on: Thursday, April 14, 2011 at 9:19 PM
Message: And we don't really need that many architects .
xUnit is a good example - AFAIK it did not take 300 architects and 1000 developers to come up with this excellent and wildly popular framework. Come to think of it, one strong architect of the caliber similar to Linus Torvalds is capable of designing a better product than 30 average ones.

I'd like to add that strong exposure to feedback is essential in building good frameworks. Repeating that "we are listening" does not count as strong exposure to feedback; the ability to eliminate weaknesses is a must - otherwise those weaknesses stay and erode the product's worth.

Slow scalar UDFs in SQL Server is a good example of a well known weakness that has been around for a while, stays there, and makes the product weaker. Fast UDFs are so badly needed, and so easy to provide for the simplest cases, that we routinely roll our own ones based on macros - it takes just a few ours to set it up, and the effort pays off very soon. Yet the weakness is still there.

Subject: Advice
Posted by: Anonymous (not signed in)
Posted on: Friday, April 15, 2011 at 2:19 AM
Message: What can be automated will be automated, and will be part of public frameworks.

We have witnessed this through the years, as parts of our framework, became available.

This is normal. Having framework will help you for specified time-frame. After that your framework will become obsolete. Since similar parts of your framework will be available for free, open source, or by other companies like Microsoft, Oracle...

I think we should separate Framework development, from software development. Have a another department, team, or employees maintain it.

Mixing the two it bad Software management.



Subject: What frameworks are - and are not
Posted by: Harald M. (not signed in)
Posted on: Friday, April 15, 2011 at 4:38 AM
Message: Neither the "up-front" nor the "harvesting" idea are the really important things - and therefore, both may (and will in many cases) fail; or may not.

The fundamental concept are *intentions*. As engineers (code designers, developers, architects, ...), we need to "want something". You "can want":

* that something is harmonized (all our DB tables have a column for optimistic locking versioning; all our GUIs close on pressing ESC, but before that ask whether you want to save; there *is* such a concept as "save"; etc.)

* that something is not harmonized

Many such intentions cannot be put into code (like the concept of "save"); people on the project must be taught to understand this - or, better, it must be discussed with them every time it is unclear (it might turn out sub-optimal at some point of time, after all).

But many such intentions *can* be put into code - often code generators or the like. And that is *easier* than telling people and reviewing and finding violations.

But there are two questions: (1) Why would ou harmonize at all? (2) Whatever rule you make, it will certainly bad at some place. How do you deal with that?

Ad (1): Mostly, because we can then repair things much easier. All of our DB layer uses GUID columns for PKs; a generator does that, and it was a bad decision. But the tools we now use to change that situation can *rely* on the fact that (almost) all tables have consistent names and types for the key columns -> so they are quite easy to write.

Ad (2): You allow exceptions. Always. And, using the rule of 3 (or sometimes a rule of 2 or even 1), you harvest exceptions into the framework - or not! You do that after a sensible discussion in the team on the effects of going one way or the other.

So: The reason for a framework is *not* reuse - that's incidental. It is revelating of intentions ("promises") AND making it as easy as possible to *keep* these promises.


Subject: Perfect
Posted by: Dimitrios Kalemis (not signed in)
Posted on: Friday, April 15, 2011 at 4:49 AM
Message: This is the most beautiful thing I have ever read.

Subject: Framework design
Posted by: Chris Boss (not signed in)
Posted on: Friday, April 15, 2011 at 7:54 AM
Message: As a developer of tools for programmers, my job is to design a framework which all sorts of programmers can use. With my software (a GUI engine with a visual designer front end) in its fifth generation, I have learned the following:

- a framework needs to be a set of building blocks anyone can use
- high level features in a framework need to be there for tasks commonly used, which the average programmer would find hard to write themselves
- low level features should be extensive so users are not limited
- high level engines should allow some form of customization (hooks) for those who need to do low level customizing
- keep it simple
- provide code reusability features as part of the framework
- ease of use is critical

I started my framework design by simply considering the simple question:

What tasks will most programmers most likely need to do ?

The early versions of the framework were very simple in nature and most features were used. Later versions were built upon previous versions adding more sophisticated features in each generation. Backward compatibility was considered a high priority, since new versions of the framework must not break code used with previous versions.

So what does the latest framework offer, after five generations ? Complete GUI design (forms/controls), WYSIWYG (drag and drop) engine (build your own visual designer with it), graphics engine with 2D Sprite engine and 3D OpenGL engine, ownerdraw and customdraw, thread engine and subclassing engine, superclassing engine, custom control engine and component engine.

Framework design is an art and a skill.


Subject: Too Many Typos
Posted by: Unregged (not signed in)
Posted on: Saturday, April 16, 2011 at 2:51 AM
Message: This article has to many typo's I can barely dijest is in a single swoop. *

(*) Typos were intentionally added.

Subject: Too Many Typos
Posted by: Unregged (not signed in)
Posted on: Saturday, April 16, 2011 at 2:56 AM
Message: This article has to many typo's I can barely dijest is in a single swoop. *

(*) Typos were intentionally added.

Subject: Using frameworks...
Posted by: markfaction (not signed in)
Posted on: Saturday, April 16, 2011 at 8:55 AM
Message: Good article, and interesting facts. In an agile context, the YAGNI rule and neglecting current changes to incorporate future changes can be viewed positively. But i have seen many in-house and external frameworks being used in projects which have added value to the business. If the framework can cater and extend to future changes, and if it can speed up development and the time to market in shipping the product, its a good bet that this is what you need. Making smart decisions on investing in such a framework, and learning from the mistakes in developing it, would pay off well in the end. It all comes down to good judgement, and smart decisions. Some experience with framework design and usage wouldn't hurt ;)

Subject: April Fools Day ?
Posted by: Geoff (view profile)
Posted on: Monday, April 18, 2011 at 2:30 AM
Message: HaHa, but you were 6 days too late.

Seriously, you've taken a big stick to the extreme end of bad programming. Everything you call bad could be said about programming in general.
In my (30+ years) experience, the idea for a framework has been born out of necessities and its implementation has evolved over time, moving logic from an application base into the framework as required. We build & support customised software solutions. We are a small team, but over the years we have probably built upwards of a hundred different systems (small, medium and enterprise). They all have our unique (well we like to think so) look and feel and they inherit this from our frameworks. Yes we have more than one, we have supported several platforms.
There's lots of different ways of accessing data and Im sure our programmers either know them or could figure out the lines of code required, but they dont have to, they just use the most appropriate method (from the framework) for the task at hand. Methods that have stood the test of time, been through unit testing and simplify the menial tasks allowing them to concentrate on what they are paid for, using their nouse to solve REAL problems. Just BTW, we use a slightly customised version of the Repository model from the Subsonic framework at the base of our .Net data access (praise be to Rob Conery).

As for needing 'oodles of time and talent' to produce a framework, I see it the exact opposite. If you build lots of solutions and you dont have a framework, you will require more of both.

Just my take. Thanks for reading.

Subject: Emjoyed this
Posted by: Al Taylor (not signed in)
Posted on: Monday, April 18, 2011 at 5:46 AM
Message: Well thought out and well written - I enjoyed this.

Subject: Mainly good
Posted by: Bill Nicolich (not signed in)
Posted on: Monday, April 18, 2011 at 6:56 AM
Message: Yeah - think twice before you go off and build a framework. True.

Mike: You've got to love the comments you get from the tech community.

Some computer science guy working on "massively parallel" systems tells you that your generalizations don't match his uncommon, one-off scenario.

One can't cover all caveats, exceptions and one-offs. However, you'd do well to anticipate the contrarian voices and let them whisper to you over your shoulder as you write - if you don't already.

Also as you write, imagine a guy constantly urging you to make everything concise.

Thanks for sharing!


Subject: favorite quotes...
Posted by: Harveysburger (view profile)
Posted on: Monday, April 18, 2011 at 7:45 AM
Message: "You’re not going to build a good one. You’re not even going to build an acceptable one."

"Well sure, you don’t think so, but hey, nobody likes to admit their kids are ugly."


good job with the article :)

Subject: Great Read!
Posted by: jjjallen (not signed in)
Posted on: Monday, April 18, 2011 at 8:39 AM
Message: Reminds me of all the VFP (Visual Foxpro) frameworks I had to endure once upon a time in my career - all of the programmers detested them but to the (unknowing?) management teams they were the greatest thing since sliced bread.

Thanks
jja

Subject: consider musical frameworks
Posted by: music_lover (not signed in)
Posted on: Monday, April 18, 2011 at 11:02 AM
Message: I couldn't live without many essential software frameworks I've used, mainly from vendors. The good ones are stable, the better are also extensible, and the best are also elegant.

Do musical frameworks have any bearing on the discussion? They arise organically and informally over time -- notes on the scale, notation, musical instruments, ensembles from solo instrument to orchestra, forms such as sonata or concerto... They gel and get reused, refined, and reinvented. Someone bold replaces the minuet movement with the scherzo... One invention appears for a single concert and disappears. Another provides a foundation for centuries of creativity.

Subject: About getting exciting about the next great framework ...
Posted by: Simon Laub (not signed in)
Posted on: Friday, April 22, 2011 at 4:52 AM
Message: [If you find yourself getting the slightest bit excited about the idea of building a framework, then back off; you are not thinking clearly.]

LOL ... You are so right !

Your arguments about the QA departments new testing responsibilities are right on the money !

-Simon
Terminators on Windows:
http://www.simonlaub.net/Post/Arnold.txt

Subject: Architecture and Frameworks
Posted by: rruge (view profile)
Posted on: Monday, May 09, 2011 at 8:53 AM
Message: Great article that rings many bells for me. Thanks Mike.

@Derek
Mike never said MFC was the shizzle. He DOES however say, "MFC was not perfect, but worked serviceably well" and I have to agree given what was available at the time.

One thing I noticed with MFC [all types of...] developers back then was that if they didn't take the time to understand it, then they would typically hork up the software making the overall experience with that framework even worse.

...which brings me to another development hurdle: getting developers to first think about the current architecture BEFORE they make refactoring changes that impact the original design.

Subject: It's so true that it hurts a lot of Frameowrk Architects badly
Posted by: bdjc (view profile)
Posted on: Saturday, July 23, 2011 at 11:59 PM
Message: I've been working in an internal frameworks team for several years at a mid-sized company, and almost everything said in this article is true. I'm sure the architects will get hurt real badly, it's like their deepest, most damning and best kept secret was suddenly revealed to the public. I can almost hear our 50-year-old framework team leader screaming: "Damn you, i hate this guy, if our CTO/CIO read this article, my entire career would go down the drain. Why do you rock the boat? we architects have been conspiring to make a good living out of our frameworks for so many years, our job security is now seriously jeopardized! With this kind of evidence in hand, the application teams are going to cut our throat, I better find a rope and hang myself."


 

Top Rated

We don't need Source Control: we're Database Developers
 As part of our long-running series of articles where we ask working database developers how database... Read more...

The Proposals Conundrum
 When you work for a small software development (or any services) company, one of the major challenges... Read more...

David Heinemeier Hansson: Geek of the Week
 Ruby on Rails, the open-source web application framework, grew out of David Heinemeier Hansson's work... Read more...

Alex Payne: Big in the IT Business
 Alex Payne worked on developing Twitter for three years. When he started, it was a small side-project:... Read more...

Seth Godin: Big in the IT Business
 Seth Godin has transformed our understanding of marketing in IT. He invented the concept of 'permission... Read more...

Most Viewed

The Future of Reflector
 Simple Talk asked freelance writer Bob Cramblitt to sit down with the two people behind the agreement... Read more...

Linus Torvalds, Geek of the Week
 Linus Torvalds is remarkable, not only for being the technical genius who wrote Linux, but for then... Read more...

Bad CaRMa
 From hope and euphoria, to desperation, firings and the ultimate demise of a company. Tim Gorman charts... Read more...

Driving up software quality - the role of the tester
 Have you ever wondered what a software tester does? Helen Joyce, test engineer at Red Gate software... Read more...

Don Knuth and the Art of Computer Programming: The Interview
 Fifty years after starting the 'Art of Computer Programming', (TAOCP), Don Knuth is still working hard ... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.