Lessons Learned from Six Years of Agile Database Development

Alex Kuznetsov describes the agile principles, techniques and tools that allowed his development team to make frequent database refactoring a reality, without disrupting users. He explains how this allowed his team to apply an iterative, evolutionary approach to the design and development of their databases, as well as applications.

As a database professional, are you curious about agile methodology? Are you considering joining one of the growing number of agile IT shops?

For six years, I’ve worked in a development team that has learned, gradually, how to apply agile techniques successfully to our development, including our database development, which we treat in more or less the same way as developing classes, libraries, and the GUI. We develop in-house solutions in a dynamic environment, which means that business requirements change quite quickly, and we need to develop or change systems quickly, in response. We find that adopting an agile methodology allows us to:

  • Communicate with users very efficiently, so that we can quickly understand what our users want, and we do not waste our time developing things nobody wants.
  • Develop well designed solutions that are relatively easy to change.
  • Develop low maintenance solutions, so that our development is rarely interrupted by maintenance tasks.

As a result, we can deliver working and useful solutions, quickly. In this article, I’ll describe the agile principles that are important to the team, knowledge of which will help you, as a database professional, to collaborate better with agile developers. I’ll also outline a few of the most valuable techniques and tools that help make these principles a practical reality, for databases. I don’t have time, in a single article to offer much detail, but I hope to at least provide a useful framework for further research, should your team be considering adopting an agile approach, or extending your current approach to cover databases as well as applications.

One proviso, before we begin: I speak from extensive, though rather narrow experience. I am not a consultant, I do not get to see multiple environments and teams. I describe what has worked well for my own team, but I do not try to be prescriptive in my recommendations, as all teams work differently. Agile is a grassroots movement and one simply cannot force specific tools, techniques or approaches upon an unwilling team. In this spirit, I encourage you to take the bits of my advice that sound a good fit for your environment, and reject the rest.

Key Agile Principles for Database Professionals

Different agile teams work differently, using various tools and approaches; this is perfectly normal for a grassroots movement. Still, there are some common values to which we all subscribe. This section describes those overarching principles of agile development that are very important to our teams, and are common to different teams. I will only mention here the ones that seem to be the most relevant to database professionals, but I encourage you to read Principles behind the Agile Manifesto for a full list.

Welcome Changing Requirements

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

We build in-house solutions in a dynamic environment, where requirements can change quickly. To justify the cost of in-house development, our overriding goal is to provide our users with the simplest solution that will fulfill each particular requirement. We spend a lot of time getting to know our users, communicating with them frequently, and striving to develop only the features they actually need, and to develop them really well. By building simple, easy to use and very well-tested software, customized to our users’ needs, we boost their productivity.

In a dynamic environment, our team, and the solutions we build, need to be able to respond quickly to the changing requirements of the business and of our users. In order to succeed in such an environment, we cannot spend months developing a single feature, only to find that the user’s requirements changed in the meantime. Agile development encourages teams to “deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.” In fact, our team sometimes delivers a working product or feature in a day or two, and then we iterate until we reach the best solution for the users.

Each time we deliver, we solicit direct feedback from the users and it is this feedback that drives the next iteration. This feedback loop is essential and the only way to ensure that at each iteration, we produce a solution that is more efficient to work with than the previous version.

Sounds obvious? Unfortunately, quite often the next version of a piece of software does not increase users’ efficiency, and the main reason for this is that the imagination and ambitions of the developers is the main driving force for change, rather than the needs and the feedback of users. This is a not a problem specific to software development.

The Design of Everyday Things

I highly recommend you pick up a copy of Donald E. Norman’s excellent book. It is full of excellent examples of the results of the arrogance of designers and architects, and an unwillingness to accommodate the real needs of users. It includes, memorably, an example of a family home that won a design award, but was not actually human-friendly enough to live in comfortably.

Establish Efficient Communication

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Agile teams invest a lot of effort in establishing clear and efficient means of communication. We use scrum, stand-ups, and so on, to encourage developers and users to get together and talk to each other. The main reason is simple: we do not want to misunderstand requirements and waste precious time developing wrong functionality.

Agile Principles, Patterns, and Practices in C#

Robert C. Martin’s book describes in detail many communication-improving techniques for agile development teams

Even with frequent face-to-face communication, there is a danger that spoken requirements are misunderstood and morph into something subtly or even completely different when implemented in software. This is a particular danger when working in diverse teams, from different countries and with different native languages. To give a simple example, if someone tells me “this must be done by Friday”, as a Russian I could interpret this as “no later than 11:59 PM on Thursday” (I’ve made this mistake before). However, another member of the team may interpret is “by end of business day on Friday”. Furthermore, in organizations with different geographic locations, to which time zone does the deadline apply?

Such potential confusion is the reason why agile developers are big proponents of translating spoken requirements directly into tests that run automatically. To continue our previous example, we’d translate the requirement that a task must be complete “by Friday” into a test case that specified exactly the date and time for task completion, including the time zone, so eliminating all ambiguity.

Implementing user requirements as automated tests

For further details on how to do this, I would recommend the following resources (with the upfront disclaimer that almost all the authors are either current or former coworkers of mine):

  • Behavior-Driven Development, by Dan North, on dannorth.net
  • The Cucumber Book: Behavior-Driven Development for Testers and Developers, by Matt Wynne and Aslak Hellesoy
  • The RSpec Book: Behavior Driven Development with Rspec, Cucumber, and Friends

Iterate and Simplify

Simplicity–the art of maximizing the amount of work not done–is essential.

As described, we try to deliver solutions quickly and then iterate frequently, based on direct user feedback, until we arrive at the most effective solution for our users. The beauty of this approach is that we don’t have to strive get it right first time. We focus on simple designs that are adaptable to change and then iterate the design to arrive at the simplest possible solution.

A common objection to agile methodology is that with more time spent on a single design phase, at the start, we are more likely to get it right first time (an approach sometimes termed “Big Design Up Front”, or BDUF). Unfortunately, the BDUF approach does not always work in real life. Even the best designers and developers often need several tries before they get it right. Maybe Wilbur and Orville Wright’s initial designs looked wonderful on paper, but the brothers needed many attempts to build a working airplane.

Likewise, most software developers need the latitude of trial and error, and a few iterations, to solve a problem well. Many solutions, including some of mine, begin life as over-complicated and over-engineered bloat-ware, and eventually simplify, if we have enough time to improve them. This evolution from complex prototypes to very simple, high quality products is quite typical in agile development.

To take a specific example, Linux, which to my mind is a truly amazing piece software, was to the best of my knowledge developed through trial and error. To quote the inventor of Linux, Linus Torvalds:

“Don’t underestimate the power of survival of the fittest. And don’t ever make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence _much_ too much credit.”

When we use agile methodology, we essentially roll out many inexpensive tries, and let the fittest survive. Of course, this approach has direct implications for how we design our systems. A common misconception is that in agile development design “doesn’t matter”. This is simply untrue. I cannot emphasize enough the importance of good design to successful agile development.

High Quality Design and Implementation

Continuous attention to technical excellence and good design enhances agility

Systems must be highly adaptable to change and only well-designed and low maintenance systems can change quickly. Poorly designed ones are brittle and so difficult to change. In dynamic environments, poorly designed systems quickly obsolete because they just cannot change fast enough, so we have to replace them. This process is quite Darwinian: the systems that survive in the long run are not necessarily the ones that perform the best right now. Only the systems that are able to change and adapt in the long run manage to survive.

Later in this article, I’ll describe, more specifically, how all this affects the manner in which we must develop databases. We must invest considerable time and effort to maintain the agility of our database systems, for example, by finding ways to:

  • Make frequent database refactoring a reality, without disrupting users
  • Minimize the database maintenance and troubleshooting that could distracts us from new development
  • Develop an extensive system of automated tests of various kinds to make frequent change sustainable in the long term.

Self-Organizing Teams

The best architectures, requirements, and designs emerge from self-organizing teams.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The need for frequent iteration and adaptability to change means that we naturally tend to build multiple, interconnected, simple solutions, each with a very specific focus, instead of consolidated systems. Of course, many will argue, correctly, that this leads to redundancy. However, we find that complex systems are less likely to adjust quickly enough in our dynamic environment. As a result, even though consolidation might save some resources in the short term, in the longer term we risk losing the whole system if we fail to change it fast enough. We develop small, tightly focused systems that do just a few things, but do them really well.

This has implications for our organizational structure. There is really no rigid hierarchy in our teams. We do not, for example, have a single team of software architects, with whom all development teams consult. Instead, each team takes responsibility for all aspects of each small, well-defined project. Exactly as specified in Conway’s law, the overall architecture of our systems more or less reflects our organizational structure.

The Starfish and the Spider: The Unstoppable Power of Leaderless Organizations

This is an excellent book that describes the power and robustness of self-organizing teams. It is definitely worth reading.

Much of the decision making is decentralized, which means that most decisions are made at the lowest possible level, by the users who requested a feature and the teams that are implementing it.

As well as making most of the day-to-day decisions, our teams are free to choose the languages, tools, design, and coding style best-suited for the project. Our company has neither common coding standards nor a common technology stack. Different teams use different languages. We build each system such that it does not depend on the implementation details of any other system, so this approach that may seem haphazard to some, works out well for us.

Impact of Agile Principles on Database Development

Over the coming sections, I’ll drill deeper into what these agile principles mean for the way in which we develop our databases. Some of these approaches are “non-standard” and may surprise a few developers considering a move to an agile team.

Automated Database Testing

No agile development effort will succeed in the long term without the automated testing that gives us the confidence that we will spot immediately, and fix, any problems or errors we introduce as a result of our frequent changes.

Likewise, we need to spend time implementing a comprehensive database testing suite. I don’t have room to cover this topic in detail, but will refer you to my Close these Loopholes series of articles, and references therein, describing how to make “watertight” your suite of database unit tests.

Besides unit tests, we use other kinds of automated tests extensively:

  • A comprehensive suite of integration tests verifies that individual modules correctly fit together. Our integration tests do not mock databases; we use the real thing
  • Stress tests verify that our modifications work correctly under high concurrency
  • Performance tests verify that performance meets users’ expectations.

Of course, the principle of evolving and simplifying the software we develop, over many iterations, applies also to the tools we develop to help us do this. Our original library for unit testing T-SQL had many features, including separate checks for the following conditions:

  1. How many rows are there in a result set?
  2. How many result sets are returned?
  3. Is the result set ordered by some columns?
  4. Which value is in some particular cell of some particular result set?
  5. Exactly which tables and rows were touched by a modification?

Through using the tool, we started to realize that this abundance of features did not make us more productive. For example, even though we’d invested time in developing a way to test our database modifications, we learned that that, in practice, we were more productive if we did not explicitly test our data modification stored procedures (see How to Benefit from Unit Testing T-SQL: choosing what not to test for more details). Instead, we began to use these stored procedures to set up test data. We removed the unnecessary feature, explicitly testing modifications, and our next iteration of the testing tool had only one feature:

Verify all the output of some T-SQL, all the result sets and all the values in them, including their order

So, we ended up with a solution does only what we need and is implemented really well. It automates our common tasks to the fullest, and it has no bugs.

Silos instead of Consolidation

As discussed earlier, our teams build small, tightly-focused and highly-adaptable systems. As a natural consequence, we tend not to build large, consolidated systems. Instead, we strive for multiple interconnected systems that communicate via simple interfaces, with each system having no dependence on the implementation details of another system.

This attitude extends to our databases. We do not have one big database that stores all kinds of data for everyone. Instead, we prefer to develop separate systems that communicate via simple interfaces without depending on each other’s implementation details. We would not try to build a system to “absorb” and consolidate individual silos.

When we store our data in silos, we accept the fact that there is considerable redundancy between these silos. However, the reward is that each system remains simple and adaptable and we remove the risk of a single large system becoming obsolete, and failing to satisfy users’ requirements, simply because we cannot change it quickly enough.

As a hypothetical example (I am no expert in addresses and shipping), suppose we have a Shipping application, with a database that currently stores US addresses and other information related to shipping via mail within the USA. We now need to support shipping to a different country, such as Russia.

According to the DRY (don’t-repeat-yourself) principle, the obvious choice is to extend the current database design to begin storing Russian addresses. We avoid repetition and store all addresses in one system. However, is the choice really so straightforward? Currently, we have simple business rules in place to enforce the presentation of US shipping addresses in the format “City, State” (“Medford, WI”, and so on). Typical Russian addresses come in a variety of formats, such as “City” (for very big cities only), or “Province, City” or “Province, District, City”. To accommodate these, and many other differences, will substantially increase the complexity of our current, simple, US Shipping application.

As agile developers, we strive to avoid complexity and all the expensive problems caused by it. Quoting Ray Ozzie:

Complexity kills. Complexity sucks the life out of users, developers and IT. Complexity makes products difficult to plan, build, test and use. Complexity introduces security challenges. Complexity causes administrator frustration.

As a result, we may instead decide to implement a new Russian Shipping application, with a separate database, tightly focused on implementing our Russian business rules. If we need an aggregated view of all the company’s shipping data, we develop that as a separate system as well.

We will definitely be paying a price for redundancy, and for establishing communication between these separate systems, but this price may be lower than the price of maintaining one “monster” system that becomes more difficult to maintain, and almost impossible to change quickly, the bigger it grows.

To put it another way, if we build cars for a country that drives on the right, we put the driver’s seat on the left. If we start to supply cars to a country that drives on the left, we build a new car with the driver’s seat on the right. We do not try to develop a car where we can move the driver’s seat from right to left; such a car is likely to be both expensive and unreliable.

Frequent Database Refactoring

In previous sections, I discussed the need for a development process that allows software to evolve and simplify over many iterations. While this is an increasingly popular approach for application development, many still object to the idea of evolving database structure, stating that iterating it is too expensive to be feasible.

While the cost to the organization, and users, of changing the table structure remains prohibitively high then these objections are quite valid. Each time we have to shut down the system, migrate the data to the new structure, and re-deploy all the applications that depend on the database, all at the same time, requires downtime for our clients, overtime for us and our DBAs, and carries a high risk of something going wrong, leading to several cycles of expensive bug fixing. Everybody wants to avoid this sort of major surgery unless it is absolutely necessary.

Changing the structure of database tables is neither cheap nor easy, no matter how we do it, especially when the amount of data being restructured is large or users need constant access to the data. This is why many people recommend spending time up front to get the database structure right, to minimize the need for future change to database design, even if you allow the rest of the system to evolve over time.

The problem I see with applying “BDUF” to databases is a strong tendency to over-engineer table structures in attempting to anticipate every possible requirement. In most cases, such efforts end in defeat, as the requirements of the users change in ways the designer simply cannot anticipate. When this happens, not only is our “future-proof” design obsolete regardless of our efforts, but the resulting schema is overly-complicated and therefore harder to change.

As an example, let’s consider a small and ambitious e-business company that, some years ago, needed a system that would store all its customers’ details in an RDBMS database. It is currently a US-only company, and uses a ten-digit phone number to uniquely identify each customer, but has ambitious plans to expand its operations to other countries. They expect aggressive growth in the number of customers and need the database to be available 24×7, except maybe a few major holidays.

In designing the parent and child tables to store all the customer’s information, we might anticipate the need to, for example:

  • Support country codes
  • Validate phone numbers for different countries
  • Support phone number extensions
  • Allow several family members have separate accounts associated with the same landline phone number
  • Distinguish phones and faxes

The team puts in considerable time and effort to make sure that the database design meets all these possible requirements. As a result, the product launch is delayed and other solutions appear on the market, in the meantime. This leads to profit and opportunity loss.

To add insult to injury, soon after our system is fully operational, the world around us changes in an unanticipated way: everybody else use emails to uniquely identify customers, and we need to catch up, and quickly. However, this is not necessarily straightforward because it will take time and effort to change our complicated design. We also have the time and effort wasted, developing features that have never been used.

So, although we have worked very hard, we did not actually manage to avoid restructuring our tables.

97 things every software architect needs to know

In this book, Chad LaVigne expresses very well the futility of trying to guess what might be needed in future: “When you try to guess at future requirements, 50% of the time you’re wrong and 49% of the time you’re very, very wrong.”

In our experience, trying to future-proof database design and accommodate for all possible future changes is usually counter-productive. Instead, when the environment is very dynamic and unanticipated change is very likely, we should design and develop databases defensively, so that we can refactor them with confidence, at a reasonable price and with low risk.

For a start, instead of trying to anticipate every requirement, we implement a database design that supports only what is required right now i.e. customer identification via 10-digit phone numbers. Then, we focus on making the schema resilient to change. We insulate all of our databases behind a well-defined interface. We do not expose tables and views to our users and instead provide an API of stored procedures. This allows us to change the procedures or the underlying tables without the risk of breaking applications. We also employ defensive design techniques that enable change. For example, we can add an identity column, and have all child tables refer to it, so that if we need to switch from phone numbers to some other customer identifier, there will be less changes in child tables. We then cover the whole system with a comprehensive suite of automated tests, so that we can change it with confidence when needed.

Also, in order to make future changes easier and cheaper, we invest time and effort in automated tests, as we discussed previously.

Refactoring database tables to keep up with changing requirements is still not cheap, but it may be substantially cheaper that trying to design up front for all possible changes in the future. In fact, in our experience, database refactoring is not prohibitively expensive at all. Our team can refactor live OLTP tables with no downtime for our users, no overtime for us and our DBAs, and with little risk. For further details on how we achieve this, please refer to the section, How to refactor a table without downtime in the article, Developing Low Maintenance Databases.

We have done it many times, always successfully, and this experience makes us very confident that, if needed, we can refactor database tables.

Emphasis on Minimizing Maintenance

If our database systems require a lot of maintenance, it distracts us from new development. It is unrealistic to expect us to quickly develop new features while we have to troubleshoot deadlocks, speed up slow queries, and such, all at the same time.

As such, keeping our database system low-maintenance is essential. The previously-referenced article on developing low maintenance databases also contains sections on other techniques that we use to keep maintenance costs low, including techniques to remove deadlocks, extensive use of constraints to ensure data integrity, and so on.

All these defensive approaches require some initial investment in time and effort, so that we do not lose precious time later, on troubleshooting.

Less emphasis on Complex Queries and Query Tuning

In dynamic environments, we do not know for how long we will require any particular module. A query that is essential today may soon become obsolete. Every time we change substantially, or just throw away, a query, so we also throw away all the effort spent tuning it.

This is why we do not want to spend too much time tuning queries. Of course, we employ as many sounds database programming techniques as possible to ensure we do not “kill” database performance. For example, we prefer to save and retrieve sets of rows, we use proper indexes and write index-friendly queries, and we do not reuse selects wrapped in stored procedures and scalar UDFs, and so on. In other words, we do all the basic groundwork to ensure reasonable performance, but we do not often go much further than that.

If this basic groundwork does not deliver acceptable performance, we would often rather add more hardware than spend a lot of time on intricate performance tuning. The reason is simple: if we need to change or retire a query or the whole system, the hardware stays with us and keeps being useful on other projects. On the other hand, all the time spent squeezing more performance out of a query is lost the next time we change our system.

Our team, and our users, place a high value on predictable performance. From our users’ perspective, a certain response time may be essential, and we may be willing to invest in hardware to ensure it. Suppose, for example, that we are required to ensure a response time of 0.5 seconds or less. One query consistently returns in 0.3 – 0.4 seconds, so it meets the requirements. Another query usually returns in 0.01 seconds, but intermittently takes more than 2 seconds, due to a different choice of execution plan. Even though on average the second query runs faster, and uses fewer resources we would rather use the first query, and add more hardware if needed.

Needless to say, complex SQL queries are less likely to deliver predictable performance. As such, they are less likely to stay around. As the time goes by, we replace complex queries with simpler ones, and we do not develop many new complex SQL queries at all.

Preference for Simple Tools and Techniques

Throughput this article, a recurring theme has been the need for simple, independent systems each of which performs a single function very well and is adaptable to change. In my experience, in order to achieve this it is important not to constrain a development team by imposition of ‘universal’ coding standards, approaches or tools.

In our organization, different teams take different approaches to database programming. For example, some teams do their own database design and SQL programming whereas others prefer to get help from the DBA. However, all teams benefit from being free to choose the tools and approaches that work best for them. In the true spirit of the agile approach, these tools will often be very simple and minimalistic. Just as the systems we build must be adaptable to change, so must the tools we use. As described earlier, over a few iterations our SQL unit testing tool has evolved towards simplicity and we could change to a different approach, if required, quite easily.

As another example, we evaluated SSIS for data loading and while it has many powerful capabilities, we only needed a small fraction of them and found it a complex tool to use.

SSIS’ 15 faults

This article by Ayende Rahien (http://ayende.com/blog/2637/ssis-15-faults) provides several great examples of how the sophisticated features of SSIS may get in our way instead of making us more productive

For better productivity and agility, we decided to develop our own data loading library. We load data using just a few simple classes, written in C#, so that we completely understand how data gets loaded, and we can debug any problems easily, and change anything quickly. Many may scorn the idea of writing our own data loading tool as “reinventing the wheel”, but it works out for us, in this case, since the library is simple to use, does exactly what we want and nothing else.

Accelerating Agile: Hyper-performing without the Hype

Dan North is an ex-colleague of mine and I recommend his presentation describing how high-speed agile teams use simple tools to succeed.

Conclusion

In this article, I’ve attempted to summarize the core principles than underpin agile development, and then describe their direct impact on how we develop databases in an agile fashion. Many aspects of agile development may seem strange and counter-intuitive to a developer unfamiliar with working practices in an agile team, but hopefully with an understanding of how the pieces of the puzzle fit together, it all eventually begins to make sense.

Certainly, the principles, techniques and tools that I’ve described have been a direct benefit to our in-house development team, allowing us to produce, consistently and quickly, high quality software that meets the exact needs of our business and of our users.

Hopefully, this article will at least have made you curious to learn more about the agile methodology, and how to apply it to database development. Good luck with your learning!

Tags: , , , , , , ,

  • 37681 views

  • Rate
    [Total: 1    Average: 5/5]
  • Anonymous

    Basis of Agile
    I am amazed that so many "common sense" policies and practices are lumped under the name of ‘Agile.’

    I was doing the majority of these things back in the first days of PC’s, learned from management and service training, before the acronym ‘Agile’ was even thought of.

    The most important part of system development that didn’t get a mention here – embedded documentation ( detailed explanation ) in the code and database scripts.

    Documentation hardly got a mention at all, but to not insist on the logic and changes being written into every script and code page is making sure that it never gets done, and next time someone has to make a change, they have to spend hours checking the logic, and often fiddling around with version control software (also not mentioned ).

    Likewise – in assuring correct process – I didn’t see a mention of Psuedocode (logic outline) , and peer review of that logic outline , before you get the programmer to ‘fly’ with it.

    Of course, much of the Pseudocode becomes the final documentation, after the programmer has gotten the thing tested and made any required adjustments.

  • AlexK

    Common sense – sure!
    I am with you – agile development is just applying common sense approaches. We only need to spell them out explicitly because some processes in IT may completely defy common sense.

    I disagree that documentation is "The most important part of system development". From my perspective, the most important part is efficient and precise communication, and IMO documentation is not the best tool for that. The section entitled "Establish Efficient Communication" describes what works better for us.

    Regarding "next time someone has to make a change, they have to spend hours checking the logic" – instead of documentation, we use automated tests to accomplish that. The computer will do it for you, and in our experience it will do it faster and better than a human. So, quoting from Agile Manfesto itself: "Working software over comprehensive documentation".

    I am not with you on "in assuring correct process" – different teams may choose different processes. No one process is correct for everyone. "Individuals and interactions over processes and tools".

    Also I do not know any agile developers who use pseudocode now. Instead, we develop source code that kind of looks like pseudocode on top level – it is much simpler and works better for us. This is a good example of how agile development simplifies things, like Occam’s razor.

    This approach is described in Robert Martin’s book entitled "Clean Code".

  • Anonymous

    Good stuff!
    Great read!

    One of the things I do, when relevant, is to add an "order by" column so that if business requirements call for it a list can be sorted in some other way than alphabetical or numerical. Like this article suggests though don’t get carried away trying to anticipate requirements.

  • mike good

    Excellent article
    Thought provoking. Thank you.

  • Anonymous

    Testing replaces documentation
    A good article but I agree with the first poster, so much of this is just common sense that certainly in databases, a lot of us as DBA’s have been doing for 20 years.The reality is if you design your database well and normalise it correctly then modifying it is generally straight forward. However I disagree with AlexK that testing replaces documentation. I think he is misunderstanding the point being made. Automated testing tests the logic you hope that you have implemented once you have made your changes. It does not assist in any way however when someone needs to make a change to existing code and first of all needs to understand the existing logic. At that point documentation saves them large amounts of time having to work through undocumented code trying to establish what it really does before they can they alter it. Testing does not assist you in establishing what the current logic does, documentation however does.

  • AlexK

    Can you really trust documentation?
    Can you really trust documentation that does not automatically validate? We cannot, because things change rapidly: we deploy about 200 times per year, half of that is database deployments. Documentation, including comments, can and usually does get out of sync sooner or later. We have learned that the hard way. The book entitled "Clean Code" has lots of examples.

    Even if it is still in sync, how do we know we understand the documentation correctly? I gave just one example about the meaning of "by next Friday".

    But we can and do use automated tests to document what the code is supposed to do, and very successfully. This makes sure that our documentation is up-to-date and correct, so we do not waste our time reading incorrect and ambiguous documentation. This is what BDD, and RSpec, and Cucumber are all about.

    I did not include a reference to "Clean Code" in the article, but the book is a must read – it is full of relevant examples.

  • Carl Perkins

    Bad Advice
    Alex, I could not believe all the bad things I read in this article. Then I paged down and saw you had just over 10 years experience – that explained it all. Of particular bad practice was the statement to add more hardware. Dude, your advice is quite bad. Eventually after adding more hardware the bad design and inefficient sql queries/architecture have to be changed. Good luck to you and I hope you awaken from your dream world. No doubt some of these practices are in use at Healthcare.gov. IMHO.

  • Anonymous

    Nice Article
    Thanks to Alex for taking the time to write all of this out in a structured and clear manner.

    Note to ‘Basis of Agile’ poster: ‘Agile’ is not an acronym.

  • LeoP

    Read and Think
    As always, all articles have to be carefully read. This article has some good points and some bad points. There are many cases where hardware is not capable to fix a bad coding or the price of the hardware solution is too high. This article did not mention unified development standards (and many other things related to the good designs). Today our developers spend a lot of time to understand database design because within last 10 years each team designed databases independently, quickly, and without any database standards. But it was with agile principles. The price the company is paying NOW.

  • Vasiliy

    Thanks for sharing
    Good post.
    Thanks for sharing and letting us beat you on your mistakes. 🙂

    Agree with others, lots of common sense.

    Specifically, I agree about:
    – importance of communication: weakpoint of our dev team with devs all over the US and world
    – minimalistic approach on choosing tools: easier to understand, use and replace (if needed)
    – TDD: our team never has time to cover all the code with auto-testing 🙂

    Disagree with statements that:
    – no documentation needed
    High-level documentation with flows and diagrams is a must have.
    As for detailed documentaiton, it is difficult to keep up-to-date. I’d rather have self-descriptive code with sufficient comments.
    – automated tests can replace documentation
    I don’t see how tests can replace documentation. Doc is a guideline for tests.
    – add more hardware in case of performance issues
    I guess you had a specific issue when "more hardware" was the best solution (time crunch?).

  • Anonymous

    Agile???????
    Yet another article evangelising Agile . It definitely is a trait of younger developers, who have been swallowed into the mire of Agile,TDD,BDD and all the other silver bullets of IT development.
    Write down what needs to be built – it is called a functional specification!

  • paschott

    Interesting read
    As others have noted, we do need some documentation, if only in the comments saying what has changed and why. We tend to reference the story for which the work was done for future reference as well.

    As for Agile – yes, it works well. Functional specs are great, but there are times that they just aren’t quite what the customer wants even though they wrote and approved them. Seeing the iterations over the sprints means that there can be adjustments along the way and that tends to result in delivering something closer to what is desired. I’ve seen far too many well-spec’d projects not do well because the time to deliver was overly long and the specs needed to change along the way. Perhaps that’s a management failure, but the end result is the customer not getting what they need.

    More hardware sometimes is the quickest solution to get you past some performance issues. It’s obviously a temporary solution, but it can really help for a short-term boost when developer time is taken.

  • Anonymous

    Vehicles
    "To put it another way, if we build cars for a country that drives on the right, we put the driver’s seat on the left. If we start to supply cars to a country that drives on the left, we build a new car with the driver’s seat on the right. We do not try to develop a car where we can move the driver’s seat from right to left; such a car is likely to be both expensive and unreliable."

    Right Hand Drive vehicles from manufacturers in Left Hand Drive countries are most definitley Left Hand Drive vehicles adapted, in many cases the pedal linkages come through the bulkhead on the left and a set of bars link across the bulkhead to the right hand side of the vehicle!
    In other cases there’s clear cludges to adapt the Left hand drive design for the wheel being on the wrong side.

    The only "Agile" environments I’ve worked in are where the concepts been used to allow even more Micro-management than we had under waterfall.

  • Charles Kincaid

    You are correct about aviation
    The Wright brothers in fact had many failures before they got it right. One of the things that held them back was holding onto statements made by a published authority. There is a line that might not be an exact quote but the observation that "Lilienthal was wrong" lead them to abandon the use of published data tables and derive their own data by testing and observation.

    There is also some quote attributed to T. A. Edison about not failing a thousand times but succeeding in finding a thousand things that did not work.

    So my point is that anything must be tested to see if it actually works in your situation. That includes methodologies such as Agile. Its good ,no doubt of that. But nothing is an answer to all prayers.

  • paschott

    Vehicles?
    I don’t think the vehicle analogy holds up when dealing with Agile. It’s pretty obvious that you have to have some base starting point (right vs. left hand drive). With software, there are some things you need for starting, but the direction can go many ways after that. Yes, good specs can give you a guidelines, but if you aren’t getting feedback from the customer on a regular basis it’s very easy to deliver something "to spec" that doesn’t actually meet their needs.

    I’m sorry that you’ve worked in environments where Agile was poorly implemented, but it can work well. It’s not for everyone and often people throw the term out there without actually adopting it, but when implemented correctly it can work well. We’ve been using it for some time and it’s allowed us to get rapid feedback from the customers and tweak our direction to give them what they actually need/want at the time rather than taking 3-6 months to code something, deliver it, then hear about all of the things that have come up since we started the project.

  • cdmantel

    Excellent article
    I benefited very much from your article. Thought provoking. Like the references/links to other works. Reason I registered on this site.

  • Rob Love

    Avoiding the root cause
    I would have to agree with the majority of the comments made here, as well as the common sense points you have made. However, I have one comment to make about Database Refactoring. I believe it to be the root cause of all woes.

    The reason Agile Development came into being, is because people began to realize that their databse structures, and subsequent applications, weren’t as flexible as they would like them to be. Hense, rather than fix what we have, lets build something new, quick and dirty, throw away, try to emply some level of code re-use if possible. Frequent Refactoring is not a solution. Its a reaction to repeatedly employing faulty stuctures in the first place.

  • Rob Love

    Avoiding the root cause
    I would have to agree with the majority of the comments made here, as well as the common sense points you have made. However, I have one comment to make about Database Refactoring. I believe it to be the root cause of all woes.

    The reason Agile Development came into being, is because people began to realize that their databse structures, and subsequent applications, weren’t as flexible as they would like them to be. Hense, rather than fix what we have, lets build something new, quick and dirty, throw away, try to emply some level of code re-use if possible. Frequent Refactoring is not a solution. Its a reaction to repeatedly employing faulty stuctures in the first place.

  • Anonymous

    Agile Sometimes
    I worked a number of years with a large financial services company. We got a new CIO that wanted to use Agile. I don’t know if it was our culture or whether it Agile principles that didn’t work so well at larger companies like mine, but we ended up not using Agile and the CIO not too much later left the company.

    The problem I saw with Agile there was we had to integrate with other existing applications. Quite often these existing apps had some very strict audit controls since we dealt with sensitive personal information. Also, we had to design for continuous availability since the company was global. The technical architecture for this requirement is very challenging even if you have a great application solution. So we would have to coordinate with other groups for our requirements (operational, technical, business, security, and quality). If there was a delay in one, it would impact others. Frequent changes would also have to redo integration and regression testing for all. This would impact bugets and business plans.

    We found that the traditional waterfall method was better for us for database and application development (i.e., requirements gathering, analysis on the requirement, design on the analysis, building on the design, testing on what was built, and implementing what was tested).

  • Anonymous

    Correctly guessing the future is where the real action is.
    It’s called ‘experience’.
    Innovative high return projects come form correctly guessing the future.
    I’ve done it many times.
    Agile is really a comfort zone for average developers in average projects.

  • Anonymous

    To Agile or Not?
    The ongoing question? I have been involved with the maintenance and enhancement of a very complex enterprise application for a very long time. As a development organization we struggle with many of the problems mentioned by Alex. Five years ago we moved towards an Agile approach with questionable success. There are still many challenges to overcome. All agile consultants have answers but the devil is always in the details which are often glossed over.

    Just a few related to what was mention in this article:
    1. Regarding frequent release and feedback. It simply is not that easy. When a PM enters a feature request in a complex enterprise solution, those of us familiar, instantly recognize the complexity. Any attempt at a short release to market is simply an unfinished feature. If we put together enough of these quick releases, we might start to get usable feedback if we haven’t already turned off the users altogether. Its a gamble. Once the feedback begins coming in, it is sporadic and usually only from a few sources, not at all representing the masses that we are coding for. Not convinced of the benefit here. This might work if you have the right processes in place, a reliable list of domain experts, and the time for endless consultings with your sources. I get the premise from which Alex speaks but those damn devil details keep overwhelming our efforts.

    2. Regarding giving small development teams complete freedom of choice and forget any sense of standardization: Good Luck. Admittedly our own development teams have rejected any form of standardization. They all believe they are the best at what they do. The results quite simply are: Completely different look from one module to the other (often questioned by our users), strange bugs popping up in what normally are routine form operations. Inconsistency in look and performance. A little standardization goes a long ways but it does take some effort and team work.

    3. Regarding documentation: There are the Pro-doc and Anti-doc and rarely do the two agree. But I should add that never once have I had a coder complain when I have provided documentation to help him understand the challenges in front of him. Anit-doc are usually individuals who simply do not wish to do the documentation themselves but more often than not they would gladly use it if provided.

    Alex, it sounds like your group is well into a successful implementation of Agile. You should be congratulated for I know this is no minor accomplishment. Having an entire department of developers on the same page is unbelievable. Perhaps you are more standardized than you realize.

  • paschott

    Frequent Releases
    Just a note from my experience. Frequent completion of stories should mean that the stories are small enough to be done. That might be the functionality behind the scenes or the UI for something else that can be finished in a week. For releases – several companies adopt a "dark" philosophy that means even though some code is finished and in production, it’s not visible to the customers until a switch is flipped. That might be per customer or a general release, but it works well to hide that new functionality until it’s ready to use. (Completely done as opposed to sections of it being done.)

    Demos are done on the smaller completed stories to get feedback from the customers. Sometimes that results in a discussion of what should be done because the original story wasn’t quite right or perhaps misinterpreted or maybe even because requirements had changed a little since the story started.

    I agree about #2 – a company needs standards. For documentation, there needs to be a happy middle ground somewhere. Self-documenting code rarely is. Good comments can help, though are rarely complete.

  • Anonymous

    Unimog
    "To put it another way, if we build cars for a country that drives on the right, we put the driver’s seat on the left. If we start to supply cars to a country that drives on the left, we build a new car with the driver’s seat on the right. We do not try to develop a car where we can move the driver’s seat from right to left; such a car is likely to be both expensive and unreliable."

    OK, we find out ahead of time which side the driver’s seat should be on (You know, during the specification phase.) versus putting it wherever we want and thinking we can quickly change it later at the customer’s request.

    But this is not really a good analogy. Because most cars are mass produced, economy of scale becomes very important. If we are smart, we do a market analysis before designing the car to determine (among many other things) if sales are anticipated in both LHD and RHD markets. If they are, we then design the car to maximize the number of parts common to both LHD and RHD builds and therefore minimize the total inventory of differing parts we have to maintain to build and support the car. Then, depending on it’s destination, the car can be built up either way from a common infrastructure. Also, most car models are designed so they can be ordered with major differences in options (eg auto vs manual transmission, all wheel drive, etc.) that don’t require them to be redesigned or rebuilt from scratch. That’s the BDUF approach working in real life.

    Also, from wikipedia:
    "Today, experimental versions of drive by wire and brake by wire vehicles are being developed, which allow the driver to slide the steering wheel/brake controls from left to right with the gauges in the centre dashboard. They are expected to become popular in countries such as Thailand that have land borders with opposite-drive countries. The newest Unimog models can be changed from left-hand drive to right-hand drive in the field to permit operators to work on the more convenient side of the truck."

    So, like pretty much every thing else in life, it depends. It depends on your customer base and your product. If you’re writing shipping software for your own company, your customer base is one, you don’t have a huge design budget, and you still get paid no matter how many re-writes you do. If you’ve been contracted to write shipping software for another company, what are the terms of the contract? If you do a fixed price bid, you may be doing a lot of free re-writes if your first delivery is way off the mark. In that case, BDUF may pay off if it facilitates early customer acceptance. If you’re writing a commercial piece of software for sale to the public, you have to consider time to market, anticipated sales based on feature list, your company’s reputation, future product offerings, etc. You may gain early sales by being first to market, but you may go bankrupt if your company earns a bad reputation for delivering buggy software lacking in features.

  • Anonymous

    Unimog
    "To put it another way, if we build cars for a country that drives on the right, we put the driver’s seat on the left. If we start to supply cars to a country that drives on the left, we build a new car with the driver’s seat on the right. We do not try to develop a car where we can move the driver’s seat from right to left; such a car is likely to be both expensive and unreliable."

    OK, we find out ahead of time which side the driver’s seat should be on (You know, during the specification phase.) versus putting it wherever we want and thinking we can quickly change it later at the customer’s request.

    But this is not really a good analogy. Because most cars are mass produced, economy of scale becomes very important. If we are smart, we do a market analysis before designing the car to determine (among many other things) if sales are anticipated in both LHD and RHD markets. If they are, we then design the car to maximize the number of parts common to both LHD and RHD builds and therefore minimize the total inventory of differing parts we have to maintain to build and support the car. Then, depending on it’s destination, the car can be built up either way from a common infrastructure. Also, most car models are designed so they can be ordered with major differences in options (eg auto vs manual transmission, all wheel drive, etc.) that don’t require them to be redesigned or rebuilt from scratch. That’s the BDUF approach working in real life.

    Also, from wikipedia:
    "Today, experimental versions of drive by wire and brake by wire vehicles are being developed, which allow the driver to slide the steering wheel/brake controls from left to right with the gauges in the centre dashboard. They are expected to become popular in countries such as Thailand that have land borders with opposite-drive countries. The newest Unimog models can be changed from left-hand drive to right-hand drive in the field to permit operators to work on the more convenient side of the truck."

    So, like pretty much every thing else in life, it depends. It depends on your customer base and your product. If you’re writing shipping software for your own company, your customer base is one, you don’t have a huge design budget, and you still get paid no matter how many re-writes you do. If you’ve been contracted to write shipping software for another company, what are the terms of the contract? If you do a fixed price bid, you may be doing a lot of free re-writes if your first delivery is way off the mark. In that case, BDUF may pay off if it facilitates early customer acceptance. If you’re writing a commercial piece of software for sale to the public, you have to consider time to market, anticipated sales based on feature list, your company’s reputation, future product offerings, etc. You may gain early sales by being first to market, but you may go bankrupt if your company earns a bad reputation for delivering buggy software lacking in features.

  • Skeleton

    Thoughts on 6 years of Agile Database Development
    Lessons Learned from 25 year of Agile Database Development and 42 years of IT experience:

    "Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage."

    This is asking for a disaster. If requirements change, you change delivery and redesign, or you save changes for after implementation. Late in development is NOT the time to redesign.

    "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation."

    This is dangerous unless it is accompanied by written documentation that may be referred to in case there are questions about or ambiguities in the spoken word. Relying on people taking their own notes in meetings of casual conversation runs the risk of varying interpretation of discussions. (read that ‘selective hearing’)

  • Anonymous

    What???
    Agile hasn’t been around 25 years – the manifesto only appeared in 2001 !!! .
    42 years of IT I can appreciate !!

  • AlexK

    it is not 1995 anymore
    Carl Perkins, sorry to disappoint you, but I wrote my first client-server app in 1995, and worked for a long time in other ares of programming before that. Yet I keep up with current developments.
    You have misstated my advice, which was to deliver sound basic performance without anything that kills performance, and stop at that, instead of spending too much time on more optimizing for diminishing returns.

    With hardware prices at all time lows, it just makes no business sense to spend days trying to squeeze more performance out of a reasonably performing query, instead of just spending a few thousand dollars in hardware – it is not 1995 anymore. My iPhone has more computing power than my RDBMS server did in 1995 – and we need to keep that in mind.

    In general, modern trends in software development heavily utilize cheap hardware. Consider, for example, Ruby – it is slow, it needs more hardware, but overall it allows for much cheaper solutions, because we save a lot on development time.

  • AlexK

    we do not have unified development standards
    LeoP, I did explicitly mention that we do not have unified development standards.

  • AlexK

    High-level documentation – sure
    Vasiliy,

    regarding "High-level documentation with flows and diagrams is a must have"
    we have something like that, but at a very high level. Using an example from this article, it would have boxes entitled "US Shipping", "Russian Shipping", and "Consolidated Shipping Reports". So, no documentation with flows and diagrams for database development, which all takes place inside one box or another.

    "I don’t see how tests can replace documentation. Doc is a guideline for tests." I encourage you to read about BDD and Cucumber, and RSpec.

  • AlexK

    version control?
    paschott,
    Regarding "As others have noted, we do need some documentation, if only in the comments saying what has changed and why. We tend to reference the story for which the work was done for future reference as well." – version control does it better than documentation, and automatically. We can integrate it with the ticket tracking system, so that we automatically "reference the story for which the work was done".

  • AlexK

    not sure what you meant
    Rob Love,

    I am not sure what you meant – it may be cheaper to rebuild from scratch than to refactor. For instance, when US health insurance companies had to stop using SSN as insuranceID, they had to change their systems, they had no choice – change was inevitable.

  • Steve W

    Data normalization and agile development
    Alex,

    When you work on an iteration of a project, do you typically try to normalize the data? How much time or care do you put into this? Do you normalize to 3rd normal form, or do you have some other criteria?

    As requirements change, do you ever have to go back and normalize existing tables based on the new information? When would you, and when would you not do this?

    Thanks for the article, by the way. Even though I’m uncomfortable with some of the suggestions right now, it really helps me to think things through.

  • Anonymous

    Over 10 years??????
    Profile on redgate says: over 10 years experience . You mention developing in 1995…..so that’s only 18 years. Some of us have nearly 30 years and also keep an eye on new technology……

  • AlexK

    grassroots movement
    Agile Sometimes,

    IMO agile is a grassroots movement – it cannot be forced on an unwilling team by a CIO.

    Also there are lots of technical limitations. All our integration/regression tests must run automatically in a few minutes. To accomplish that, we need to run them in parallel, with database on local instances, on RAM drives, etc. This speed requires some work to achieve. And of course, keeping the systems small and simple – otherwise the sheer number of permutations to test prevents quick change.

  • AlexK

    To Agile or Not?
    To Agile or Not?

    1. Regarding "When a PM enters a feature request in a complex enterprise solution, those of us familiar, instantly recognize the complexity" – this is why we do not grow complex enterprise solutions at all.

    "This might work if you have the right processes in place, a reliable list of domain experts, and the time for endless consultings with your sources" – because our simple in-house solutions may be several times more efficient to work with, eliminating busywork, our users are very motivated to help us get them the best solutions. As we communicate face-to-face a lot, over time communication becomes very efficient and it does not take much time.

    2. " A little standardization goes a long ways" – with efficient communication, when the users want consistent look and feel, they get it. Happens all the time.

    3. "Anit-doc are usually individuals who simply do not wish to do the documentation themselves but more often than not they would gladly use it if provided." – not necessarily. There is another category. Typically agile developers, including me, think that documentation is better than nothing, of course, but in most cases we can and should do better, especially with modern tools that make it so very much easier.

  • AlexK

    Unimog
    Regarding " If you’ve been contracted to write shipping software for another company, what are the terms of the contract? If you do a fixed price bid, you may be doing a lot of free re-writes if your first delivery is way off the mark." – I have zero experience of agile development in such environments.

  • AlexK

    poorly implemented?
    paschott,

    Regarding "I’m sorry that you’ve worked in environments where Agile was poorly implemented" – can you elaborate how did you get to this conclusion?

  • paschott

    re: poorly implemented
    Alex, wasn’t referring to you in that case. The threading on this forum leaves a little to be desired. This was in response to the anonymous poster saying that they’d never seen Agile implemented in a way that didn’t lead to more micromanagement.

  • Anonymous

    Background reading?
    Have any of these Agilephiles ever read "The Software Project Survival Guide" by Steve McConnell ?

  • AlexK

    Re: Data normalization and agile development
    Steve W,

    We definitely normalize to 3NF, unless we denormalize to enforce business rules with trusted constraints – we do that from time to time. We take data integrity very seriously, because our clients need clean data, and having the data normalized helps a lot.

    As the requirements change, we change the database schema – IMO it is much cheaper/safer/faster than to develop against inadequate database schema and use kludges aka workarounds.

    A few years ago we have learned how to refactor live OLTP table without downtime for users, overtime for us and the DBAs, and with very little risk. That was a real game changer. Since then, we keep the database schema current.

  • Robert young

    Right Angles (Angels?)
    — A few years ago we have learned how to refactor live OLTP table without downtime for users, overtime for us and the DBAs, and with very little risk. That was a real game changer. Since then, we keep the database schema current.

    Orthogonal is your friend. Keep to that, and schema changes are mostly invisible to code that doesn’t participate in a change.

  • AlexK

    orthogonal design good, not enough
    Robert Young,

    While orthogonal design is a good and useful principle, it is not nearly enough if we need to split a table into parent-child, with like a hundred modules involved in the change.

  • Atradius

    Listen to your customers?
    Interesting article – but the most successful companies in the market do not apply these principles.

    Take for example Apple. It has been said, by Steve Jobs himself, that they do not ask their customers "what they want". He quoted an anecdote attributed to pioneer car manufacturer Henry Ford, who said "If I had asked my customers what they wanted, they would have told me ‘a better horse’." Apple did not "listed to customers" to build a visionary product. You just cannot.

    Communication with customers may be good, but you’ll have to know what communication means – "listen to people" every day and change to what they are thinking today (many people change their ideas every day) ends up with myriad of changes. Intelligence is NOT to be found in "collective thought". That’s communism, and communism is dead – it doesn’t work.

    Are you going to let a street cleaner running a country? You probably think that would be a good idea. It is not – special knowledge is required to run a country.

    One idea which would solve this would be "organizational specialists". These would not be customers which use the programs, and they would not be programmers. But they would know how to organize a workflow (for real, not just BPL evangelists), and they would be capable to isolate proper organizational changes which benefits both the programmers and the customer. It would be them which communicates with the customer. They would, however, take the customer opinion with a grain of salt.

  • nepdev

    Complexity Increases This Way
    Complexity does not increase with this methodology – it increases, in fact, dramatically.

    Yes, each development team sits in their own corner doing their own thing. And from their perspective it is sort of easy – they don’t need to know anything about the whole picture. They do only this small section. So it seems "simple".

    In the end of the day, all these pieces are still a "system" as a whole. And now, with all these many simple systems which interconnect with each other, you have an enormous puzzle. But it is still ONE SYSTEM. And now, since nobody actually planned the entire thing, you have nobody who has an overall view of it.

    A change in one small part might require interface changes, and who has any plan which other part uses that interface? So you won’t do interface changes? Sometimes you have to. But then you do need an overall plan. Which is going to be quite complex, given the heterogeneous way this operates.

  • paschott

    “Customer” Definition
    The point above about successful companies is a valid one, but the customer in Agile is not necessarily the final end user. It is most often an internal person – the project owner or some other person who is handling the process. Often whatever is produced doesn’t get into the hands of actual users until much further down the line.

    Of course, I still argue that you need feedback from actual users at some point in order to validate that you’re doing the right thing. That might come really close to the end of the cycle, but I’ve been involved in some projects where what we delivered was completely not what the actual users wanted. I’ve also been involved in some projects where the users didn’t realize they wanted this until it was given to them. Knowing that before the actual release and adjusting can be a good thing. Microsoft does this to some extent, bringing people in to their usability labs (or at least they did). It’s a wise move at that point to gather real-world feedback.

  • Anonymous

    Atradius: Apple is definitely agile, and so are Google, Amazon and Facebook
    @Atradius: quoting from Forbes: "We have always known that Apple [AAPL] is agile, having successively disrupted the music industry, the cell phone sector and the tablet market, as well as itself. But is Apple truly “Agile” in the sense of the Agile Manifesto? Apple is conspicuously absent from the Agile/Scrum/Lean/Kanban conferences. Is it possible that they could be agile, without being Agile?

    Thanks to Adam Lashinsky’s terrific new book, Inside Apple, which I reviewed here yesterday, we now know. Apple is Agile."

    As paschott already mentioned, "Customer" might not be every user.

    It it very easy to google up and see for yourself that Google, Amazon and Facebook are agile as well. So, all the big four are agile.

  • AlexK

    @Atradius
    Atradius,

    I encourage you to read the following book: "Built to Last" by Jim Collins. Its Chapter 7 is entitled "Try a lot of stuff and keep what works". It describes how some of the most innovative companies use trial and error.

    Surely the Big Four are all agile, and so is Twitter:
    http://gigaom.com/2010/10/07/inside-pivotal-labs-the-agile-force-behind-twitter-and-groupon/

    Having lived most of my life in a Communist country, I disagree with the following: "Intelligence is NOT to be found in "collective thought". That’s communism, and communism is dead – it doesn’t work." The opposite is true: USSR was a rigid hierarchy with almost all decisions made at a very high level.

  • AlexK

    @nepdev
    Regarding "all these many simple systems which interconnect with each other, you have an enormous puzzle" – I do not think this should be the case, because well-designed systems consist of loosely coupled components that do not depend on each other’s implementations. As such, adding more components does not affect my database.

    I could care less if one more new client app is developed in Closure or in Ruby as long as they keep talking to my app server via http using the same interface – this does not increase the complexity on my side.

  • Anonymous

    Russia with Love
    — The opposite is true: USSR was a rigid hierarchy with almost all decisions made at a very high level.

    This differs from (American) corporatism, how? Not to take the thread into the political weeds, too much, but you’ve made a distinction which has no difference. There, they call the oligarchs as such. Here, we call them the 1%. In both cases, a minority directs the majority.

    As to communism (lower case c), it does work. Israel is very much a communal country. Open source, and linux in particular, is communal production.

    And, as to collective intelligence, it’s quite all the rage with the meme-ophiles: goes by the name "wisdom of the crowds". Whether one subscribes to such, or its antithesis, Rand-ism, will get a bunch of Oxbridgeians in a lather I suppose.

    Which brings us back to software. Jobs was of the point of view that, with clever marketing, one could produce a prefabricated post hole and convince people to buy it, since they weren’t previously aware they needed one. The Agile folks, on the other hand, assert that developers can only figure out what to make by constantly polling the users. Both methods work, but in largely non-overlapping venues. Jobs — toys. Agile — anything else.

    Waterfall, despised by the Agile set, is hierarchy in motion. Agile is anarchy in motion. Note well that healthcare.gov, in all its splendid software failure, was Agile applied where waterfall was more reasonable, measure twice and cut once. The Agile folks keep taking more boards and cutting. When lumber is scarce, that’s not smart.

  • Adam

    More detail
    I’d really like to hear more detail about what you are loading data with and how you deploy multiple data sets, dev/qa/unit test to other environments.

    How you are doing load tests and performance tests?

    I’d also like to learn more about what tools you are using to deploy databases to different environments. When you refactor you have to handle destructive changes and creating these scripts adds a lot of complexity and risk to the deployment. That is the hardest thing about changing requirements and refactoring.

  • AlexK

    Russia with Love?
    Regarding "– The opposite is true: USSR was a rigid hierarchy with almost all decisions made at a very high level.

    This differs from (American) corporatism, how? Not to take the thread into the political weeds, too much, but you’ve made a distinction which has no difference. There, they call the oligarchs as such. Here, we call them the 1%. In both cases, a minority directs the majority."

    Thank you for sharing! Can you please elaborate: how long have you lived in the USSR, at what age, during which years? Just wondering…

  • AlexK

    @Adam
    Adam,

    "I’d really like to hear more detail about what you are loading data with and how you deploy multiple data sets, dev/qa/unit test to other environments.

    How you are doing load tests and performance tests?"

    That’s huge, but I am going to write more about automated tests.

    "I’d also like to learn more about what tools you are using to deploy databases to different environments. When you refactor you have to handle destructive changes and creating these scripts adds a lot of complexity and risk to the deployment. That is the hardest thing about changing requirements and refactoring."

    Instead of one destructive change aka waterfall, we may refactor tables in a series of low risk changes that can rollback right away. I wrote a section about that in the article entitled "Developing Low Maintenance Databases".