Click here to monitor SSC
  • Av rating:
  • Total votes: 23
  • Total comments: 9
Mike Mooney

What is DevOps really?

08 May 2014

Ops and Development people are busy, and focussed on their jobs. They have plenty of ideas for  improving the whole software delivery process but the task of developing, maturing and communicating a different way of working isn't often in their job-description. Others are only too keen to do so, but on their own terms. So, let's get clarity. What is the central  DevOps idea really?

What is ‘DevOps’ really?  Although the term is now being treated as if it were a marketing buzzword like “cloud” and “big data”, there is a real grassroots initiative behind the hype that grew out of some good ideas generated by a few smart people working on application delivery. They went on to develop these ideas until they did real, concrete, awesome things. Others saw how successful these new teamwork methods were, and tried to figure out shortcut formulas to get there, without fully understanding the original ideas. This has caused problems and confusion because it is impossible to implement DevOps practices without understanding and putting in practice the underlying ideas. Unfortunately, there is no way to shortcut the DevOps ideals, or to implement them only by using an expensive software package. Although there are tools, procedures and techniques that you can help once you have adopted the DevOps ideas, there is an essential prerequisite; a positive working relationship between teams. This is essential in order to identify the problems and find appropriate solutions.  The focus must be on the teams, the individuals within them, and their interactions rather than on processes and tools. 

At its core, DevOps is not a tool. It’s not a process. It’s not even a methodology. First and foremost, in its most valuable sense, it’s an idea to strive for.

History

To me, DevOps is just the next stage in a succession of grassroots ideas by successful software teams trying to find better ways to be more effective, and extend this to the whole development process. All of it is traceable back to the ideas of the Agile Manifesto

Although there have always been new ideas to make team-based software development more effective, the past several decades have seen a recurring pattern in how these initiatives have evolved in their scope, starting with programming, and ending by encompassing the whole delivery cycle:

  • For a while, the focus was on getting the complexity of enterprise software under control. This seemed to have, as its objective, building and writing that code as quickly and efficiently as possible.  You saw the introduction of RAD and 4GL tools, development patterns like object orientation, and design approaches such as UML and RUP.  This focused on the rapid communication of ideas between analysts and developers, and rapid development directly from that communication.
  • Once developers had figured out ways to make individual and independently-developed components easier to build, they had to ensure that their code fit together nicely with everyone else’s. It was the effort to solve this problem that gave birth to the idea of continuous integration. This helped to build the stability of interaction and coordination between development teams.
  • As the development teams got faster and became more flexible, they had to work closer with the project managers that defined and managed what they should be building and when they could expect it to be done.  Agile project management processes such as Scrum filled this need.  This helped to improve the communication and effectiveness of the whole project team, from developers to QA to project manager, and even product owners and business users, who were very under-represented in previous approaches. 
  • Another challenge of building software so quickly, with so many changes along the way, was that of validating the software.  When you are moving fast enough, and handling changes to the project scope and desired functionality, it’s hard to capture what the actual correct functionality should be, and whether the software you are building meets those requirements.  This brought along several approaches to automating the test process; unit testing, integration testing, and UI testing.  Developers starting using patterns such as Test Driven Development, and started working with, and communicating with, the QA team to ensure that there was a shared vision of the expected quality of the system.  This increased communication between development and QA resulted in less focus on futile distractions like bug counts and arguing about whether something is actually a defect by strict definition, and more focus on everyone working together build the highest quality system they could.
  • Having partially conquered many of these problems, many product teams took the Agile ideas a few steps farther to get the business users more involved.  While it was always important to make sure that the product being built was what the users actually needed, it was very easy to lose sight of this when focusing on spreadsheets and feature matrices.  More importantly, they wanted to ensure that they were genuinely solving the user’s problem: This required working with the users, asking questions, getting to the root of the problems, and offering potential solutions, rather than just demanding a requirements document.  To help along this communication and discovery process, ideas such as Behavior Driven Development and Specification By Example were developed to ensure that the business users, the only people who really know what needs to be done, are more involved in the project (serving as Pigs rather than just Chickens, if you will).
  • Now, having confronted many of the hard parts of the process of efficiently building software that actually solves the user’s problems, it became more obvious that there was work to be done in improving the process of shipping and supporting that software.  This has involved working with operations teams to streamline the deployment process and monitor the systems throughout various test and integration environments. 
  • This “DevOps” approach has already solved a long list of long-standing problems in this business, and so might be expected to thrive. However there is a risk, which becomes obvious if we study what happened to past attempts at improving the software development process. The initiative and development of a movement that is rooted in grassroots experience is somehow morphed into a commercial opportunity. Could it be that a movement that has been driven by Ops people and Developers will become hijacked, and subsequently distorted, by our marketing colleagues for commercial reasons, by hyping it into the next Big Thing, the next Game Changer, the next Silver Bullet? If so, then that is a Bad Thing.

 Learning From History In Improving The Software Delivery Process.

Repeatedly, over the past decades, there have been grass-roots attempts by IT people to improve the software delivery process. Each one has been an every-growing vortex, pulling in more people from diverse teams.  The history of trying to improve the development process ‘from within’ started with just the developers, and then expanded to other development teams, QA, project management, business users, and now operations.  Each step of the process involved building bridges of communication and cooperation across teams. Each of these phases then went sour. They all went through a similar buzzword-ification, when a complex idea became over-simplified by a slick phrase.  Generally speaking, they all went through the same steps:

Some pioneering teams start to develop new processes to help them succeed in working together and with other teams.  Being analytical folks who can react and respond to new information, the most successful processes are developed over time with feedback from the other teams and a critical eye towards continuously improving the process.

  • Other folks notice this success and want to capture it, so the processes start to become more formalized and prescribed.  While the original processes were very customized, the industry starts to get a better of idea of what parts work well in more generic settings.
  • As the patterns become better understood, the repetitive friction-points are identified and companies begin to build tools to automate away that friction and give people freedom to focus on the core benefits of the process.
  • More people, looking for the quickest way to get the most value from the concept and begin to think that the quickest way to accomplish the change is to adopt the tools and formalized processes.
  • Eventually, the IT industry attempts to capture the buzzword-magic by bottling it as a set of processes and tools, missing entirely the real teamwork magic.
  • While lip-service is paid to the idea of communication, and cross-team meetings are set up, the valuable ways of enhancing communications take a backseat to the process and tools.  This is, of course, the complete reverse of the Agile Manifesto that drove the concepts to the great idea in the first place. This is a great temptation for the IT Industry as a whole because cooperation and communication takes a lot of work over time to build, and that is not something you can easily package up and sell in a box. 
  • The magic is dead. Complex ideas are reduced to a marketable recipe, and the Software Development industry tries to follow the recipe and so often fail to be effective with it, the ground-breaking folks who originated the ideas become disillusioned and are off to solve the next problem.

 Next Stop: DevOps

So what is this DevOps thing?  In its simplest sense, from the developers’ perspective, it aims to expand the development team beyond just developers and QA and PMs to include the operations and deployment teams.  The result is what you might call a “product delivery team”.

The first step, like all of the other previous steps, is to “stop being a jerk”.  Get yourself OK with that first, and come back when you’re done.  “But I’m not a jerk, some people are just idiots”.  Yeah, that means you’re still a jerk.  As soon as you find yourself saying or even thinking that someone on your team (or worse one of your users) is an idiot or a moron or useless and their job responsibilities are pointless, you have more deep-seated issues to work out first.  Before you can succeed, you MUST treat everyone on your team and other teams with dignity and respect.  And the hardest part about this is that you can’t just go through the motions, but deep down inside you need to actually believe it.  If you think that this is too touchy-feely, or you don’t think you can do that, or you don’t want to or you don’t believe that it’s necessary, that’s fine: If you’re determined not to learn to be a team-player, then don’t choose a job that demands team-working skills.  The idea of the "smart but difficult-to-work-with-developer” is a crock.  If you can’t work effectively with other developers and people on other teams, I don’t care how many books you’ve read, you will never be a good developer, in my humble opinion.

The next step is to actually talk the other people.  Recognize the fact that, as hard and important as your job may be, theirs is probably just as hard and just as important: The only way that your team is going to be more effective is if everyone works to make everyone’s job easier, so set up a meeting with them, and ask, with a straight face and genuine interest, “what makes your job more difficult, and what can we do to make it easier”: Then watch their face light up.  I guarantee that they have a list of annoyances, manual steps, and remediation procedures that waste their time every day; they will be happy to have an opportunity to gripe about them in a welcoming setting without having to worry about being labelled a “complainer”.  Examples would be “we don’t know when something is deployed or needs to be deployed”, “we don’t know what changes are including in each release”, or “every time I have to deploy something I need to copy files all over the place and edit some scripts and the dev team always forgets to include a certain file”.

Now, you will be tempted to smile and nod and shrug your shoulders and explain that it’s not a perfect system but it has worked so far.  Suppress this urge, write down the concerns, and start talking about them.  Throw around blue sky ideas of possible solutions.  Get an idea of not just want hinders them, but what would actually make them succeed. 

OK, now what is the official title of this part of the DevOps process?  There are probably several names, but I prefer “stop being a jerk, talk to people, and find out you can solve their problems”.  What tool should you use for this?  Usually Notepad, Evernote/OneNote, or a piece of paper, and an open mind.

Now, before you have finished talking to them, pick a few of the most irritating and/or easiest problems and promise to fix them.  It is a good idea to schedule a follow-up meeting a few days or a week or two ahead, before you even leave the room. In this meeting, you’ll need to show the progress you’ve made towards solutions and get feedback about whether it actually is going to solve the problem or what you might be missing. You could also take this opportunity, now you’ve given them something to visualize, to see if they’ve thought of a brand new process improvement that would make their work ten-times better.  It could even be that they now realize that what they first suggested was wrong, so maybe we need to try something else; this is not a bad thing, so try not to get frustrated if this happens.  Instead, repeatedly stress to them that your main goal here is to make their life easier, not just because they want to hear that, but because it’s 100% true.

Sound simple?  It really is.  But that is the core of DevOps.  If you do this first, everything else will start to fall into your lap.  There are a bunch of tools and procedures and techniques that you can leverage to solve the problems, but you need to be sure that you are actually solving the right problems, and to that you need to build a positive working relationship to help root out and identify the solutions to those problems.  The tools are awesome, but you HAVE to focus on the “individuals and interactions over processes and tools”.  But once you have this in place, you can do DevOps.

Again, it’s not about a tool or a process or a methodology. It’s not about cross functional teams or configuration management. It’s about development and operations and everyone else who is responsible for delivering and supporting a software system to come together, recognizing that they have a single shared goal of building and supporting the best software they can as efficiently as possible, and working together as a team to accomplish that.

Mike Mooney

Author profile:

Mike Mooney has been a software developer for 15 years, and lives in Chester Springs, PA with his wife and three sons, and co-founder and President of MMDB Solutions(http://www.mmdbsolutions.com) and his blog is located at http://mooneyblog.mmdbsolutions.com

Search for other articles by Mike Mooney

Rate this article:   Avg rating: from a total of 23 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.


Subject: Well Said
Posted by: Grant Fritchey (view profile)
Posted on: Friday, May 9, 2014 at 6:48 AM
Message: That is one terrific take on DevOps. Well done. I couldn't agree more. It really is all about checking egos on all sides in order to focus on what's really important, making the business better.

Subject: DevOps
Posted by: Thomas Gilchrist (view profile)
Posted on: Monday, May 12, 2014 at 8:46 AM
Message: Nice article, Mike!
I think that DevOps, if correctly implemented in the development process, can be really beneficial, though it's still not as popular as Agile or Scrum.
BTW, just published an article on this topic too: <a href="http://www.oxagile.com/company/blog/devops-your-team/">http://www.oxagile.com/company/blog/devops-your-team/</a>

Subject: Can you add some suggested reading?
Posted by: Alex_Kuznetsov (view profile)
Posted on: Tuesday, May 13, 2014 at 7:59 AM
Message: Mike,

I think this is a very useful introduction, but it would be even more useful if you suggested how to continue education about DevOps. What should we read? Which conferences should we attend?

I would suggest GOTO Chicago, which is just a couple of weeks away, and the speakers and presentations are truly outstanding.

Subject: DevOps born from Lean?
Posted by: sdmcnitt (not signed in)
Posted on: Wednesday, May 14, 2014 at 10:35 AM
Message: Thanks for this article.

I just read The Phoenix Project by Gene Kim and co.
http://www.amazon.com/The-Phoenix-Project-Helping-Business/dp/0988262592

It was an OK read (actually gave me nightmares) but seemed to imply that the DevOps movement was more rooted in Lean concepts than tracing its roots back to Agile Manifesto.

I am currently looking into DevOps and its relationship to Shadow IT. Your next article?

Subject: What it is to our boss
Posted by: Anonymous (not signed in)
Posted on: Monday, May 19, 2014 at 11:40 AM
Message: http://bit.ly/1mOfJhc
Apt jpg.

Subject: hated the article
Posted by: Anonymous (not signed in)
Posted on: Tuesday, May 20, 2014 at 7:09 AM
Message: Rated one star, because that was the worst rating I could give it.

You took what could have been an informative article about processes and communication structures regarding devops and turned it into some sort of adolescent rant about personality defects. They're two separate articles. The one I wanted to read (about devops) was not here at all.

Subject: Good, but DevOps is nothing new.
Posted by: Anonymous (not signed in)
Posted on: Tuesday, May 20, 2014 at 9:31 AM
Message: Good summary article, but you miss the fact that the great wheel of IT History didn't start with Agile. The quote, "All of this has happened before, and will happen again" is oddly appropriate here, as I will be citing the ancient times COBOL.

Much like the cloud is really just a modern incarnation of the classic Data Center concept popularized by folks like Ross Perot ( I am still waiting for a cloud provider to come up with the "innovation" of processor time slice charges), the concept of and the programmers who were of the right personality type to work what is now referred to as DevOps, were once known as Operations and Maintenance.
O&M was a good spot to put your new folks, and seasoned vets who were also reasonably good at working with the customer, who like creative problem solving and didn't really fit well into the structure of a straight ahead dev-team. The types who would end up throttling (or be throttled by) the design pattern/process advocates.
You never wanted your code monkeys, weekly bathers or people who were too hung up on the perfect design pattern for a solution to be in that group, because it's focus was on the app and the customer -- get it working, keep it working, and change cosmetics and if really needed do as minor a change as possible when the user demanded it by excessive whining or by contractual obligation.

For this team, duct tape, chewing gum and bailing wire are always valid solutions, and St. Angus MacGyver was their patron.

Think a ships engineer versus the ships designer.(LaForge/Scotty vs. Brahms for STNG fans)

The problem then as now is discerning what should be done by deep development, and what can be done quickly and pushed out to keep the system running, and the user as happy as possible.

The other major problem was, and will be -- making sure the product doesn't become stagnant, or worse yet something akin to a Frankensteins platypus because you are relying on this kind of team for your major developments -- this team will make it work, but there are times they really shouldn't.

Look back before agile, and before VB6, java and .Net, and you will find the way the never ending problem was solved last time brought about the advent of client/server which brought about the decline of mainframes in general, and the advent what we all know today.

As was said earlier "All of this has happened before, and will happen again".

Subject: WOW...Great Article!
Posted by: Anonymous (not signed in)
Posted on: Tuesday, May 20, 2014 at 3:01 PM
Message: Love what you said above.

"If you can’t work effectively with other developers and people on other teams, I don’t care how many books you’ve read, you will never be a good developer, in my humble opinion"

It truly doesn’t matter how much you read or how many tests you can pass if you can't play as a team member.

Great Article overall!

Subject: Re Processes and Communication Structures.
Posted by: Phil Factor (view profile)
Posted on: Thursday, May 22, 2014 at 9:59 AM
Message: It is difficult to write about DevOps without upsetting a few people. However, I like what Mike says here, because it does highlight, in a very direct way, the need for 'people skills'. There has been plenty of discussion about this in the DevOps community, because relating effectively with colleagues, and 'multi-skilling' in general, is easy for some developers and Ops people, but not others. We can't fix this by giving participants the security of processes and structures because we know from bitter experience that bloody-mindedness will always win. I'd hate to lose the talents of the many skilled developers who can't cope with intensive team processes such as Agile and Devops. I'd like to devise a methodology that takes humanity as it is, rather that how we'd like it to be, and finds a role within development to suit the individuals skills, sociability and temperament. There is always a need for a cubicle in the IT department.

 

Top Rated

Swizec Teller : Geek of the Week
 Why do programmers work best at night? Is this related to the idea that drinking alcohol improves... Read more...

Jonathan Allen’s Top 10 Favourite Simple Talk Articles
 We asked several well-known readers of Simple-Talk to write about their all-time top ten favourite... Read more...

The Proposals Conundrum
 When you work for a small software development (or any services) company, one of the major challenges... Read more...

David Heinemeier Hansson: Geek of the Week
 Ruby on Rails, the open-source web application framework, grew out of David Heinemeier Hansson's work... Read more...

Alex Payne: Big in the IT Business
 Alex Payne worked on developing Twitter for three years. When he started, it was a small side-project:... Read more...

Most Viewed

The Future of Reflector
 Simple Talk asked freelance writer Bob Cramblitt to sit down with the two people behind the agreement... Read more...

Linus Torvalds, Geek of the Week
 Linus Torvalds is remarkable, not only for being the technical genius who wrote Linux, but for then... Read more...

Bad CaRMa
 From hope and euphoria, to desperation, firings and the ultimate demise of a company. Tim Gorman charts... Read more...

Driving up software quality - the role of the tester
 Have you ever wondered what a software tester does? Helen Joyce, test engineer at Red Gate software... Read more...

Don Knuth and the Art of Computer Programming: The Interview
 Fifty years after starting the 'Art of Computer Programming', (TAOCP), Don Knuth is still working hard ... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.