What is agile software development?
Agile is an approach to work. Agile software development refers to a collection of methodologies and frameworks that promote adaptive planning, evolutionary development, and continuous delivery. Instead of focusing on a final output — the finished product — agile teams prioritize working in short increments with frequent release cycles to deliver value faster.
Software development is an adventure. And like most adventures, it is unpredictable. That is why agile philosophy centers around rapid iteration — solutions evolve organically through cross-functional collaboration between self-organizing teams. Customer satisfaction is on the horizon, with their feedback incorporated throughout the development process.
Every organization will practice agile in its own way. Company culture, industry, and functional discipline all play a factor. And some groups simply gravitate towards one methodology more than others. Software developers, for example, often prefer scrum. Whereas IT groups who manage long lists of customer tickets or issues may prefer kanban.
But other functional groups within an organization find value in agile practices. Openness to change, trusting collaboration, customer-centric thinking — these concepts easily apply to other types of work. You may have seen agile principles infused into content marketing, project management, and product management.
This guide is meant to give you an overview of agile software development — its history, evolution, and some of the challenges organizations find when implementing agile. But it is not a tactical primer for companies that are looking to overhaul workflows and “go agile.” Those types of transformations are sweeping in scope and require deep thinking about what needs to change and why.
Let's explore the following:
Remember that early agile practitioners did not prescribe a specific framework or doctrine. The goal was to empower teams with a mechanism for responding to change quickly and continuously improving. There is no “right” way to weave agile thinking into your team’s processes. But in the spirit of unencumbered innovation, we suggest taking what you need and leaving what you can.
What is the history of agile development?
Agile development is older than you think. Kanban was conceived in the 1940s. Iterative and incremental development (IID) dates back as early as the 1950s, with evidence of use by teams at NASA and IBM Federal Systems Divisions. Other adaptive methods such as Evolutionary Delivery (EVO) and Rapid Application Development (RAD) were in use through the 1970s. However, it was not until the late 1990s that agile gained widespread traction. So what pushed what we now know as agile to the forefront? Two things: Backlash against bureaucracy and colossal business change.
For decades IID, EVO, RAD, and other adaptive methods were seen as fringe and not taken seriously. Most organizations relied on the single-pass software development lifecycle known as waterfall. Waterfall defines requirements upfront and delivers projects in sequential phases. It can be effective for projects with a fixed scope, but vulnerable to failure if requirements change during development. And as anyone who has worked in technology knows — it is nearly impossible to anticipate all requirements at the outset.
The rigidity of single-pass software development created breakpoints between management and engineering teams. Do you remember the beleaguered engineer from the comic strip “Dilbert”? First published in 1989, the comic’s satirical plot points encapsulate many frustrations shared by developers who felt stifled by office politics and top-down micromanagement. The main complaint was that busy work and process were prioritized over productivity.
Around that same time, new technologies emerged that enabled faster product development. Widespread access to the internet disrupted entire markets. As new software-based products and services gained traction, expectations began to change. Accelerating time-to-market became a business priority as companies struggled to survive in highly competitive marketplaces. Customers wanted products to be continuously improved with new functionality. And they wanted to be involved in suggesting what that new functionality should be.
With all of this percolating, a group of 17 software practitioners gathered in Utah in 2001. That meeting is the one most widely reported in historical accounts of the agile movement. But the group had met in Oregon in 2000 to discuss what they called “light” or “lightweight” development methodologies. No one was particularly comfortable with the potentially negative implications of “light” and so the word “agile” was agreed upon when they gathered the following year to formalize what became the Agile Manifesto.
Although there have since been many independent signatories, these are the original authors of the Agile Manifesto:
Arie van Bennekum
Robert C. Martin
It is worth noting that many of the original authors had developed their own methodologies such as Extreme Programming (XP), Crystal, and scrum by the time the group gathered in 2001. These methods, coupled with other novel approaches to programming drawn from the past, were an ideal foundation for developer communities ready to adopt a new mode of working.
What are the values and principles of the Agile Manifesto?
The authors of the Agile Manifesto defined four core values and 12 principles. Those values and principles are meant to guide how agile software development teams should approach their work. Together, the 16 tenets serve as the foundation for many of the agile methods teams use today — with special attention paid to flexible planning, efficient communication, very short feedback loops, and adaptive cycles:
“We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
We follow these principles:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within the development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity — the art of maximizing the amount of work not done — is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”
What is an agile philosophy?
There is Agile and then there is agile — with a little “a.” Some believe that unless you are following a certain framework or set of rules, that you are not really agile.
Method tailoring, process pragmatism, and a comfort with the messy reality of how people plan and deliver should all be welcomed by agile thinkers. What matters is embracing a flexible mindset and continuously seeking ways to improve how work is done so you can maximize the value you give to your customers.
Merriam Webster defines agile as “the ready ability to move with quick easy grace.” Any team would be happy to be described that way. And ultimately, each team must choose their own path towards quick and easy grace — regardless of methodology.
What are some agile frameworks?
Agile is an umbrella term for a philosophy that you can apply to a variety of different software development methodologies. So when we refer to an agile framework, what do we mean? Does each one adhere to the Agile Manifesto? Not quite.
All agile frameworks are rooted in the values and principles behind the Agile Manifesto. But some were defined prior to the 2001 manifesto and others came after. While each approach provides a unique set of workflows and practices, together these frameworks embody elements of the philosophy behind what we know generally as agile software development. And as teams continue to discover new and better ways to work together, new frameworks will emerge.
By far, the most popular framework among agile teams is scrum. But the table below provides an overview of a few more you may or may not have heard of.
Agnostic agile practitioners might balk at being included in a list of agile frameworks, since one of the key principles is not to adhere to the specifics of any one framework. This approach is about leading with agile principles and then choosing any framework that best serves the needs of a specific organization.
Adaptive software development (ASD)
Adaptive software development is based on RAD. It focuses on the rapid creation and evolution of software systems. This approach repeats a series of speculate, collaborate, and learn cycles. Key characteristics of an ASD approach are that it is mission-focused, feature-based, iterative, timeboxed, risk-driven, and change-tolerant.
Crystal is a collection of agile software development approaches. Crystal practices iterative and incremental development, active user involvement, and delivering on commitments. This approach emphasizes the importance of people and process and the interaction between them.
Dynamic system development method (DSDM)
Dynamic system development provides a four-phase approach for implementing software projects. These phases are feasibility and business study, functional model or prototype, design and build iteration, and implementation. This approach uses RAD techniques to accelerate the software development process.
Extreme programming (XP)
Extreme programming sets specific practices for software development. XP characteristics include sitting together, pair programming, test-first programming, continuous integration, collective ownership, and incremental design. This approach is intended for small, co-located teams working with dynamically changing software requirements.
FAST is an acronym for fluid scaling technology. This agile framework puts the emphasis on self-organizing teams (or “tribes”) who form, change, dismantle, and reform dynamically over value cycles as short as two days.
Feature-driven development (FDD)
Feature-driven development is a model-driven methodology intended for larger teams working on a project using object-oriented technology. FDD defines five processes: develop the overall model, build the feature list, plan by feature, design by feature, and build by feature.
Kanban is a method for managing the creation of products with an emphasis on continuous delivery. Kanban uses a pull-based workflow system to help teams work together more effectively. The three core principles of kanban are to visualize what you do today, limit the amount of work in process, and optimize flow.
Lean software development (LSD)
Lean software development applies lean manufacturing principles and practices to software development. The key principles of LSD are to eliminate waste, ensure quality, create knowledge, defer commitment, deliver fast, respect people, and optimize the whole.
Scaled agile framework (SAFe®)
The Scaled Agile Framework® is typically used by larger organizations who want to apply agile and lean principles at scale. It offers guidance for action across three main levels of an organization: team, program, and portfolio. An optional fourth level addresses organizations working with value streams.
Scrum is an iterative and incremental agile software development framework for managing software projects. Teams work in time-boxed sprints of two to four weeks and follow scrum ceremonies — such as sprint planning, daily scrum, sprint review, and sprint retrospective for inspection and adaptation.
What are some common issues when implementing agile?
Let’s start with some of the common tactical issues that development teams face when implementing agile:
Fixed time, resources, scope, and quality
Lack of sponsor support
Lack of overall product design
Product owner role is not properly filled
Scrum master is also a contributor
Problem-solving in daily standups
Attempting to take on too much in an iteration
Adding stories to an iteration in progress
Adding unrelated tasks to sprints
Allowing technical debt to build up
If you have experienced this hit list of pain before, then you likely know that agile itself is rarely the issue. That is because (as we have reiterated a few times so far) agile is a philosophical approach to work.
Some organizations choose to slowly wade into the agile world by choosing a hybrid approach. This refers to an agile framework combined with a traditional waterfall (so-called “wagile”) or project-based development structure. Hybrid agile can be a helpful transitional path and some companies may find that it is a lasting solution for their teams.
Some even go so far as creating their own agile frameworks. UnitedHealth Group established the Optum Scalable Agile Method (OSAM), which is a modified application of SAFe®. According to a slide presentation which shares some of their challenges implementing agile at scale, one of Optum’s largest agile portfolios has more than six release trains, 35 scrum teams, and hundreds teammates. OSAM offered flexibility to evolve a massive organization over time.
Others face challenges when trying to adhere strictly to one framework. So you may find that your team does better with a blended approach. Blended agile refers to two or more established agile frameworks used together. For example, “scrumban” blends the structure of scrum with the flexibility and visualization of kanban.
In order to implement agile, you have to fundamentally change the way people think, operate as individual contributors, and collaborate as teammates. The larger and more established an organization is, the more difficult it can be to change “old guard” mentalities and processes. The collective aim should be to critically assess your approach before you venture into the details.