to talk this morning about why
agile software development works not just how because you can go out in the
world and find a lot of books about agile software development but all of
these books are about how agile software development works but they don't really
focus much on why it works and those of us who've been following agile software
development for a while have are very interested in why these things work
because some things in agile software development are very very non-intuitive
they don't seem to make sense until you've tried them and for some of the
really key practices and agile software development we have essentially shown
that they work and then reverse engineered why they work and a lot of
what Martin are going to talk about this morning as why these parts of agile
software development work so very very well we both work for Thought works and
we were well known agile software development company and we've proven to
ourselves for more than a decade that these things work really well and so
what Morton are going to talk about this morning really is the essence of agility
and this is the first part of the essence of ability when i passed over
mine
so when that foot works we started working with agile software development
are really what we saw was just little bits of information and we saw it was
really hard to get people to take agile software development seriously over the
last 10 years we have seen that steadily change and now in many parts of the
world Angela seananners if not V mainstream are certainly a mainstream
alternative and we see major conferences we see certifications we see a lot of
these trappings of a more mature approach to developing software but in
the course of all of this development
one of the things that has also happened is that we begin to see that a lot of
the things that people
say about agile software development no longer ring true with what many of the
original developers of agile software felt and this is in fact a very common
phenomenon but affects all sorts of different things and it's something that
i referred to a semantic diffusion the original ideas as they get passed from
person to person to person
the ideas get more and more diffuse more and more fuzzy and as a result we often
find ourselves now going into an organization that he's been doing agile
perhaps for several years and we go in and we look at what they're doing and
it's nothing that we would recognize as an agile approach and this is a problem
because it means that people are not really understanding the opportunities
that this technique has for them and therefore they're not able to truly
judge whether it's applicable to their situation now semantic confusion as I
say it's a common problem with many things and and really one of the key
things you have to do to deal with it is to just constantly remind yourself what
is the call
essential ideas that you can concentrate on now in my case
understanding that comes back to an essay I wrote back 10 years ago right at
the beginning before we even came up with the word agile called the new
methodology and if you're interested you can go to that URL or just google for
new methodology and you'll find it
and in that essay I really highlighted what I saw as two key distinguishing
features that differed made the differences between the agile world and
the more traditional plan driven world that was the main context at the time
now the first of these was the attitude towards planning in the traditional
notion of plan driven software development planning is seen as an
activity you carry out before you do the actual work that you going to do
there's a english phrase and i have up here
plan your work work your plan you plan first and then do you think
essentially this is a predictive approach towards planning and it is
predictive because the idea is that the plan is a prediction of how the work
should go and it's also very much affect your success criteria because to be
successful the measure of success in a plan driven world is to say that you
have actually carried out your work
according to your plan and so when often you hear people saying we had a
successful project because it was on time and on budget
that's a predictive mindset we succeeded because it followed the plan now this
kind of predictive approach is useful in many many situations and but one of the
things that he does have in terms of software development is that it depends
on the fact that you're able to get a clear and stable set of requirements
because if you can't get clear requirements stability then everything
else that follows from the plan is not going to be stable either
and so not just as your plan depend upon your requirements being stable but your
entire software development process depends on your plans on your
requirements being stable and herein lies the difficulty because whenever i
go to software projects I ask these questions are your requirements table
in fact i'll ask you this question now how many of you are working on software
where you've got requirements that are unstable but the you but things are
changing
how many i find themselves in that situation as usual
it is most of the audience how many have stable requirements everything solid are
not shifting there is usually one
but there are a few people who didn't raise their hands at all
so the plan driven community knows this and so I say well we need to stabilize
our requirements and they come up with all sorts of techniques that can be used
to do that in order to do that kind of stability but we find that this is
actually really kind of heart and this is the first shift in the agile world
your world says well if we need we the situation where our entire software
development processes do is dependent upon having stable requirements and we
can't get stable requirements
well maybe what we should do is get rid of a dependency break that dependency
and come up with a software development process that is tolerant towards changes
in requirements and changes about how things go
and that is one of the core distinctions but he goes a little bit further in this
phrase that I really like from Mary Poppins dick
what she's saying here of course is that not just should we be tolerant of
changes of requirements we should actually see them as an advantage and
the fact that we can change is a big plus for us and this is very very
central now the question is how do we do this
well it means the basic idea of planning that we do some planning and we execute
that plan in the rear agile world we switch over to what we call an adaptive
planning approach which is that we go through this cycle of plan and execute
many many times on a project
so you will see every week every couple of weeks we replan readjust based on
where we are
and furthermore we look to release the software at frequent intervals
one of the primary things that we find we are doing it for works as we come
into an organization is not just do we have to get the development side working
but we also have to get the whole or
as a tional deployment and delivery into a cycle so that we can put software into
delivery every couple of weeks or so that change causes a great number and
profound shift so the first shift from between agile in a traditional world is
this switch from predictive planning to adaptive planning now one thing I should
point out in order to make adaptive planning work you have to change your
approach and you're thinking towards software design and to come up instead
with an evolutionary approach to software design but the design is
something that is also constantly changing and evolving as you go
now I don't have time to really talk about how you do that
there's a whole bunch of techniques that are out there that enable this kind of
stuff to happen and there's another paper i can point you to called is
designed dead and another thing i wrote ten years ago that really talks about
this change in the nature of design these technical practice is a very
important part of a giant one of the common problems we run into when we see
companies struggling with agile is that they take the project management stuff
but don't take the technical stuff and they get into a great deal of
difficulties by doing this because they don't have software it's designed for
change
this is particularly a problem with people who say they're following the
scrum approach although it's not inherent in scrum it commonly goes with
that
so if you are doing it an agile approach or considering an agile approach
make sure you pay attention to these kind of technical things as well so
that's our first distinction adaptive this is predictive planning
so for the second distinction
I want to show this rather old photograph does anybody recognize who
this is not not forward but your philosophically very close to him
Taylor yes Frederick Winslow Taylor one of the most influential people about who
has affected life in the 20th century perhaps more than anyone else and his
great phrase was this one here really saying that in the past people who are
doing the work would decide how to do to work and that should his view was that
should change but instead a separate group of people should decide what
process people should follow and then people should conform to that process
it's an approach often referred to as scientific management
it is strongly influenced the whole way we think about working how we do work in
the 20th century in the terms of the software process you can think of it as
saying we have process designers who come up with a software process that
people should follow and then when we actually want to apply this to a project
we go out and we find some people who fit into the rough categories of the
process and then we slot the mean to the various places within the process and
that is the traditional approach towards thinking about process now here for the
shift i have a slightly longer quote for you to read
now the point of what Alastor is saying here is that we tend because we're
computer people to be used to manipulating these highly precise things
called computers and plugging them together
in fact I saw people give talks with I said software process is software - we
can treat the way we build software like a computer program but the problem is
the key things or do things the key actors in software process our people
and they are not predictable they do not act in this very circumscribed linear
way and so his viewers and this is very much the key agile notion is that
instead we need to think differently when we have people as primary actors in
the process and in fact rather than coming up with a process and fitting the
people into it
we reverse things we get a good group of people who can work effectively together
and then they choose the process that they follow and so that is a second
shift shift from saying the process comes first and the people fit in to
saying the people come first and they decide what process is the most
effective for them to follow because even if you get good people
it's been observed that you will often find that a process can often stifle
people and stop them working very effectively
so it's very important that the software development team choose their own
process and that to me is the essential difference between agile and plan driven
approaches the attitude towards planning and the attitude towards people
ok the next aspect of why this works is communication
so let's talk a bit about communication as it applies to software development in
our world communication Trump's technology meaning that we view
communication as a more important aspect of software development that we do
technology now I'm sure no one in this room has ever been on a failed software
project but if you would for just a moment imagine what a failed software
project might be like so fantasize for a moment about what that might be
and think about did it fail because of the technology reason or did it fail
because of a people issue the the famous book people whereby DeMarco and Lister
have a quote that says it's always a people problem in software people always
end up being the most important thing and communication between people becomes
a really really important thing and part of the reason for this is that we are
highly social creatures
notice that the worst punishment in prison is solitary confinement the worst
punishment in prison is being taken away from the other murderers and rapists and
thieves and put by yourself
that's how social we are we would rather hang out with really undesirable people
than be alone and this is reflected in some of the literature in the agile
software world
mr. Coburn has this great graph he's done an analysis of the effectiveness of
community of communication channels and he models it is similarly to the way
that you would model a heat transfer from a heater or air conditioning duct
the further away you are the less effect that you get and what he says in this
chart is that the most effective way that you can communicate
aight technical information as to people are white board because you can see each
other you can see body language you can hear tone of voice you can see facial
expressions you can ask questions and get responses you can draw pictures
really really super effective way of communicating a very rich communication
channel but if you go to the opposite end of the spectrum
you see two people on email or or two people on paper which doesn't even allow
Q&A
this is exactly why trying to come up with a really elaborate set of
requirements and shipping them six or seven time zones away to another country
and expecting them to produce software with no further input is doomed to
failure
software is a very highly nuanced thing and trying to communicate all that
nuance in a very flat piece of paper or an email is never going to work
particularly well which is part of the reason why all the early outsourcing
efforts were not particularly successful because you cannot specify enough on
paper to allow them to be able to create something and you break that feedback
communication loop with those developers who are not isolated
many many time zones away that's very common in the United States for some of
the states now have a law that you are not allowed to speak on a cell phone
while you're driving is that true here it is ok
have you ever wondered why now certainly there's a little bit of fiddling with
buttons but i'm sure most of you here have broken the law at some point and
talked on a cell phone while driving
have you ever noticed how distracting it is just the talking on a cell phone is
much more distracting and yet i don't know of any state or country that makes
it illegal to talk to the person in the seat next to you when you're driving and
part of the reason it's so distracting talking on a cell phone is because huge
parts of your brain are trying to reconstruct the missing communication
channels that exists when you have a face-to-face communication
you don't even realize that those parts of your brain are engaged
but they're engaged in there working very hard because the only communication
channel you have is this staticky voice over your cell phone and you're trying
to recreate all the other parts of this communication channel and of course the
early founders of the agile software movement including Martin realize the
value of communication because if you look at the agile manifesto
there are 2 items on here that specifically call this out as an
important feature individuals and interactions over processes and tools
which is not to say that we don't like the things in the right but we like the
things on the Left better and customer collaboration over contract negotiation
notice this is very much a face-to-face communication rather than try to specify
everything out in infinite detail on a piece of paper somewhere and there's a
an interesting story that i can tell based on some project work that thought
works has been associated with and this happens in the African country of malawi
i'm always a very very poor country
in fact they have a very little electricity they have very little
infrastructure and they have these medical clinics that they have set up
these practitioner clinics and have nurses who work there who have gone
through a short training . and the villagers walk for many many meters
kilometers to get to these sites that some will start before daylight and
walked for several hours to get to these clinics and the essentially in the
middle of nowhere and two for health issues and one of the big problems in
Malawi is malnutrition and so these nurse practitioners
one of the things they do is try to gauge whether the children in this
particular rural area or malnourished or not
and they've done this for many many years but part of the problem was that
the collecting all these this information this data and on paper and
then shipping it to a central office
but of course my love how he doesn't have a particularly good infrastructure
so it takes a long time for that information to reach the government
where it kind of stacks up and piles of paper
waiting for someone to come along and react to it and the problem of course is
it takes so long to react to it that there will be really serious outbreaks
of disease and malnourishment that go unnoticed for weeks at a time because
they don't have timely information about how this works
unicef looked at this problem and they and they ask themselves is really
difficult question how do you automate something in a country that has very
little electricity but they noticed something interesting
the people in malawi are very very social in fact the penetration of cell
phones is more than the penetration of electricity
nokia makes a special model of cell phone that keeps a charge for three
weeks at a time or more
the the people in malawi literally walk to the village to charge their phone
because they don't have electricity and they buy prepaid phone cards so they can
keep in contact their family and UNICEF notice this and realize that this was a
useful IT infrastructure cell phones and in particular
SMS on cell phones and so they set up a project called rapid SMS that allows the
nurse practitioners to could construct a very specifically formatted SMS message
that has information about the child is standing in front of this person right
now
now by using this infrastructure they can get almost immediate reports from a
region and be able to collect that information and react to it quickly
enough so that it makes a difference and that is that time that makes all the
difference
and this project is an open source project called rapid SMS and you can
contribute to this city life it's a purely open source projects written in
python using the Python web framework called Jango
in fact thought works at our last North American away day made some
contributions to the rapid SMS project as an open source project because we
support but the kinds of things that are doing they're finding the correct
communication channel was the key to solving this problem
it turns out that shows up a lot and agile software development as well
finding the correct feedback
click finding the correct communication channel sometimes makes all the
difference
which brings me naturally to the next topic which is about feedback and Martin
is now going to insult your French cuisine
no I don't think i'm going to insult it i'm just going to show it to help
illustrate I'm the role of feedback in software development
I'm gonna mention something what i consider to be a very high point of the
world gastronomy which is my fruitcake that I make
see I told you he was going to in soldier now i'm not i'm your French do
great food i mean i consider friends to be the second greatest country in the
world
oops but now
fruitcake is part of this story the other part of this story is taking a
shower and typically in a hotel room but you've never been in before
so what's the difference between taking a shower in an unknown hotel room and
making a fruit cake
well this has to do with two ways of feeling thinking about process and this
is a distinction that was made by cash way back
I'm right back in the early days of the agile world essentially said that there
are two forms of process out there
the defined and empirical process and the fruitcake and the shower or examples
of these two so let's begin by saying well in a very simplified view of any
process is to say well we have certain inputs and controls
we have a process that we do and a certain outcome that we expect from the
process when we were working in the world of a define process such as a
fruitcake
what we do is we understand the process really well I know how to bake the
fruitcake I've done it many many times
so I get the right amount of ingredients are combined them the same way each time
I put them in the oven for how long i know it is to be and out of the time
every time comes that same fruit cake and that is capable of keeping me going
from
that is the defined process
it's well understood i can set everything up in advance
let it run I get the output when I take a shower in an unknown hotel room
I use the other kind of process the empirical process and that's because I
don't really know where about dial is in terms of the heat of the shower and so
what I do is I will run the shower for a bit
stick my hand in is that too warm is it too hot adjust the shower appropriately
I get in maybe but ok a beginning but as I warm up i want to share a bit hotter
so again I will adjust the shower so what i'm doing here is i'm looking at
the output inspecting that output and then adjusting the inputs based on what
the outputs are and that's because of the fact i'm dealing with something but
much less known
ok it's still a relatively simple process i really have only one dial to
touch but I don't know in advance
exactly where that dial will be and that is a difference between these two things
now the key thing that can sway brought out from this he came up he understood
these ideas for interactions with process engineers from the chemical
industry
originally I can had been a developer of software to support traditional
approaches
he had clients like coopers & library and who he would sell software for to
support their software development process and he wanted to understand
better how software development processes worked and so he went to
dupont to talk to their process engineer people to get a sense of what was going
on and that's where he picked up this ID this this notion of empirical and
defined processes and the key point is you have to use the right kind of
process for the definition for the right kind of process
so if you have a chemical plant and you know all the inputs and your how
everything operates well then you can use and
would use to define process but on the other hand there are unknowns and
uncertainties involved then using a defined process is actually quite
dangerous
instead you have to choose an empirical process and this of course is how it
notions of software in software as i mentioned earlier on we have all of
these unpredictable nonlinear components called people running around so you
can't use a defined process effectively in those circumstances
instead you need to use an empirical process and that means you need to have
some way of inspecting the output and you putting it through in a feedback
loop to affect how you doing things and that's why feedback is such a central
part of so much agile thinking
whenever people and our colleagues go out to clients were always looking to
the ways in which to introduce feedback loops into a software development
approach
so I want to talk a little bit about some of those feedback loops that we
introduced
there's a lot more than I've got time to mention but i'm going to mention a few
of them so they're to illustrate one of them
i'm going to show you our office building in Beijing
we only have one floor of this office and the interesting thing is that you
can tell from the street which floor it is but it is the fault works law and you
can see this perhaps a little bit if I kind of zoom in a little bit and you
notice what one of these flaws
the windows are covered with little blobs and things and that's post-it
notes stuck onto the walls go into the office and use what you see is post-it
notes covering over the windows and that is a demonstration of how we like to
feed back the project plan of how we work going to any fault works project
almost anywhere around the world and you will see card walls on this wall which
show where we are in certain part of the project various tasks
that we need to carry out I'm captured as user stories will be stuck up on the
wall and that position on the wall indicates how far we are if we decided
start work on them yet
are we working on what stage in our little workflow and what they go through
and you can walk in and pretty much by going up to the card wall you can get a
real sense of what the team is actually working on but where this is
particularly interesting is that this is not something that's only visible and
only relevant to the managers of the project instead it's visible for
everybody and this is one of the core elements of the planning approach but we
encourage is a planning isn't just something that's visible and
understandable to the project manager
it's something that's visible and understandable to everybody
everyone can see where we are when I used to go into traditional pro software
projects and look at the project plan
it was something very complicated and something that most developers on the
team didn't care to want to understand and then you it was all fiction anyway
but with up an agile project you want everybody to be participating in that
plan
so it's very important that everybody can see what's going on and everyone
understands what's going on
and this is even the case if you use electronic tools
here's another team room this one in atlanta and a project that we've been
doing there for several years and here we may not have so much the physical
card walls
we have electronic ones and what you see other projectors here
projecting on to the screen and what they're showing is the card wall from
our project collaboration tool called mingle and that gives us a virtual card
war
this is something that's also very important in offshore projects where
we're collaborating between India and Europe we need to be able to see the
card war everywhere that matters
and so we have an electronic card war but mimics the physical one that we use
when we're all together as a team
most teams will prefer
physical card war because it's feels closer and you know you can shift the
cards yourself but an electronic one is often necessary if you have this degree
of distance
now another important part of a feedback loop and that we see is knowing where we
are in terms of the build system
one of the key elements of agile software development is we have this
notion but is only one way of measuring progress and that is by working software
as we build pieces of working software and we get them into a production-ready
state we can see our progress
no other indicator is as important as this and that is back again because of
this notion but we have to be sure about where we are in the software development
approach
so as a result we find that the build process becomes very important we always
have to know those are software integrate does it build well what stages
does it go through in the various gates of acceptance testing to the point at
which it is a production-ready some projects early on like to use green and
red lava lamps to show whether the build was okay or not and we often come up
with quite elaborate
I'm techniques for showing where we are in the state of the projects build this
one here
we're using something that is a more computerized display but again this
notion of where are we in the process and so that when the build breaks
everybody can tell and everybody can do something about it
this gives us a feedback loop as to the help basic health of the software and
that it's worth mentioning this particular view is a tool called CC
board which is an open source tool
what you're seeing here the status of multiple builds on that project and
green of course means that the last bill successful the darker green that you see
pulses because it means that I'm now building and the last
it was successful pink means that the last bill failed and red pulsing means
that i'm currently building it in the last build failed
this is a pretty common at a thing that we do a lot on in the software
development world to try to support agile software development we created
this is part of Project Work
but then we open sourced it so that other people can take advantage of it
and thought works as a company has a very long history of open sourcing these
infrastructure elements to help support agile software development like
continuous integration cruise control like testing tools like in unit and
selenium
this is a as I said a tool called CC board which is basically a front in on a
continuous integration server so that you get one nice comprehensive view of a
large number of concurrent builds that may be going on to support different
things in the project
this notion of using testing as a feedback loop is a very central part of
things that we constantly questioned whether we're actually building software
that works effectively by layer upon layer of testing in early stages we want
a very fast testing loop so we use fine-grained unit tests to test things
very quickly and then as we go further down the pipeline
we used longer and more forceful tests end-to-end acceptance test performance
tests and the like to get more and more sure of a sock software quality of what
we have and we do this right from the beginning of the project so we have a
constant feedback loop as to the quality of the software and river it's
appropriate enough and that again is the use of feedback
now these feedback loops are things that are very broad in terms of the whole
project team the next feedback loop i want to look into is something much more
and close involving the way in which people collaborate and work together in
particular around the notion of of one of the more controversial aspects of
extreme programming which is that a pair programming and i just want to point out
now
a little marker for the future but it will be important for us to explain
exactly what a plastic kangaroo
is doing on the project team and Neil will explain that later
critically important i'll get to that of it so pair programming here we have a
couple of software developers actually from a while ago and thought works who
are working on a piece of software together and this is a thing that we now
have used widely at footworks pair programming is very controversial early
on and there we use it very heavily
there are many reasons why it's effective but I'm to illustrate one of
them i would like to show you to my favorite places in the world
this is the first i'm not too far from here
hopefully many of you have visited it's in the Loire Valley
I visited a couple of years ago one of the most interesting and beautiful
gardens in the world designed very much to our follow the french style of
Renaissance Gardens very formal and at the same time very beautiful
the finger i particularly like about this shot is that it is the vegetable
garden it's the working vegetable garden and the vegetables are used in the
restaurant at the Chateau and yet at the same time they are arranged in a way
it's very aesthetically pleasing another garden i like very very much
and quite a different garden is this one in heligan in Cornwall England
it's a garden that was built originally in the 19th century and then he became
lost because and the family that owned the estate died out the estate got into
a legally strange state and as a result the garden got all overgrown and it was
effectively lost and then about 20 or 30 years ago some enthusiasts managed to
raise money to buy the estate and they cleared away a lot of the stuff that a
grown up over time and restored the gardens to what they were like in the
19th century this particular section of the garden is referred to as the jungle
cornwall have
as these areas where you get quite a cleft in the valley and that gives you a
microclimate that allows you to grow plants that you can't grow otherwise in
England
so this area of the garden has a lot of tropical plants and the you just
wouldn't be able to go anywhere else
now these are two very different gardens one is very ordered straight lines
geometrical shapes the other looks like as if it's supposed to be completely
natural
I mean it's very unnatural those plants are not native at all to England but
when you're walking through the jungle
you have this feeling that you could be just in nature and what we see is that
these two gardens are appealing to different parts of the brain
you may have heard that in the time that the brain
you can have divided into these two processing hearts and the left side of
the brain which is the more logical side which is all about logic and order and
and and things being arranged into nice patterns and the right side of the brain
which is more holistic creative
I think of it as the romantic side because I think of it in terms of
romantic poetry or romantic music this kind of way of thinking about the brain
of course is very common one book that I very much recommend if you're interested
in in thinking about how the brain works in terms of software development is this
one here by and hunt but in going back to our notion of the brain and how it
divides into the logical romantic parts
how does this plane to pair programming well it plays into it because of the
fact that when you see two people programming at the computer like this
what's happening is a different parts of the brain are taking priority in the two
participants
typically the person who's typing on the keyboard there in the ville and Ray part
of the brain the logical part they're thinking about how do I make this
particular piece of software work how does it fit together and they're
thinking in very calm
crete logical steps but the person who is participating with them often
referred to as the navigator
as opposed to the person on the keyboard being the driver that navigator their
brain is in a much more elegant mode
they're thinking in terms often of broader pictures and they are often able
to ask the why that goes with the person on the keyboards how
and this of course is a very important part of development often in terms of
software development when you're focused on trying to get something work
you often can miss the fact of whether you're even taking the right path to get
there and he needs somebody who's just a little bit detached who isn't
concentrating on exactly how to fit the pieces together
who is able to say hang on this is the whole wrong approach that we're taking
here we need to take a different and easier path
what this means then is that because you can only use one half of your brain at
the tire time it's almost like you have a 2 processor but only one bus
so the only way to apply one full brain to software development is to usual to
half brains at one time so as a result with pair programming
it's not so much you've got two people working at the same time on the same
piece of work you do you actually got one full brain operating and have a last
part of of the feedback loop and the dog want to mention for agile software
development and comes back to this lovely quote from Kent back
perfection is not something we do but somewhat weak not something we achieve
but something we're constantly doing with constant trying to perfect what we
do and so we see these feedback loops in terms of software development with pair
programming which is a continuous review process in the planning approach because
we're constantly seeing our plan and seeing how it works in the testing
approach which is constantly checking that software is healthy but it also
approach applies to our entire process of software development
so a common thing you see on agile team
is that every so often usually every week or two we do retrospectives where
we look at a whole software process and we say how well is it working
what are the things that we are doing well what are the things that we're
struggling with what are the things are puzzling us and how do we change our
software process in order to take advantage of these things and this is
even
nothing brings more this notion of the team owning the software process not
just as the team decide how we're going to carry out our software development
process
the team decides regularly
how to change it and that was a result we see a constant tweaking and changing
of the software process as a phrase I like often like to use it saying if a
team is doing extreme programming
