ood morning i'm so glad that you made it up from the morning after some
patties day that wasn't a good choice was it yeah
and so those from outside Boston welcome to our fair city and you're getting at
least a glimpse of what we have to deal with for the last month in terms of
temperatures
so we're going to talk about agile and architecture and this is a topic thats
it's been a long-running topic and that right from the very beginnings of the
agile software movement it was a question of what kind of role is there
for architecture and back at the very first agile oriented conference in two
thousand actually talked on that topic concerning a very easy role for design
as I called it but it's different to what we would normally expect and in
order to really understand that we've really divided are talking to two main
parts in the first part will actually talk about what on earth is architecture
in the first place
and in the second part will focus more on the question of how do you ensure
that architecture is happening on your project
so like Martin said agile as as a practice and principles have been around
for a while and the principles have been developed into several practices that
focus
maybe in the team space a lot of things around
user stories are paired programming but when you bring in a lot of teams that
are doing agile and being agile
how do you make sure that architecture is still happening
so first the definition of architecture on a whole just as a definition
it deals with organization planning having course structure structural
elements but it's also a kind of troubling word I've tended to actively
avoid the word architecture a lot in my writing I tend to use design instead at
fault works we're very nervous about talking about architects and
architecture
very few people like to the architect on a business card and that's partly
because of the nature of what we often run into it particularly in larger
companies where people with the word architect on their title often seem very
divorced from the realities of software development on a day-to-day basis and
but that doesn't necessarily have to be the kacks so when I think of
architecture
I think of a few things usually when we are talking about architecture and we do
talk about it
we tend to discuss things that are hard to change things that might be expensive
expensive in ways of money or or expensive in hard to change
yeah and this actually goes to a about particularly memorable discussion about
what the nature of architecture is it was on a one of the agile or extreme
programming mailing lists to remember Grady Booch talking about what's kind of
what's the traditional definition of a time it was a high level view of the
components and interconnections and there was then push back without saying
well is that really a good definition and this led to amongst other things too
hard to change notion which i think is a very good one but i think it was taking
a step further very nicely by Ralph Johnson when he said well actually
architecture is the important stuff whatever that happens to be and it
happens to be that important issues the things that you have to care about a lot
do often correlates with stuff that's hard to change and this was reinforced
really nicely by a fairly obscure talk that we saw in a again in an early XP
conference by an italian economist and he was primarily talking about lean
manufacturing and simply effects of that but one of the things that he brought
out was a one of the call
drivers of complexity is irreversibility if you can't change your mind and go
back that then constrains you very much
makes dealing with situations very complex and he his comment was in fact
that the traditional manufacturing processes that the traditional
production line
it tried to deal with complexity by reducing him
he states the system could get into well lean manufacturing focused on trying to
remove that problem of irreversibility by making steps reversible you could
simplify things and now i think is a very good way of thinking about the
importance of this hard to change notion
so Neil kind of hit on the next . we want to talk about when he said that
architecture is a large large nebulous thing that we could talk about we could
talk about a lot of things and we say architecture and they've only given us
40 minutes to talk so we have a divided architecture into two different areas
application and enterprise and we want to be clear about what one we're going
to talk about
we want to focus on application architecture there are a lot more things
that can go into enterprise level architecture but for this talk we want
to make sure that we're thinking about architecture on application level and of
course when we say that we have to say well what do we mean by an application
it's very fuzzy term I like to think of it is there are three sort of main
things that drive you
boundaries of an application one is a code base and a team of developers
working on some common code base second is a degree cluster of functionality
that's perceived as a single thing by its users have been third and perhaps
the most important a single source of funding applications tend to have a seat
a single group or organization funding channel that drives into it
so with those boundaries
you want to go back to architecture and then think about the agile piece of it
how do you make it a little bit more flexible a little bit easier to change
so to do that we want to think a little bit about what is agile and some of the
principles of the manifesto and then also how is architect architecture been
done traditionally so first one of the principles of agile which is working
software over documentation up
I think
one of the I had a question ask me an interview yesterday which was is their
perception architecture does not apply to agile and I think one of the reasons
for that is some of these principles i think instead of being interpreted as we
want
working software over documentation
they've been interpreted as we want working software and no documentation so
if we think about traditional architecture we might think of it as in
in the waterfall sense
so this is a diagram that kind of shows what waterfall might happen and what
faces there are these discrete phases has have outputs of some sort and
documentation is often one of those outputs for example the design phase
like Martin was calling architecture might have architectural diagrams as an
output so when you think about what agile is saying to say working software
over documentation or what I think a lot of people interpreted as working
software and no documentation kind of raises the question
well are we doing architecture in the sense i want to change this diagram and
show you what it kind of looks like when we think about in an agile fashion so we
bring it all together and not even kind of asked the question a little bit more
well how do you have any time to do architecture how do you have any time to
think about architecture
well instead of discrete phases where we're having some output at each level
we want to be thinking about the value that were creating and each one and
repeating that an agile
and it's important to realize that this is not just true of the activities in
the traditional waterfall sense you
you do all your design you do all your architecture when you finish it and hand
it over to the programmers
it's not just that it's also that the separation in terms of people but the
people who do it a difference in organized differently they don't have
much collaboration
while in an agile team it's a much more close-knit
I mean yes you will have some of these people who are taking an architect's
role but they are much more a team lead within a group of developers and
everybody is to some extent thinking about architecture and thinking about
the day-to-day programming as well certainly the lead the architect has
much more of the architecture responsibilities and is doing much more
than of that kind of thinking but they're also still doing a certain
amount of day-to-day development work at the same time my view has always been if
you're going to be a lead on a development team you have to be doing
some programming
that's why I'm on the stage by the way yeah somebody has to be actually be
doing real work on this stage
so coming back to diagrams and outputs and and what value they had what is what
is it providing i personally don't think that waterfall would have had a design
phase and doing waterfall would have a design phase without having value it's
going to provide some value but what is it lacking
so when you think about having an output and then moving on to the next phase and
separate people
then you have to think about what happens when you learn something else
and something has to change
so another principle and the agile manifesto is responding to change over
having a plan
now I'm going to say this in a different way again
responding to change and having no plan that's one way that people interpret
that statement
and I've heard different synonyms for agile which is the wild wild west or
cowboy coating so we're not going to have any architecture no plan
nothing that's not actually true
what we want to be thinking about instead of having a plan that's set in
stone
we want something that can change and we can iterate on
we want the value of a diagram something that can communicate things but
something that can also change and that change ability
I think raises some questions about the whole metaphor of architecture itself
I mean the reason we use the word architecture is we're drawing a metaphor
from the construction industry and as in typical with software development we got
the metaphor completely wrong because in the software if in the building industry
what an architect it does is concentrates on the experience of people
working and using in the building
I mean if we were to say what is the equivalent of an architect in software
development
it would be user experience designers it wouldn't be anybody concerned with the
internal design of a system
I'm very conscious of this because I'm married to someone who does do that work
she's a structural engineer and that's completely different markets and i can
tell you traditionally structural engineers and architects get on just
about as well as the average programmer and enterprise architectures in our
industry so i think a better metaphor for what architecture should be like in
the software world
let's put it is a nice little talk you can hunt around on youtube I never
colleague of ours eric dunn and berg and the talk is called architecture without
architects and what his point of view is that if you want to look for a metaphor
a better metaphor for architecture city planning
because with City Planning you've got something that's much more dynamic and
vibrant
things are changing all the time in city planning and you have to react about
while at the same time trying to keep some kind of coherence across the whole
picture
it's important to have some kind of planning activity because otherwise you
end up with an overly chaotic City
but at the same time you can't come up with a design and let it set because
things are always going to change around you and I think that's a much better
metaphor for what software architecture is about
so Martin introduce the talk as two parts
we talked a little bit about what is architecture but so if if if
architecture is happening when you're working in an agile fashion
how do you make sure that how do you know
and this is important because it yes it is very possible for agile projects in
any kind of project to suffer because of the fact that architecture isn't
occurring and one of the examples I always . back to is a four works project
from about 10 or so years ago where we were building this big sister Matt had
multiple teams in in multiple countries and somebody looked in and did a review
of it and found we had three completely different incompatible object-relational
mapping frameworks being using different parts of the code now
that didn't stop the project from being successful it was a very successful
project but it certainly calls more trouble money than we might otherwise
would have had we got a bit more discipline across the whole picture and
that is where a lack of architecture can get you into a dude into difficulties
and so it is important to pay that continuous attention like the city
planner to make sure that things are coherent
so we don't want to take you through a few activities that we think are
examples of where architecture is happening in agile software development
that you might not have thought of
or you might not be getting the value or the architectural thinking out of them
that you could still get because it's not all about diagrams and and it's
definitely agile is all about collaboration and communication over
over over things like documentation so so one of the first things that we want
to talk about is something we thought works call an inception
but you could also call it something like a kickoff or or just the start of a
project or product
so what happens in an inception is typically all the key stakeholders get
together and sometimes we've seen that architects are left out of that out of
that meetings
so why why is that well people aren't valuing architecture the way we should
we are not thinking about it as something that is important or things
that are hard to change we want to be talking about that early on and the
phase of the project and this is I think one of the interesting shifts that you
need to make when you think of things in terms of an agile context as opposed to
the traditional one in a traditional context what do we do at the beginning
of a project we want to make all the important decisions because we have to
get all those decisions right
early on but in an agile context you want to kind of do almost the reverse of
its you want to say to yourself how can we stop things becoming architectural
how can we make things by but and what I mean by about how do we make what seems
like an irreversible thing
reversible so what a great example of this was again in an early for works
project we were one of the early projects that was trying to use an agile
approach with traditional relational databases and a lot of database people
said to as well where you can do all this agile stuff on the code but the
database schema has to be defined at the beginning because it's a right pain in
the neck to change
so you've got to fix that you've got to go through and get a full data model
I remember being in sections where people were trying to come up with these
data models but fortunately we will save by one of our colleagues promotes
analogy who questioned that whole approach and said well you say the
database is hard to change
let's let's work on that let's try and figure out how to make it easy to change
and he came up with a scheme that said well the database gets modified by a
series of migration for each migration is very small it might be splitting the
single field in into two separate columns or something of that kind of
moving a column from over here to over there and every change not just changes
this
I'm about migrates any data within that so you have the necessary
ddl to migrate the data and then that way you can change the schema steadily
all the time and you might have a database out there in production but you
can still change it to a new version by running the migrations and if you've
used tools like liquid base or rails migrations or things like that you're
familiar with have this scheme works now
and in fact this is how we work on pretty much all our projects right yeah
he's still doing it good
but that was because of the fact that promote said well let's take something
that seemed to be irreversible and make it reversible and what does that do it
means that this database schema is no longer an architectural concern and it's
no longer hard to change
you wretched the best architects remove architecture by figuring out how to make
things shiftable are throwing another example as well that of contracts
between sister serve different services
I remember talking to somebody those are big noise in the SOA world and you say
well service contracts that have to be fixed up front because they're so hard
to change and immediately all my spidey bells went off and said hang on hang on
that must be a problem
and again as whatever I keep mentioning our colleagues but we do seem to be a
bit single-minded on this and in Robinson came up with a technique of
consumer driven contracts which allows you to change service contracts more
easily
so again the hard part we need to get up front is to say where are the things
that are hard to change and how can we make them easy to change do that and you
have you get a very big win and I'll just add in one more thing is during an
inception
I like to make sure that everyone in the room kind of agrees on what is hard to
change there are they vary based on a lot of factors organization sighs
capability think things like time lines that are if timelines are really short
or really pressing on the project
there are some other things that might be hard to change so defining that up
front and having an agreement not only from checking all stakeholders but from
business stakeholders
and their understanding what about what is hard to change and then also what
might be easier to change and what might change in the future if we learn more
having that up front having an idea is always a good thing to start with the
next thing we wanted to talk about was stories in agile teams especially are as
we write stories to break down function to break down functionality and deliver
it incrementally the stories are usually based on user value
well architecture has user value as well and changed the architecture should also
be tracked and and shown what value were adding or what we're changing as we go
along
a lot of the times after inceptions we will have written down or documented
some some spikes that we want to do the definition of spikes are exploring these
things that are hard to change and figuring out
maybe the path that we want to take if we have some ideas
what what we want to explore so we know is it hard to change or is it something
that we can decide later at the more responsible moment and in doing this
what you're doing is allowing the team to talk about what are the key things in
the software system and this reinforces what i think is very key . the
architecture of the system doesn't lie in any architecture diagrams or any
other static artifacts that are produced that doesn't mean they're not useful as
will mention later on but they are just one representation of the architecture
not the most important one the most important one is the shared
understanding between the people in the team as to what power system fits
together
Kent Beck had a very nice little test you would do as a consultant
he would wander around teams and so describe to me how the system works
using only four objects and he said that he did that if he did that and most
people pick the same or roughly the same for objects
he knew they had a good coherence across the whole team
but if everybody was telling him a completely for diff completely different
set of four objects
then he knew there was no coherence so the architecture might be written down
or a diagram or something but it wasn't in people's heads and that's where the
most important place for architecture is it's in the heads of everybody who's
taking part in it
the other thing around stories or incrementally changing functionality
that deals with architecture is is acknowledging the fact that as you're
working on a story
you may be dealing dealing with architectural changes not only one it's
an architecture related story but something that's the new feature and are
creating and changing architecture is happening while you're paired
programming or or just developing a new feature on your own even so thinking
about architecture in in someone's head it's it's happening as you're going
along and changing the software one of the best architectural elements is
something that people often forget which is example areas of the code and
particularly when you start beginning in iteration and you're thinking about the
stories are going to be played in that iteration
it's often useful to say our you're going to be accessing the flippin um
system go to this module over here
that's it a an excellent example of how that should be done well people when
they a writing code they often look two examples that currently in the code base
and use them as inspiration how to do things and so what you want to do is to
try and make sure that the really good bits are the ones that are highlighted
and people copy those they don't copy the half our half attempt that you made
when you didn't really understand what was going on but you haven't got around
to fixing up yet
it's really important to highlight the exemplar parts of the code and I
actually then become very important drivers in architecture of that if
that's done
on our teams we usually have these things that we call showcases where we
are showing off new functionality how things have changed for product included
in this is sometimes technical aspects of the technology aspects of how how the
code is working
this is an example of a tool called gattling which is a performance metrics
a tool and we showed this in some of our showcases because we want to also
highlight different pieces of how the code is working how the architecture is
working and in this particular project
it was a very important that we have a performance that met we were replacing a
system so we wanted to meet or exceed the performance of the current system so
being able to show aspects of architecture and technical thinking and
showcases makes it also makes us also remember that the architecture and
thinking about how how we're implementing the code to solve these
business problems is is important and it's important to help us not step on
our own toes
later on by thinking about what kinds of things we are designing for another
aspect of that Martin kind of hinted on that I want to talk about is is what
some people might call meetings but i'm going to call collaboration
so on a lot of projects i've been on we do something called a mob code review or
or mob code refactoring I think there are a lot of different ways that within
a team you can collaborate on changing the code and if if you want changing
your architectural path
and we focused a lot on collaboration and talking a lot of it we've given
face-to-face examples such as inception meetings or iteration planning meetings
or just the fact that team is physically together which allows a lot of
face-to-face communication and we have to accent this a lot because our
observation and follows without from original a journalist who feel that you
get the highest bandwidth and fidelity of communication when you're actually
physically I'm co-located the most people not necessary for all people but
I think for the majority of people but collaboration isn't just a face to face
stuff a little early example of collaboration and a role for an
architect that i came across someone about projects was the did detect lead
the architect he went sure that he booked a certain amount of time every
week for reviewing code commits to three times a week all I can remember we good
roughly about kind of frequency he would sit down spend a couple of hours and go
through all the commits that had gone into the code base he wouldn't
necessarily look at in detail it everyone he would triage them a bet you
would so are we have I know this developer he's really good at whatever I
know it's a bit about there are a bit more book more if they
well this developer here she's normally really sharp but she doesn't know bc
area code base terribly well so maybe I'll take a look here went through in
that kind of way reviewing certain commitments and then if you found
something that he wasn't too happy with it so
ok I need to spend some time pair with that developer and show them how I think
it ought to be done instead
and this isn't a face to face thing so much except of course the pairing later
on but again it's a role in making sure that the architect is able to see what's
actually going on in the project and get a sense of what's going on in there and
and this is I think a really important point because one of the things that
always amused me about the the traditional waterfall style is that when
the architectural designers came up with a grand design
nobody ever asked them to look to see if the code that was written actually
matched it when it had been written and when I mentioned that to him to my wife
the structural engineer her reaction was you really surprised that well in in
tradition in proper construction engineering
you always go and look to see if the construction people have done what had
been said because half the time they won't
you have to be keeping an eye on that sometimes and just standing around
sometimes for other reasons but you part of your responsibility to look to see
what happened
partly because of course you want to ensure that the plans are carried out
but also so that you can learn so that you can see what works and what doesn't
work you can't disengage yourself from the process by reviewing commits the
architect that gives you one mechanism to see what's actually happening on the
ground and to see if some of the decisions being made early on are
actually planning out in practice that technique was suggested to me as
something that I could do when i first started as a tech lead at one of my
projects and that was also done on a distributed team but between cincinnati
ohio and Bangalore India so it also is very good to me to look at commits and
that will help spark different conversations between distributed teams
or between two areas are several different areas i think it's it's a
really good technique to use to say oh i just saw this and then that starts a
conversation about about how the code is changing and and the kind of plan that
you're you're working towards
so come back to this diagram again
and the last thing that we want to talk about is his plans are ok and oftentimes
in our offices we have big white boards that are all all of the walls and
there's always there's always a drawing on them whether or not it is a business
process or something technical
we're always drawing and revising so one of the important things is being able to
change that as we go and that white boards make it really easy but we also
have to do that to communicate to external stakeholders people that depend
on us maybe an Operations team maybe maybe someone that is a client of our
API
we need to be able to communicate what is happening
internally we can't just keep everything in our heads it's important to
understand the direction but to be able to have tools now it doesn't necessarily
need to be a diagram you have to find that right balance of writing something
down at showing code showing something in person
there's a balance there and there's also can be done in a very collaborative
style again a another story from one of our projects was a situation where
they've been building this large code base and people are getting a bit
uncomfortable about the the way in which V dependencies between the various
modules was taking shape
so what they decided to do was to first figure out what the problem what problem
actually looked like and use a code analysis tool to plot out the
dependencies between the major modules and they saw that and I said that this
is a lot more messy than it should be
so then got together in a meeting put the diagram up on the wall and said okay
what do we want this to look like and this was the architect on the team plus
the senior developers I don't think it was the entire team because that
wouldn't fit it fit into any any of a meeting rooms but it was me or Harvard
dozen a dozen or so of the senior developers and they ended up drawing a
picture of what they felt that I ought to look like and then they wear out to
the whole development team and said we want to move in this direction so
wherever whenever you're in the code base and you see one of the dependencies
that currently present but isn't in the direction but we want to head to we want
you to fix it and do little fixes over the course of the next couple of months
and then over the course of two months that code gradually got into the shape
that they wanted it to have him and then in the last week or whatever
a couple of the senior developers just went in and finish the job off and
crossed off all that the finalized and tease and that kind of measure can be
very useful not just because it's a collaborative effort at coming up with
what do we want that future to look with but the fact that everybody's engage in
moving towards that kind of position and then to make sure that things were cult
at the end they actually put into the build process that the ability to fail
to build should an illegal dependency crop up
not necessarily be
was the one any cases but because they wanted to know if anybody is inserted in
dependency that was against what was there in the diagram so that is the role
the tools comply as a verification mechanism to help do that
so we've been talking a lot about certain aspects of special teams of
practices that a lot of agile teams have and and I want to kind of step back and
say what do we see often and what are some things to kind of end on what are
some suggestions
so one of the things we see a lot is we have these agile software development
projects that are working in an agile fashion and then we have maybe this
separate group that is doing some sort of architecture well we've suggested
some ways that architecture is happening in agile teams and I want to call out
that it's a two-way street to make sure that architecture is happening
so the developers on the team
in fact DA's and Q a's project managers we should all be talking about what what
we're changing in the project how it's changing what's that impacting we should
be talking to people that are external to the project people that are trying to
make sure that architecture is happening so they know that thinking is happening
but often times we see that those responsibilities on the team to be
reporting or or asking for permission for architecture to change so it is a
two way street for people that have the responsibility to make sure we have a
plan and some organization within our software to also participate on teams
and utilize some of the practices that are happening already
to gain that knowledge and help maybe draw those diagrams or communicate in
the right ways
by utilizing things that are currently happening that show
architectural thinking
we kind of been talking about how architecture is the important things the
things that are hard to change the good news is there's less that is hard to
change and it's getting a lot more and more easy to change a lot of different
things
Martin gave the example of promote and a lot of questioning whether databases are
hard to change and now we have the question of whether or not relational is
the right thing and and a lot of no sequel databases are even built in with
with migration systems rails is built in with migrations
so people are a . adopting change and making that the norm things like
environment provisioning or infrastructure as a service that these
are all things that 10 years ago 15 years ago would have been in our list in
an inception is hard to change
so that's good that we have less of those but we still want to be important
we still want to be thinking about what is important in our system we still want
to make sure
architectural thinking is happening and it's not thinking that is really the
most important thing I mean we'll talk a lot about different techniques
appropriate architectures and things like that and we've done a bit of this
in this talk will be a lot more of it out there in the conference but in the
end the most important thing is the will to do this that you have people who care
and that you give them the space and the time to do that a lot of times
particularly more senior developers get really over stress and have to do a
gazillion things that what you're discovering that glorious life at that
foot works now where there's so many different things you could be doing
you always want your best most experienced people to be doing that that
but it's important from the whole project management side to make sure
that the senior people have time to think about architecture
there are various ways that you can do that and the technique of second aside
time for commit reviews that are mentioned earlier on is one case of that
another example that I've seen on a few projects
is saying that although that the developer the the architectures pairing
on stories and taking part in the programming
we're never allowed to sign up and be responsible for a story's delivery
they always have to help other people as opposed to actually taking that role
themselves and that also is a technique that allows for a certain amount of
provision of time because in the end it's that time and and will that is
crucial because like so many other things in software and getting a good
architecture for your system is primarily a people problem
so with that we want to say thank you for listening to us we hope that you
have a little bit more context around different things that are probably
happening on agile teams and how they can be utilized to show and demonstrate
architectural thinking and to enable those that are already thinking about
architecture on your teams and if you want to talk to more about to us at all
about what we've been talking about
we'll be hanging around the foot works both most of the time I do have a book
signing at the person before the round 1030 today but other than that i'm going
to be around the footwork spoof during the breaks and of course attending
doctor talks and molly is going to be doing at an interesting contest between
microservices and monoliths are not quite sure which side you're on
so that grand debate is going to be later on today and I think you should
find that quite interesting if you find that discussion and absorbing and thank
you very much
