Agile Testing UK meetup

While in England for DDD Exchange, I caught the opportunity to participate in a meetup of the UK Agile Testing community. Gojko Adzic led a very interesting workshop about how to write acceptance tests.

Neil McLaughlin, who happened to be in my team in the hands-on simulation wrote a nice report on the evening, and then Gojko himself posted about what he showed us. I'll try add something, from the participant perspective.

The developer mindset

Given we had quite a few people in the room, around 60, it was really interesting to note all different behaviors, but there were also some striking similarities. Most of the discussion we had were focused on

• Formal correctness of the test
• Splitting tests to be testing one concern at that time
• avoiding duplication in test fixture setup

When discussing the outcome, Gojko pointed out that the main concern for acceptance tests was readability, instead. Readability is a subjective evaluation criteria, and might lead to a not-so-precise definition about how to make it right. But that's the only way to write a useful acceptance test. But the developer's mindset really had a strong influence on us, it was really hard to give up.

Constrained by the tools

Despite Skills Matter (that was hosting the evening) doing a fantastic job in providing an impressive amount of whiteboards for all the team, some of the discussions were constrained by the amount of space available on the whiteboard, so we ended up pruning some concerns, thinking they were of little value to the discussion. It turned out we were wrong.

But, as I said, we had an amount of writable space that was a lot larger than what I normally found in a typical workplace... and still this was constraining us. Buying one more whiteboard is cheap, making wrong assumptions because of writable space constraints is a lot more costly.

We're problem solvers, after all

During the exercise, Gojko was wandering around from one team to another one, playing the role of the Domain Expert. However, most of the teams seemed more interested in his Teacher role and asked some questions about how to solve the exercise, but barely none about ambiguities in the domain. We preferred to use assumptions instead.

Even if I knew the trick, it worked on me as well. But it's scary to note that even when writing acceptance tests, which are a lot more translation than design, our built-in problem solving brain leads us so easily on the road of guessing instead of asking.

Subcontracting is a recipe for disaster

A friend of mine recently told me about a tricky situation he was trying to solve. A software company was supposed to develop an application, but they subcontracted development to another company. After a while, the project started going out of control: features were late, quality was poor, bugs were not fixed and release date slipped indefinitely. Things were so bad that lawyers started to warm-up.

As many of you, I had a sense of deja-vu, when hearing this story. The thing that struck me was that this type of project is doomed from the start! Why do people repeat the same mistakes over and over and over?

Money for nothing
Let’s dig into the scenario: customer asks Company A to produce some piece of software.
The price can be defined in many ways, but it’s normally related to a rough estimation. Anyway, Company A gets the contract. Even if they are not going to write the code, they want to be paid for the marketing, and everything that comes before the project starts, somebody will be paid to “manage” the project and some money will be allocated to cover project risk.

Budget splitting from Company A’s perpective.

Ideally, such project should provide a good ecosystem for communication between the customer and the developers, allowing developers to explore and to build consistently upon frequent customer feedback.

The supposed development ecosystem: project manager deals with high level issues, while detailed communication happens informally between the team and the customer

The subcontracting scenario
In a common subcontracting scenario, Company A decides to outsource development operations to Company B. However, since the original deal was between Customer and Company A, a new deal is necessary between Company A and Company B. Sometimes this deal is completely visible to the customer, but sometimes Company A is just pretending to develop the application while Company B is developing it behind the scenes.

Anyway, the new deal between Company A and Company B is based on a different project budget, since some costs (marketing, risk and project management) are accounted on Company A.

Now the project may be worth about 1/3 less than its original budget (the exact figure may vary according to greediness and other human factors) but Company B still needs to allocate budget for its own project management and risk coverage: nobody wants to be sued for somebody else’s mistake.

The project budget, now seen from Software Company B’s perspective.

Adding two separate management boxes around the project doesn’t provide any real value in return, the financial effect is that now the project is a low-budget project, so company B will probably start looking for the cheapest software development resources available.

One would expect that the extra money put in Project Management and Risk Coverage would turn into a perfectly managed, risk free project. Well... not exactly so.

By the way, this is often the schema which is applied in offshore software development, where the common belief is that developers are so cheap that you can compensate the extra costs related to offshoring with huge savings in product building. I am not going to dig into offshoring in this post, but it’s definitely not that simple.

Messin’up with things
Unfortunately, this type of two-levels management doesn’t only duplicate costs, it does more: it makes the whole process inefficient, and it’s actually effectively working against project success.

The subcontracting development ecosystem: formally the referring person is still belonging to Company A, but development is performed by Company B. The key communication channel is now obstructed.

• Communication flows from Customer to Company B through Company A. If Company A is the one to be paid, then all of the key discussion must pass through A, transforming the supposed Project Manager into a bottleneck.
• Indirect communication means delay, important information may rot in a mailbox, before being forwarded to the person associated with the solution.
• Indirect communication means also information loss. Some information must be understood, not only listened or read. Sometimes forwarding it’s not enough to ensure some key detail is not missing.
• Since the information is sensitive for two different contracts, all parties become more inclined to write key issues instead of using face-to-face communication. Some issues are then carefully evaluated and involve more negotiation on an official level. The entire process becomes then waterfallish: having three players, with higher risk and no effective direct communication, paper (specifications, change requests, and so on) becomes more important than face-to-face communication eventually slowing down the process.
• The customer role becomes awkward. The customer is the real customer, but for Company B, Company A is ‘the customer’. Unfortunately, the quality of information you can get from a customer who is not the user of the software it’s a lot lower. So the risk of doing the wrong thing it’s extremely higher.
• If Company A is still pretending to be the developing company, direct communication between Company B and the Customer might even be forbidden, making things even more awkward.
  

The overall effect of all these factors combined is often pure havoc. A project with a decent budget becomes staffed with inadequate people, in an ecosystem that obstructs good communication between the customer and the developers, and where management roles are duplicated adding costs and subtracting value.
Is there any hope?
In theory, there is one scenario where it could still work: it’s where Company B is so good at building software that its estimations are significantly smaller than Company A ones. But to put this scenario in the real world one should answer the question: “Why a company that is so good at building software needs marketing from another (and poorer) Software Company?”

As developers, we’re probably associating this situation with the decorator pattern, which is simply adding features (a more effective marketing) to some other (the development) without changing it. Well ...this is simply not the case with subcontracting. The act of subcontracting has a negative impact on development and should be avoided or limited. As a customer, it does make a lot of difference if the software company is actually developing the software or simply playing the role of a broker. Having seen many of these situations going straight to non-recoverable state, I really think that special care about these issues should be taken in the pre-contract negotiation phase to prevent unavoidable surprises.

The alibi of creativity

One of the most intriguing parts of the discussion, that was part of my speech in IAD2008, was a question (interestingly asked by one of the few non-developers) regarding possible negative effects of time-boxing (especially in the form of “pomodoro”) on a creative activity such as developing software.

I quickly dropped my opinion, but I was more interested in making others opinion on the topic emerge. Then I had some tome to think better about it: here are my thoughts.

Coding is not a creative activity
I mean ...there’s creativity involved, but most of the time we are solving problems, in ways which are probably been explored by many others before us. So, basically, coding is a problem solving activity. Creativity slips in when the problem to solve is new, non conventional or pretty hard, or does not strictly involve coding (in this respect, the process might be far more creative than coding). Sometimes we have the luck to deal with applications which are pioneering, but many more times, this is not the case.

This doesn’t mean that we have to be mere follower’s of somebody’s else ideas. But “being creative” or pretending to be, too often has the undesirable side effect of “reinventing the wheel” which is definitely not what we want. Put in another way, “being creative” is often just an excuse for being “too lazy to study”.

Time-boxing does not constrain creativity
Ok, then there are times where the strict time constraints imposed by time-boxed activity, a pairing session, a pomodoro, a day or a sprint, is not enough to allow for the “inspired” solution. As Simone Genini pointed out, “as a manager, you don’t want to wait for developer’s inspiration”. So what you need is a repeatable attitude for this problem solving activity.

Back home I realized that this happens in creative fields too: comics are written on a monthly basis (even if not every author can fit in the schedule), ad campaigns are creative, but organized as time-boxed projects, and so on.

Even in software development, the time-box constraint is less stringent than it might be: if one of your best developers came out with what he consider a sub-optimal solution, something he doesn’t really like. You can be sure that he’ll keep thinking about it, and probably will attack the problem again, once a better solution appeared to him in his dreams or under the shower. Just allowing more coding time to be explicitly allocated to that task won’t probably get the solution any closer.

The scrum mother - re-explained

Honestly, I never thought my last post could have been that controversial. But I had so many diverging feedbacks: some loved the post, some demolished and told me it was terrible, and some others liked but got the meaning the other way round)... Maybe it is necessary to clarify things a little bit.

I am very convinced that authority stands in the middle of being a good Scrum Master. SM is not a leading role in Scrum, he/she just preserves the integrity of the process, without effectively taking part in it. SM could participate in a discussion, but must not take any decision. SM must ensure that the discussion terminates with a shared decision.

That’s why I came up with the analogy of the Italian mamma and the old-style Italian family. The father has the authority, and brings the money home (hopefully), the mother manages to keep everything running. Preparing the food does not deliver any value to the outside (unless you own a restaurant) but allows family member to be healthy and do their work.

But I guess the metaphor could have turned weak, because I was referring to a very specific type of family, and every single reader had his own idea of mother, and the concepts overlapped diverging a lot from my intentions.

How to raise kids
Probably, the flaw of the mother example is related to the different perception of the mother role within a family. I’ll explain what I meant, going straight to the SM role, to be as clear as possible. SM are not supposed to prevent developers from hurting themselves. SM are supposed to allow the team to grow, by letting them do and recognize mistakes, by allowing them to develop self-confidence and do something on their own. You must be around when your babies are learning to go on a bicycle, but if you keep holding them, or force them to use those small extra wheels ...they’ll learn later. The same applies to swimming, where confidence is just about everything. If you keep being always around, they’ll start crying the first time they’re alone. Delayed growth, lack of confidence and a lot of time spent just “being around” by the senior management.

Sustainable Software Architecture

A software development project is a bounded activity. One of the key goals of Software Architecture is to find the best trade-off (or the sweet spot) for a given project ecosystem.
Such ecosystem is the result of many combined factors such as:

• project size
• team size
• team location (es. co-located vs distributed or offshore team)
• team skills (experienced developers, contractors, folks just hired, and so on)
• team members availability (not all teams are well-formed at the time the project starts)
• architect avalability
• turnover
• logistics
• marketing constraints
• deadline constraints
• etc. etc.

I learned to know that keeping software architecture clean and preserved from all this is a dead end road. Those ecosystem constraints affect the way a project is carried on, and will also affect the optimal architecture for a given project. Put in another way, there's no optimal architecture for a given problem, unless you include all the variables in.

As Kevin Seal pointed out at the last week SM event: "Architecture is about things that are expensive to change", and in the open source era, the most expensive things to change are time (which can't be reverted) and people, which have normally a long and inefficient learning cycle.

This makes choices like "choosing the implementation language" an expensive one, because despite the free availability of development environments for different platforms, team skills might have to be formed, and properly skilled.

Make the architecture fit the ecosystem

Answering to questions like "What's the right tool for a given load requirement?" is a non trivial job, but it's the one we're (hopefully) prepared to do. It's right in the software architect mindset.

What I've found more tricky is the need to define just the minimum affordable level of architecture, in a given application. In an ideal world we would like to have the most reliable architecture, making coding easy and meeting all of the nonfunctional requirements. We would also like to have all the team members getting familiar with that, and understanding roles of every architectural component and the reason they've been implemented the way they are.

Too much of a dream. Architecture definition is often a time bounded activity also. For a consultant,  the Architecture Specification Document might be a deliverable mentioned in the contract, so even if there are more efficient ways to deliver architecture information (podcasts, meetings, comedies, tattoos, ...just to name a few) a document must be prepared. But the contained information must be delivered some other way...

Pairing with programmers, or simply coding are great way to get a grasp of coding reality (meaning that architects might learn something really useful from the way architecture and other coding tools are used), but also a bit naive when the matter is "How to deliver architecture information". Architecture have (I mean they must have) a broader scope than developers, taking into account long term factors, while a developer is generally focusing on a problem that has to be solved now. And developers are not all the same. Seasoned veterans and college guys have peculiar skills ad interests, and a completely different background.

Moreover, properly training the team might be a goal for a company investing it its own development team while it might be not a viable option for an external or a contractors-based team. This might sound a bit cynical (it is), but even if I prefer teaching (after all, it's part of my job) as much as I can about the architecture in place, I have to face the fact that information will flow on a limited bandwidth channel: there will not be as many chances as I would like to discuss the architecture, there would not be as many questions, and developers might not be that curiuos, after all.

Finding the sweet spot

Generally, I think that the ideal sweet spot is doing as much architecture as it can be managed, by the team (buy I usually learn about the point location by surpassing it). This is not a fixed line. A good team will probably ask for a more elegant architecture, al long as the project advances, pushing (or pulling) architecture evolution. In some critical situations, like long projects with a lot of turnover, the architecture should be robust enough to prevent developers to harm themselves, keeping the learning curve as small as possible. Some more work for the architecture team, but usually a bit sadder.

The dark side of user stories

I've always had a twofold feeling towards user stories. Despite being critical about many drawbacks of the Use Case approach (the one I hate most is the time wasted in filling a template with the same useless information repeated over and over), I think that User Stories have some limitations that a team willing to turn agile has to be aware of.

Interlude N°1

Seven years ago, a younger me collecting requirements for a small application that should have handled the operations required to set up a secure banking account. Every person interviewed was telling me a different story in terms of "special cases" (the whole story seemed a collection of if … ), and the resulting workflow seemed messy and impossible to tame. I tried a different approach and started modeling considering states instead of actions. It turned out that the workflow was pretty simple, after realizing that there was no before-after correlation between some actions, and a loose workflow could have mastered the whole stuff with 2 screens, and a bunch of checkboxes.

Interlude N°2

JAOO Conference 2006. Jeff Sutherland talking about Scrum to a subjugated audience. Telling something like "If the product backlog is not ready, I give the whole development team a day off.". Silence. People dazed and confused. "There is no chance that any line of code written without knowing its purpose will turn useful. Instead we'll have to pay for writing that line, for testing that line, for refactoring that line and to convince the team that this line shouldn't be used.". Pause. "The other reason is that there is no better way to have the management hurry up and finalize their decisions than having the whole development team having a day off". I liked that.

Interlude N°3

Business application. Very complex domain. Requirements were heavily related to the Italian fiscal system (which means that the application should mimic the laws, but the laws are not intended to be understood, moreover many of them are flawed from a logical approach – but this is a whole different story, that can't fit here…). It took the lead analyst a couple of weeks to be able to nail down a specification for one of the complex use cases, and he was working together with the domain expert every day of the week.

Back to our story

One of the key points of user stories is that they act like placeholders of the real specification. Instead of writing a detailed spec, structured like a Use Case document, some months in advance, a User Story provides the minimum amount of information required to identify the story, estimate it, prioritize and relate it to other stories. Before development of that specific story starts, Story Cards serve basically as a low-fi management tool (estimation and prioritizing), allowing to defer the detailed requirement specification to the very last moment. This way, requirements will be more precise (because they could have changed in the meanwhile and because the whole team is more mature and expert of the problem domain) and would require less documentation, due to the shortest release cycle.

Unfortunately, there are some preconditions that make this magic happen. One is the availability of a domain expert. It's a key factor in many agile processes (not to mention the XP "cohabitation") but really reads like a simple thing, in a book. In reality one might not be that lucky. Domain experts might not be so available (I agree with you that this is a business suicide – don't get me wrong), or not so expert, or maybe simply do not have the right perspective. Well …these are basically the reasons why the analyst role has been invented. Forget what we've seen in the "Rose years" where an analyst was a man who produced useless diagrams. An analyst is a knowledge cruncher (this definition comes from the DDD book): somebody that becomes able to master the domain like the domain experts do (or possibly better) and to derive a model from fragments of information.

Many times, User stories are seen as a way to leapfrog the analysis phase and the analyst altogether. A good design will "emerge" as a result of iterations. Well, …the use case of Interlude 3 was one iteration. But if requirements were collected by developers straight from domain experts (as in Interlude 1) the result could just let you completely stuck with crappy code for ages. Many teams do not suffer so much from this limitation. Maybe they have rare beasts like developers with brilliant analysis skills, or the analysts really embraced the agile mindset and perfectly fitted in this role. Sometimes, instead, the analyst is perceived like "the person who writes the specs", and since the specs are not that necessary anymore, so is the analyst.

Hmmm … stormy weather approaching.

Tags: agile, OOA, development process

Doing Agile In Italy: Paperworks

In the last Agile Day in Bologna we had quite a few demonstrations of how to make Lo-Fi tools (such as Post-it, StoryCards, etc.) work, to master an Agile development process. Fascinated by Tim McKinnon's presentation, I then went to a local office supply shop to find some tools to play with. Unfortunately, when I started describing Tim's nylon attachable pocket board, the girl at the shop started looking at me quite …oddly (ok… if I describe it this way, nobody could understand, except the few that attended Tim's speech, but I assure you I did my best. I also looked for pictures of "the thing" in the 'net but I couldn't find one – …maybe because I don't know the name).

Anyway, I went to the shop. And what I asked for seemed odd. There was something similar, like cork boards – in this case you need some pins to attach story cards to the wall – or something else that could contain, buy not show the story cards. I then shifted on Post-It papers. Which are cool as long as you have the right board to stick them on. They also have some drawbacks: since they're sticky, you do not write anything in the B-side (while you do that in story cards or CRC cards). I also bought some Super-Sticky post it. They stick everywhere. But there's a smaller choice of colors so it feels like you're in Coldplay's "Yellow". I felt a bit frustrated but also eager to try to squeeze the best from what I had.

In a couple of days I had a chance to play with my Post-Its. My task was to define a development process that was suitable for a SOA environment, agile enough to be productive, formal enough to be used in a Banking Institute. We started playing with our Post-It and … worked. We were able to spot contradictions and missing pieces in a very short time. Writing the document will be the long and tedious stuff, but I guess in this case it's unavoidable: agility still has to cope with the suit-and-tie mismatch.

Tags: Agile, SOA, Italy, Development Process

Resources for a Maven controlled release cycle

While setting up a development and release process based on Maven and CVS, we've found a lot of useful documentation provided by the University of Calgary. Unfortunately we've found it after we've done 90% of the job already, but at least it was a confirmation of our choices... by the way, here is the link.

Tags: Software configuration management, Development process

The dark art of cheating software estimates

Most widely used techniques for estimating software projetcs are

• guess a number
• ask a number and multiply it by pi
• pick the release date, count the days backwards and multiply them by the available people
• function point analysis
• use case points analysis
• ...
  

I personally prefer UCP, because it best fits our usual development process scenario, where the use case list will also be the starting point for the real development phase.
In UCP the estimation is the result of two main factors: the application complexity (roughly measured vie the number and complexity of the use cases and actors) and the environment factors which - as everybody knows - heavily influence the outcome of the project.

The second reason why I like UCP methodology is that when I did retrospective analysis on finished projects, the results where pretty precise. Which is obvious, if you think about it, because retrospective analysis is exactly the way estimation methodologies were tuned. There are two important points so far:

1. The UCP methodology is pretty accurate, if you correctly evaluate the starting factors
2. You can still make mistakes, because factors might change (new use cases might be added during development, or environmental factors may change, or reveal themselves wrong)

The outcome of the estimation process is a number, or a range of numbers, which can represent hours, days or months of development effort. It's a measure of human time needed to construct the desired application. Here start the real problems: you pass the number to the management, they multiply it for the salaries and realize that the project will be too expensive. Put simple: you say time and they say money.

Here comes the cheating
The official estimates will be close to reality, but reality is bad news... so what will you do? The problem is that the link between effort and price looks completely blocked before the project starts. During the project, everything might happen: you might be 2 months late, hire new people and train them, and so on. Each time, you add a random factor that invalidates your fixed ratio between worked hours and the overall price. Still what happens 99% of the times, when showing estimates to the management, you'll be asked to reduce them.

Sometimes there are business reasons for it. It's like buying a car, the salesman knows that the final price will be $50.000, but will tell you about $39.900 and the optionals... Some other times it's like a pavlovian reaction: time is a cost and must be compressed. Wasn't it sad enough, I've never heard of a smart suggestion coming out in this phase. Normally you can have one of the three:

a)"Let's skip the analysis phase"
b)"Let's skip the design phase"
c)"Let's skip the test phase"

If you still are optimistic, I have to warn you that the a) and b) almost always imply c). Or, put in another way, c) is always implicit in such a situation.
The most annoying thing is that the same overall result could have been reached talking only about money. Just lowering prices. But everybody assumes that the other one is lying or maybe the way (presumed) cost reduction is achieved makes the difference for somebody.

But let's get to our numbers, as I said before good news are that prediction are rather accurate, still they might fail. One way is having a wrong count of use cases, meaning that more can appear along the way. But a new use case is a new feature (it's the $500 optional on our car) so it's not a problem, as long as the price varies accordingly, and the time also does. Often, what happens in the closed room it's a bargain of money vs time, sort of "I'll pay you this extra money, but you'll have to deliver it all at the original planned date...". hmmm
External factors are more tricky, cause they're harder to evaluate. Sometimes they're just mere assumptions and it takes time to realize if they're right or not. An example of a tricky one is "motivation": you can assume motivation is 3 on a 0 to 5 scale, cause you simply don't know. Then it's hard to find a special moment in the project lifecycle when motivation drops to 2, triggering a recalculation of the estimates. If you have a boss saying "I noted that mood dropped in the development team, can you please update the estimates according to that?".
So your initial assumption are kept "locked" shielding the official numbers from the force of reality. But every time the estimates are kept safe, old, and untouched, you can assume that they're just a lie, and distance from the truth will have to be filled somehow. The difference is that if the truth is exposed, people tend to behave accordingly; if the truth is under the carpet everybody feel free to cheat a little bit more.

Tags: project management, development process, software estimation

MDA Skeptic - part 2: Impact on roles

The first thought that stroke me while getting deep on the subject, is the fact that moving the focus from the code to the model means also that the development team must be reshaped accordingly. Roughly said: you need more modelers and less developers. Traditional development phases get affected as well: modeling actually becomes coding, so the borderline between development and implementation becomes blurred.

Modelers
To cope with MDA, modelers have to be pretty skilled in UML. One might argue that they must have been anyway, but this is not always the case. Normally, you define a reasonable subset of the features you might want to model at a certain level of the development process, according to the available skills of the team, and you postpone or ignore the others. For example, a good UML modeler knows that there is a tight semantic relation between choosing an association or an aggregation and the resulting code for constructors, but in some contexts this can be an unnecessary complication for the expected result of the analysis phase. With a MDA approach some of these sometimes “unnecessary details” become key modeling abstractions, so modelers need to manage them as well: their UML knowledge must be better than what they’re used to and since their model is running, they need to be a bit more in the developer perspective.

There’s a sort of Darwinian selection here: some folks normally can keep on modeling as long as somebody else is doing the dirty job of making things work (reviewing or sometimes also ignoring the given model), they probably are not going to make it with MDA – “Am I supposed to run my diagrams?” – while some others (probably with developers skills) could face the challenge. Anyway, since you need to spend more time modeling than in a normal process, the more reasonable choice is to have some of your developers be trained in high level UML to join your analysts in refining the model.

Sounds like I am getting too far on the skeptic’s side, but that’s not exactly the case. If you can have good modelers in your team, forcing a strict link between the model (and have it look like a real good OO one) and the actual code that’s the best option. My favorite approach in this case is pretty close to Evans’ Domain Driven Design. As a consultant, I have to say that this is not always the case: sometimes you have to work with pure modelers and have the team dynamics get the software done somehow.

Developers
We are asking a great sacrifice to the development team, we are asking them to abandon what they have more sacred: their fully featured IDE. Moving the focus in the modeling area can result in a loss of productivity in the small scale (on the large scale the MDA promise is to code less so it might still be convenient after all) due to a lower confidence with the development tools. After all Eclipse, X-Doclet, Together and so on already do generate some code, and good developers are really fast in that.

To be honest, development IDEs will still have to be used, for custom implementation to be mapped or mapped, but this appears more like an exception to the development process than the rule.

Probably the main point here is that the developer is now dropped in a more complex context than he is used to, having to leave a mono-dimensional development environment (the code) to a multi-dimensional one where the code is just the projection of different models, mapping, links, markers and so on. In the small scale the situation is just getting more complex for the developer, which resembles the situation happened during transition from procedural to OOP code (I still remember the panic of COBOL developers trying to follow execution flow across class boundaries when debugging their first OO application).

Resistance is a common phenomenon before a revolution, and MDA aims to be a revolution, so it just makes sense. The question is when will it get the required critical mass?


Tags: MDA, Model Driven Architecture, Code Generation, Software Development Process