Scaling software delivery teams – up & out

In the world of technology infrastructure (servers, networks, storage etc.), it’s normal to talk about both scaling-up and scaling-out your platform. In this context, scaling-up means making the things you already have work faster by adding in better CPU’s, more memory and speedier networks while scaling out involves creating additional parallel execution paths to get more work done in a distributed fashion.

Both approaches are sensible strategies when done at the right time as they both will increase the overall transactional throughput for your platform. The approaches are complementary and often should be used together.

A parallel from scaling hardware to scaling teams.

I now want to carefully draw a parallel between the world of scaling hardware and that of scaling modern software delivery teams. I think it fits pretty well, to a certain point, but you can be the judge.

Scaling software delivery teams

The goal of scaling your team is to increase overall feature delivery throughput, i.e. the volume of quality product you can produce as a company. Let’s talk about what scaling-up and scaling-out might look like in this context.

Effective scale-up

The goal of scaling-up a team is to deliver higher quality and/or faster software. When attempting the scale-up, the activities are generally team practice focussed including things like:

  • Adopting an agile framework such as Scrum.
  • Coaching peer solution design and review.
  • Regular improvement focussed retrospectives.
  • Hiring members with more development experience.
  • Consciously training and up-skilling team members.
  • Continuous delivery best practices like regular small scope automated releases.
  • Getting platform operational telemetry (errors, performance) feeding back into the team that build the software.
  • Better tooling and team working conditions – great hardware, the right tools.
  • Providing the headspace to focus for a reasonable period of time.

The best companies are continually scaling-up their teams, it’s not a phase, it’s a continuous improvement cycle that keeps on going. The best candidates expect this to be part of your culture.

When to scale-up

You should consider incrementally scaling-up all the time, layering on improved practices within regular continuous improvement cycles. It is much easier to do this when you are a small team as the practices will more easily propagate out as the group grows.

So there’s no bad time to scale-up. In the start-up world, a good time to consider a scale-up focus could be as soon as you validated your product in the market, where you create a little bit of headspace to avoid piling on technical debt. You will need a solid senior resource base to achieve this, some people who know what good practices look like, and more importantly understand the right time to add just-enough of them in to mix.

Effective scale-out

As a quick reminder of what we mean by scaling-out our team, we are hoping to get more parallel and independently productive work-streams, i.e. more people creating a higher feature velocity. To achieve effective scale-out of software a delivery capability requires more than just bringing on teams through hiring or outsourcing, you must support this headcount growth very deliberately to recognise the return-on-investment.

In a software product (SaaS) environment, the activities to enable the scale-out of teams have become much better understood in recent years. It still feels a little bit more like art than science as the best-practice patterns are very debatable, evolving, contextual and involve human’s changing their behaviours – hence scale-out is a regular source of organisational missteps.

Common scale-out patterns

  • Applying architectural platform patterns to remove dependencies between teams.
    • Layered architectures – for example, a split between an API team, who works with a high degree of isolation from front-end mobile and web teams.
    • Microservices architecture where aspects of the product offering are implemented as a collection of loosely coupled services that can be more readily understood and extended by the teams that manage them.
  • Creating shared-service teams that support and scale-up (up-skill) the more focussed feature (product area) teams.
  • Flattening your org structure while focussing on sharpening your mission and encouraging effective inter-team communications, allowing higher degrees of team accountability and self-organisation ¹

When to scale-out

Scaling-up teams is not an easy exercise, the additional synchronisation and communication overhead to support decentralisation of your product stream causes you pain and costs you money. It also requires clever change management to bring everyone along for this journey.

With that said, when it’s obvious you need greater feature throughput, and you have been in this position for many months with no end in site, scaling-out the team is probably the right thing to do.

Do I really need to scale-out?

When you have a small delivery team, maybe less than 20 people, with no concrete plans to grow, you may never need to action scale-out. A straightforward monolithic architecture will give you the best bang for your buck, the team’s natural size should make transparent communication and alignment easy enough.

Even if you have plans for rapid headcount growth, most platforms should start this way during the validation period – keeping everything really simple and lean. Premature segregation of product streams or architectural loose coupling can make the application harder to refactor quickly when the early learnings start coming in during your product validation.

An excellent technical lead can lay some of the foundations, i.e. the architectural seams to allow for future scale-out strategies when the team and the platform need to grow, which is often be years after the validation phase.

Common anti-patterns when scaling teams

  • Scaling-out poor practices: Businesses that scale-out without ever having a focus on scaling-up, effectively amplifying the inefficiencies that already exist.
  • Scaling-out with no supporting platform architecture: Companies not understanding that to onboard more teams, the platform and delivery support mechanisms must also morph to meet this demand. They end-up growing with no deliberate plan other than to increase headcount. At best the company will miss out on a significant competitive delivery advantage, at worst they may end up clogging the arteries of product delivery – the lifeblood of any SaaS business.
  • Over decentralisation: Tech leaders who are drunk on the microservices kool-aid with not enough platform guidance and oversight, and now own a hard-to-manage diverse application portfolio.
  • The interchangeable teams’ myth: that is scaling-out generic skilled (non-specialist) teams, expecting them all to be equally effective experts across all of the software system.

Ironically, high-demand products with paying users allow a business to gloss over fundamental team scale-out issues for many years. It is also these products that can afford the version 2.0 rewrite build, often in parallel while sunsetting version 1.0. Most SaaS companies will never get this luxury so will need to adopt more incremental strangler strategies to remove their scar tissue, or die while trying.

A note on additional headcount efficiency

As you grow your team headcount, the impact on the overall velocity of each additional developer will be significantly less than linear. That is to say, one-hundred developers are not 5x faster than twenty developers – not even close! The effectiveness of your scale-up practices will be the crucial factor in determining the incremental output of new hires.

Final thoughts

A note on incrementalism

Change is hard, it’s best undertaken with an incremental approach. Too much change too quickly will create havoc with staff morale, retention and product stability. Too little change breeds apathy, a lack of competitiveness and will also impact retention of your most talented team members. So find your Goldilocks zone of change.

Summary

To summarise, scale-up (up-skill) your teams continuously. Apply scale-out (decentralised) patterns at the right time if-and-as required, and do it very deliberately during early stages of headcount growth. While effective scale-out practices can be applied later-on, they will cost you a lot more.


Footnotes

¹ Warning – that’s highfalutin language.

Leave a Reply

Your email address will not be published. Required fields are marked *