Menu

Subscribe

An Elegant Puzzle Book Review: an Overdue Read for Software Engineering Leads

I've recently read An Elegant Puzzle: Systems of Engineering Management, written by Will Larson. After working at Yahoo and Digg, Will was an engineering manager for two years at Uber, leaving for Stripe right around when I started and transitioned to engineering management. Though I don't know Will personally, the fact that some part of his experience comes from Uber made me very curious to dive into the book. And what a delight it's been reading it, taking book notes as I went along.

Note that none of the below links are affiliate links or sponsored ones. See my ethics statement on the lack of such links.

An Elegant Puzzle is to date the most hands-on perspective on engineering management within a high-growth, tech-first organization, that I have read. It's a long overdue book for engineering managers and leads. Having read many management and engineering management books, what set Will's book apart is it starts right where the others end. An Elegant Puzzle wastes no time - especially not in the beginning - on covering the generic manager's toolkit, such as 1:1s, giving feedback, team building, which many other books devote a good chunk of their content. Instead, it talks about the engineering pain points that come with a high-growth organization and team, once management fundamentals are in place. What to do when our systems are slowing us down, but we have too many migrations? What's a good way to pay down tech debt? How do we say no, when there is so much work, but not enough people? How do we grow seniority evenly across the team?

For me, reading the book was full of "aha!" moments. When I joined Uber, the Amsterdam office was 25 engineers. I started to manage 5 people when moving into engineering management. Three years later, the Amsterdam office is 150 people - doubling every year - and my team tripled. Many of the problems I faced are challenges the book discusses in detail. Challenges like setting a vision/strategy for my team, building a robust hiring and onboarding pipeline, or dealing with a never-ending stream of migrations.

The tone of the book is casual: it feels like we're sitting with Will, having coffee, while he talks about problems he's faced at different companies, systemic approaches he's seen work best, then giving examples of things that worked for him, in the past. I like how the book rarely presents "best" approaches, instead, Will shares what worked for him - with a healthy dose of systems thinking - and approaches he recommends to his peers and managers on his team.

The book reads well as a reference - I personally prefer the physical book for this one.

This book as the best resource I've come across for engineering leaders with some experience under their belt. For people who work in a similar environment like Will does, the advice is very relevant. These are places that are considered "top tier" companies, meaning the engineering environment is strong, leaders are technical, compensation is close to top-of-market, and engineering is a value-add, not just a cost center.

Working in Amsterdam, places with such "Silicon Valley tech culture" like Uber seem more of the exception than the norm, as many engineering leads in Europe work in a different environment and different constraints. Managers are often less technical, some places still only starting their DevOps transformation. In places like these, tech debt is something the business often does not want to hear about, and tech can still be seen as a cost center over a revenue generator. For managers working at these companies, the ideas in this book are still valuable to go through: companies that operate along with the principles outlined in the book hire and retain easier. They also move faster than their competition. Implementing the ideas might involve challenging the org's status quo and transforming the culture - for the better.

An Elegant Puzzle is not just for engineering managers: product managers and engineers working at high-growth companies will find it a good read. Other disciplines working with engineering - such as recruiters or operations - will find it an empathetic read. The head of product at a large startup recently told me how she was devouring over the book: "I found the model in Will's book a simple model for building high-performing teams really useful (Are you falling behind? Add more people! Are you treading water? Pay down debt!)". A recruiting manager, who read the book, said "I would highly recommend this book to any of my colleagues in the recruitment world who are or aspire to be people managers. While the book was written for leaders of engineering teams, the concepts translate to any situation where you are managing people."

You want this book on your bookshelf

If you are an engineering lead in a high-growth organization, I recommend having the (physical) book within arms reach, together with The Manager's Path. And if you're working in a high-growth tech company, reading it will build a lot of empathy with the type of problems engineering managers and engineers deal with day to day in those environments. I am hoping that this book is the first of many software engineering-heavy management books: it is definitely the first of its kind that I've read. You can buy the book here.

Oh, and I've also decided to write a book about pragmatic software engineering practices within high-growth startups and tech companies. Subscribe to email updates at the bottom of the post, if you'd like to be kept in the loop on developments on the book.

Book Notes

My recommendation on reading the book is to dive into relevant sections, starting at the table of contents. All chapters, as well as subsections within the chapters, can be read independently. I'm summarising the most relevant parts of the book and quotes that stuck with me

The interesting thing about the book is Will published the drafts of the chapters on his blog, Irrational Exuberance, as he wrote them. So all the content can be found in its raw form on this blog. I'm linking these for easy reading.

The book is a series of independent problem areas that engineering managers face, in a high-growth environment. They can be read in any order. The book structures these sections into loosely mapped chapters:

  1. Organizations: topics on organization and team design
  2. Tools: systems thinking and tools to manage change at the org, team and individual level
  3. Approaches: common, but difficult situations engineering managers often find themselves in and dealing with these
  4. Culture: efforts that can shift the culture within an organization
  5. Careers: hiring, evaluating performance, promoting and retaining people
  6. Appendix: tools to operate an organization at the line manager, middle manager, and manager of managers level, as well as the reading list.

#1 Organizations

The book starts with the chapter on organizational design. Managers of managers will find this chapter especially useful, along with people who have - or want to have - insights into reorgs.

Sizing engineering teams - the very first section - was a section I wish Will shared more insights. It's one of the opinionated pieces, and it would have been good to hear what led Will to come to the size of teams he prefers (6 to 8 reports for a manager). Being in a high-growth team myself, we're also targeting around 10 reports per manager, after seeing what has and what has not worked out in people bandwidth. However, this is also dependent on the maturity of the team, seniority of the manager. I've had anywhere between 8 to 15 direct reports the past years, and the lower number is my strong preference as well, though I've yet to put an exact reason as to why.

Productivity in the age of hypergrowth is one of my favorite sections.  It was both a "deja vu" and an "aha!" moment for me. Working at Uber, one of the fastest-growing tech companies every, hypergrowth has been something I have gotten used to, over the years. What was weird to see is how many systems we were decommissioning all the time, how stressful oncall was, and how many production incidents we kept having. I kept asking myself: "Is this normal? are we doing something wrong?" The "aha!" moment came with Will's observation saying that systems survive one magnitude of growth (10x):

If your company is designing systems to last one order of magnitude and doubling every six months, then you'll have to reimplement every system twice every three years. This creates a great deal of risk–almost every platform team is working on a critical scaling project–and can also create a great deal of resource contention to finish these concurrent rewrites.

However, the real productivity killer is not system rewrites, but the migrations which follow those rewrites. Poorly designed migrations expand the consequences of this rewrite loop from the individual teams supporting the systems to the entire surrounding organization.

Other sections in this chapter:

#2 Tools

This chapter is on managing change at different levels. Some parts of it overlap with the previous chapter when it comes to managing organizational change.

Introduction to Systems thinking is where I'd recommend starting the book. I've intuitively been using this toolset, but the book is a great example of successfully applying this tool across the board. Developer velocity is an excellent example, the takeaway being:

Once you start thinking about systems, you'll find it's hard to stop. Pretty much any difficult problem is worth trying to represent as a system, and even without numbers plugged in, I find them powerful thinking aids.

Migrations: the sole scalable fix to tech debt is my other favorite section, together with Productivity in the age of hypergrowth. My past three years at Uber, migrations have always been top of my mind and top of mind of all engineers working around me. It's refreshing to get the take from a different source, and a take that is an informed one. This section is gold, both summarizing why migrations matter ("they are usually the only available avenue to make meaningful progress on technical debt") and tips for running good migrations (de-risk, enable, finish). Given I've done many migrations, this is a charter that felt short for me and missing some other important tips on making migrations successful. Things like purpose-built monitoring, shadowing and reverse shadowing traffic, as well as dealing with the fact that migrations almost always take longer due to the long tail being a lot more complicated. I suspect I'm the minority on this one though - and perhaps this is a queue for me to write an overdue post on this one.

Running an engineering reorg dives into how good reorgs should be run. With a team that grows fast, reorg is just a matter of time, and it's easy to get these wrong. Hidden in this section is a thought that I strongly agree with, based on the well-running organizations I've seen within Uber:

At quickly growing companies, I believe there are two managerial skills that have a disproportionate impact on your organization's success: making technical migrations cheap and running clean reorganizations. Do both well, and you can skip that lovely running to stand still sensation, and invest your attention more fruitfully.

Presenting to senior leadership is some spot-on advice I wish I had gotten earlier. At Uber, I've had multiple initiatives I wanted to get buy-in from the higher-ups. The first few times I tried to convince my management chain why it was a great idea, the response was less than enthusiastic. Over time, I figured out how to go about this better, but Will gives detailed advice on how to prepare better:

There absolutely is not a single right way to present to senior leaders, but hopefully, the template is a useful starting point.

1. Tie topic to business value.
2. Establish historical narrative. One or two sentences to answer the question "Why should anyone care?"
3. Explicit ask. What are you looking from the audience? One or two sentences.
4. Data driven diagnosis. (...)
5. Decision making principles.
6. Whats' next and when it'll be done.
7. Return to explicit ask. (...)

I've had a lot of luck with this format in general, and I think you'll find it pretty useful as a starting point.

Other sections in this chapter:

#3 Approaches

The chapter discusses situations engineering managers will often find themselves in, from having to say no to partnering with your manager or setting org direction.

Saying no is a section that resonates very well with me. In a high-growth organization, there are is always more impactful work that needs to be done, than the team can handle.

(Saying) no is explaining your team's constraints to folks outside the team, and it's one of the most important activities you undertake as an engineering leader. (...)

Articulating your constraints depends on the particulars of the issue at hand, but I find that two topics are frequent venues of disagreement. The first is velocity: why is this taking so long when it should take a couple of hours? The other is prioritization: why can't you work on this other, more important, project?

The Your philosophy of management section exemplifies how, over time, every manager will develop rules of thumb on managing, and Will explains what his management philosophy is. It's a good one to start thinking about defining your own and revising it regularly.

At its core, I believe management is an ethical profession. To see ourselves, we don't look at the mirror, but rather at how we treat a member of the team who is not succeeding. (...)

I believe almost every internal problem can be traced back to a missing or poor relationship, and that with great relationships it is possible to come together and solve almost anything. (...)

Often in this profession we're asked to deal with difficult situations. No set of rules can guide you safely through every scenario, but I have found that postponement is never the best solution.

Instead of avoiding the hardest parts, double down on them.

If you have a poor relationship with your manager or a member of your team, spend even more time with them. Meet with them every day or have dinner with them. (...)

(...)

As a final thought, the best management philosophy never stands still, but (..) continues to evolve as it comes in contact with reality. The worst theory of management is to not have one at all, but the second worst is one that doesn't change.

Other sections in this chapter:

#4 Culture

A short chapter on efforts that can shift the culture within an organization.

Make your peers your first team is advice I wish I would have read earlier. When I moved into management, I focused most of my effort on the team itself, making sure we were a great team. The same time, I neglected building relationships with fellow engineering managers and other stakeholders. And it never occurred to me to think of my engineering manager peers as my first team, until recently.

The best learning doesn't always come directly from your manager, and one of the most important things a first team does is provide a community of learning. Your peers can only provide excellent feedback if they're aware of your work and are thinking about your work similarly to how you do. Likewise, as you're thinking about your peers' work, you'll be able to learn from how they approach it differently than you anticipate. Soon, your team's rate of learning will be the sum of each other's challenges, not longer restricted to your own.

Long term, I believe that your career will be largely defined by getting lucky and the rate at which you learn. I have no advice about luck, but to speed up learning I have two suggestions: rapidly expanding company, and make your peers your first team.

Other sections in this chapter:

#5 Careers

A chapter on interviewing and hiring processes, as well as coaching engineers grow in their career.

Roles over rocket ships, and why hypergrowth is a weak predictor of personal growth shares good advice on why tenure is a weak predictor on personal growth and how working at a high-growth company doesn't mean you get to grow quickly, without doing much.

Even hypergrowth companies tend to have teams that are largely sheltered from change by either their management or because they're too far away from the company's primary constraints to get attention.

By tracking your eras and transitions, you can avoid lingering in any era beyond the point where you're developing new masteries. This will allow you to continue your personal growth even if you're working in what some would describe as a boring, mature company. The same advice applies if you're within a quickly growing company or startup: don't treat growth as a forgone conclusion, growth only comes from change, and that is something you can influence.

Career levels, designation momentum, etc. is a section that talks a lot about how performance management systems and career levels have to change at a rapidly growing company. Calibration process changes, level splits, expansions, or drifts are all things I've seen happen at Uber, as we doubled every year. While this section is specific to (very) high-growth companies, as a manager, it's good to have an idea of challenges to expect when the old system breaks under a lot bigger organization.

There are, surely, hundreds more interesting topics when it comes to how performance systems work in practice as opposed to in design. Although they seem quite simple, I keep learning something new each time I go through a performance cycle, and I suspect that is a widely shared experience.

Other sections in this chapter:

#6 Appendix

The Appendix comes with a nice surprise: instead of the usual references list, it contains the Tools for operating a growing organization section. This one is a brief refresher on key processes a team lead, a front-line manager with multiple team leads, and a manager an organization wants to have in place.

Line management: Around the time your team reaches three engineers, you'll want to be running a sprint process. There are many successful ways to run sprints, try a few and see what resonates for you. (...)

Middle management: As you move into middle management, you'll become responsible for two to five line managers, and will need to shift away from day-to-day execution to give your line managers room to make an impact (and free yourself up to make a larger impact as well). (...)

Managing an organization: As your organization starts to get even larger and you're mostly managing middle managers, the playbook shifts again. (...)

Along the way, remember that your old problems still exist, it's just that other folks are dealing with them instead. As you roll out new processes to solve your personal pain points, you should be handing off processes to your managers, and keeping them intact and running.

The book closes off with the Books I've found very useful section, where the referenced books are accompanied by a short summary.

Where the Book Could Be Even Better

Given Elegant Puzzle brings so much original content and ideas, with full of applicable ideas for people like myself working at high-growth tech companies, there's not much that I'd hold against it. However, a few things would have made the book an even more pleasant read.

Diagrams in the book are often a few pages away from where they are mentioned and I did not find them adding much context to the text itself. This is especially true for the Kindle version, as well as the large, whole-page diagrams. There are many inline diagrams that work well, but I found myself not getting much from the whole-page, yellow diagrams. These full-page diagrams are one of the few things that I found to work better on Will's website than they did in print.

Inline diagrams like this read well. I wish there was more of these, and fewer of the yellow, full-page ones.

The first chapter struck me as an odd start. Given so much of the book builds on systems thinking, I would have put that chapter/section first. Coming back to the first chapter later made more sense to me.

Some chapters are less cohesive than others. Some sections seemed to be an odd shift in style, content and length. For example, the The Briefest of Media trainings is a short section is between the Career Narratives and Model, document and share sections. There's little anything connecting the three. Also, the formatting is quite different between all three of them. Don't get me wrong, all three sections are good. However, lining them up based on a train of thought might have made for a better read.

The book has no real end. This one I did not mind, as I treat the book more as a reference for various problem areas. However, I do think there was a missed opportunity to have a closing section, recapping the journey the reader and Will went together. Instead, after finishing reading the Designing and interview loop section, we immediately get to the Appendix: and then the book suddenly ends.

All the above, however, are small nitpicks. As I concluded before the book notes, you do want this book on your bookshelf: grab it on Amazon.

Finally, if you are interested in getting updates on my book on (pragmatic) software engineering that I'm starting to write, subscribe below.

Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 tech newsletter on Substack.