Why it's time to take microservices to the next level
If you’ve ever worked with engineers, you’ll have heard them talk about microservices. A few years ago, you might’ve been left with the impression that they were ‘The Answer’ - the thing that would solve all your problems.
‘The Answer’ used to be SOA (Service-Oriented Architecture). There’s always ‘The Answer’. It just keeps changing. This is partly because the enabling technology continues to advance rapidly and partly because engineers just love starting again. They like to build brand new systems using the shiniest shiny things (whilst simultaneously learning how to use said shiny things).
I make a point of referring to them as ‘services’ (because that’s all they are, really). At some point, I predict that we’ll stop talking about microservices altogether. Does anyone remember when we used to talk about ‘the microcomputer’? That was because computers used to be massive and (micro)computers like the BBC (Model B) really were ‘micro’ by comparison. But as they became commonplace, the ‘micro’ was dropped. The same thing will happen here.
But anyway. The principles are sound. Small pieces of functionality that each do one thing well, loosely-coupled, but with surrounding tools and infrastructure that provides clarity, transparency and consistency. They’re not a panacea, they’re not the proverbial Golden Hammer. With care and experience, they can provide the means of launching, scaling and evolving a robust and performant system. But to work well they all need to be built in a similar way with consistent supporting tools and connections. Things like log aggregation (can I see all the logs from all the services in one place?), tracing (can I follow one transaction across multiple services?), service discovery (how do my services know where the other services are?). That’s fine in a team or department. But what about where you have to think bigger - a bank? If the goal is pluggable architecture that can be reused and upgraded, how do we enable that at Enterprise scale?
If the goal is pluggable architecture that can be reused and upgraded, how do we enable that at Enterprise scale?
I once had a conversation with someone who worked in a large financial services company. Their systems were a hot mess. Through a series of mergers, acquisitions and reorganisations, they’d found themselves with 10 Lines of business (LOB). Each had its own application stack and infrastructure. And at some point, somebody (or somebodies) in each LOB had happened upon ‘The Answer’ (see above). This resulted in 10 or so separate sets of microservice architectures, all wildly inconsistent. This wouldn’t have been so bad if never the, er, 10 shall meet. But at various points, leaders in each LOB had decided - no doubt during discussions around the avoidance of reinventing wheels - to reuse components that already existed, connecting seemingly-random services across LOBs.
They’d created not so much the ‘Architectural Ball of Mud’, but a whole sack of balls of mud, some tied together with string, some with Sellotape and some with fishing twine. It was a mess, leaving the team in constant fear that something would go wrong and they’d have no way of working out where the problem lay. To the tentative question: ‘do you have any form of log aggregation?’ came the exasperated reply: ‘mate, we’re drowning in logs. We don’t know what to do with them all!’ (the result of 10 instances of five types of log aggregator, all ‘aggregating’ to different places).
So. What should that look like and how do you get there? Microservices are a great concept. ‘Does one thing well’. But the definition of ‘one thing’ broadens as you go up the stack. If I’m building a simple ledger service, I need one of those things to be ‘Authentication’ so I’ll have a microservice that handles that. But at a level up, maybe I’m building a card payment and current account app and one of those things will be ‘a simple ledger service’. So my building blocks are always the right size for the service I’m building. It’s kind of fractal, I s’pose. Software Architecture 101 - you need to ensure you encapsulate your service. Your service consists of a cluster (or ‘constellation’, if you will) of microservices, but there should be a very clear and consistent interface.
you need to ensure you encapsulate your service.
In the nasty example above, part of the problem was that microservices were connected across service boundaries, breaking the encapsulation. Getting this right isn’t complicated but it is hard. It requires alignment and buy-in across an organisation at a senior level. Newsflash: the technology may not be the hard part. We’ve seen this a lot at 11:FS and we believe it’s one of the major barriers to innovation, time to market and speed of change. But it is solvable. We wrote about it here.
We’re here to help build up your tech teams and offerings. Get in touch with one of our experts.