In the Microservices world, APIs are hell.

No, let me put that another way: REST is hell. SOA is hell. RPC is hell. Anything that looks like a pipe? Hell.

The promise of the Microservices Architecture is that of being able to expand the scope of an application by dropping new services into an existing composition, and have that new service consume and produce data to improve the composition as a whole. The reality of Microservices is, very often, one of constantly rewriting services to account for the addition of new services, either to supply them with endpoints, or to make those services aware of the new data structures available for consumption, or to ensure that data is properly routed through the right services as it is transformed.

It’s not such a problem when you have straight relationships between services - for example, a user registration service, a user log-in service and a data store service. Neither of the two user-facing services would have to be aware of each other and the database service only exists to read and write database records. The problem starts when you want to insert new functionality into the composition.

Lets look again at our hypothetical application. At the moment, it registers users and gives them a way to log in.

Now lets say we want to create a record of default settings when the user registers and send out an e-mail to confirm their registration.

If your instant thought was “the registration service handles that”, stop now and consider the words of Doug McIlroy from so very, very long ago.

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”.

This is the first point of the summary of McIlroy’s Unix Philosophy, or the Unix Architecture if you want to give it a more important sounding name. It’s an important point, one that bears repeating as often as you can remember to do so. Make it your koan every morning when you wake and every evening when you drunkenly stumble into bed.

What is the Unix architecture in the end, but a linear composition of microservices? Admittedly our hardware requirements are somewhat broader, but the principle is the same. It would be admirable if we could compress the footprint of our services to the same degree as Unix. Perhaps one day we will.

Back to our composition.

As you have gathered by now, the User Registration Service should do one thing, and one thing only: register users. It shouldn’t be mucking about with other part of the system than that. It shouldn’t be defining arbitrary settings. It most definitely should not be sending e-mail.

This new function should be the responsibility of a new service, which we can call the Settings Service. That new service has to somehow insert itself into the conversation flow between the existing services and generate a new default settings record for the new user.

The question is: How to achieve that?

The RPC way

In the “traditional” world of RPC (or REST, or whatever else), where services have endpoints and talk directly to one another down the infamous series of tubes, we have a few options:

  1. Have the registration service populate those settings itself,
  2. Have the registration service pass the details off to the new settings service,
  3. Have the database manager service populate or pass off the registration information to the settings service,
  4. Have the new settings service periodically poll the database new users and populate settings for any users it finds.

We’ve ruled out (1) immediately. Our existing service is not going to do new things.

We can also rule out (3), as it is beyond the scope of the database service to be handling such things. It’s a dumb service: all it should do is consume and produce database records.

This leaves (2) - have the registration service pass off the details to a new service - and (4) - have the new service poll the database. I think it’s safe to rule out (4) as inefficient, though in an RPC world that might sometimes become an option for reasons that will be apparently shortly.

For now we’ll go with (2), and create a new service that the registration service will send data to.

… except we have hit an immediate issue: We need to rewrite the registration service so that it is aware of the new settings service, which means writing a new API to communicate with and having a new - and potentially fragile - binding between these two services.

You might think that’s not so unreasonable, but we’ve yet to consider the other service we mentioned: e-mail. Now we have to make the registration service aware of a third service, with a new API, resulting in another link.

The problems will grow every time you want to add more functionality. Perhaps you want your users to be automatically registered into a slack channel, or to be automatically added to a google address book, or… well whatever you can think of, you’ll have to add a new service for the registration service to be aware of.

What are we doing here, but adding new features and complicating existing programs?

Now you may have been clever. You might have seen this coming from the start and said to yourself “I can come up with a way for services to advertise what they can do and discover one another, and know what end-points they have and how to talk to them.”

Which is great! But it’s also unwieldy and - to be frank - needlessly complicated in many cases. The cases we just considered in particular would work under such a system, but they would be untidy. All these services do is listen for a particular piece of data and act on it in some way. They all want the same piece of data, so why should we create this web of relationships and send out that one identical piece of data multiple times to multiple services?

Do we have a choice?

The Event-Driven Way

The registration service sends an RPC request to the database service, recording the new user, and then emits an event about it.

Several other services are listening for that event and do their thing when it comes through.

Isn’t that so much easier?

Now I know what you’re thinking: Isn’t that just the “services advertise to one another” thing from before?

It looks that way, but it isn’t. Yes, services can be aware of one another in an event-driven architecture, but they don’t have to be. They don’t care what services consume the event they just published, or where an event comes from; they see the event and act on it, whether that’s to emit other events, change the state of a react component or just quietly make a note of the event for logging purposes.

The Ties That Bind

The Event-driven architecture, and the related Event Sourcing, is a ridiculously simple way of doing things once you’ve grasped the idea of the event loop. It simplifies the interaction between services, allowing the implementation of new services at nearly no cost to the existing composition.

In fact it’s so easy that a neophyte is likely to take it and immediately say “everything will be event-drive now!”, and subsequently write themselves into a complete mess as they try to cut their way free of the tyranny of the RPC web.

Notice that I didn’t do that, though? In our hypothetical composition, the Registration Service is aware of the database service and communicates directly with it via RPC. This is a specific design choice in this scenario, for two reasons:

  • The registration service needs to know that the database service is available in order to ensure proper feedback for the user. By contrast, all of the other services only need to know that there has been a successful registration, but the registration service doesn’t need to know that they know, or even that they exist.

  • The database does not want to be aware of any services. It could be an event-driven service, but then it would have to listen to a special “add-this-to-the-database” event, which all services would have to emit when they want to do database things, or it would have to be aware of the multitude of events emitted by different services at different times that needed to be recorded. It is much, much simpler for it to present a set of simple interfaces with which other services directly communicate.

In the RPC-only way of doing things, the Registration Service would have to know about those services and send that event to every single one of them, wasting processing time and bandwidth, and taking on unnecessary features to do so. If it has to take part in numerous tasks that are tangent to its single purpose, it is no longer doing just one thing, but a myriad.

In the Event-Driven way of doing things, all the Registration Service has to do is register a user.

RPC is hell - if all you use is RPC.

Event-driven architecture is also hell - if all you use is Events.

Like so many of the best things in life, it’s in the mixing that we find the best results.