Architecting Microservices for Kubernetes

A modern app is best described as a network of services. This includes both internal services that you are developing and hosting, and external services hosted on the public internet (such as the Salesforce API, Twilio, or some weather API). The services you host yourself inside Kubernetes are often called a service mesh, and that service mesh is often managed by Istio. We’ve looked at many of the tools and strategies related to Kubernetes, and as this series draws to a close we’ll look at designing the services themselves.

Let’s take the example of an eCommerce application. If it were monolithic, it would be one giant run time, or perhaps split into a couple of tiers. With microservice architecture we need to break it down into logical, functional units, such as a shipping service. The shipping service will be an internal REST API that we develop from scratch, and it will have various endpoints to serve various related functions. For example, perhaps an endpoint to retrieve current rates based on a packages weight, and another to actually process a shipment. To do it’s work, the service would likely interact with several external APIs from organizations such as Fedex and UPS.

The first step to architecting microservices, is to determine what the initial set of services will be. For eCommerce, we might have one for users, one for shipping, one for inventory, another for payments, and perhaps another for ratings and reviews. There could be others, for example perhaps there should be one for orders, or instead we might decide orders can simply be part of the users service. To make that decision we might consider the scope and complexity of the ordering components, and if they will be the responsibility of the same team developing the users service or not.

Before we get into architecting specific services, we may want to make some general decisions for the entire application. For example, it’s popular to have separate databases for each stateful service, but there could be contextual reasons that the tradeoffs there are not worth it, and instead you might share a database across the application. Other decisions like service discovery, authentication, and monitoring and logging would likely be standard across services.

When looking at a specific service, one of the first questions to consider is if the service can be stateless or if it must be stateful. If it can reasonably be stateless, it probably should be. For more on managing state in Kubernetes, see our earlier post on that topic. The shipping service could likely be stateless, since it simply needs to make requests and return results, it does not need to internally store state. Unless we decided it was critical to cache current shipping rates rather than request them on the fly, then that cache would be some amount of state, though we might be okay with it that cache being somewhat ephemeral.

By contrast, the users service would most definitely be stateful. We need to keep track of usernames, profile information and more inside a database, and that is definitely stateful. In that case, we have to be much more careful to properly isolate state so that we can scale horizontally with causing inconsistencies in user experience.

Another concern is how much variety you allow in programming languages, dependencies, and versions for each. On one hand, an advantage of microservice architecture is that you are free to choose different combinations for each service. On the other hand, that makes it harder to keep track of known vulnerabilities, and keep everything up to date. You might decide that based the size of the dev team that will be responsible, and what expertise they have.

You’ll also need to consider how large each service should be, and at one point you might consider breaking one service into two. There is no hard and fast rule to this, but there are some common factors that might influence it. If you have a very large dev team with many cross functional teams, and your code base will quickly grow to be massive, it would likely make sense to have smaller services, since the added complexity would pay off in flexibility for each team.

There is no end to the things to consider when making architectural decisions, but hopefully this gives you a sense of the task ahead. Remember that the modern app is a network of services, both those you build, and external services you consume. Create a diagram mapping the core components of your app and how they will interact, and work toward a logical grouping of services. Keep your stateful services small where possible, and computationally intensive services stateless where possible. And always, if you’re looking for some guidance, feel free to reach out!