Microservices - Not A Free Lunch!

Microservices - Not A Free Lunch!

TUESDAY, APRIL 8, 2014 AT 8:54AM

This is a guest post by Benjamin Wootton, CTO of Contino, a London based consultancy specialising in applying DevOps and Continuous Delivery to software delivery projects. 

Microservices are a style of software architecture that involves delivering systems as a set of very small, granular, independent collaborating services.

Though they aren't a particularly new idea, Microservices seem to have exploded in popularity this year, with articles, conference tracks, and Twitter streams waxing lyrical about the benefits of building software systems in this style.

This popularity is partly off the back of trends such as Cloud, DevOps and Continuous Delivery coming together as enablers for this kind of approach, and partly off the back of great work at companies such as Netflix who have very visibly applied the pattern to great effect.

Let me say up front that I am a fan of the approach. Microservices architectures have lots of very real and significant benefits:

• The services themselves are very simple, focussing on doing one thing well;
• Each service can be built using the best and most appropriate tool for the job;
• Systems built in this way are inherently loosely coupled;
• Multiple developers and teams can deliver relatively independently of each other under this model;
• They are a great enabler for continuous delivery, allowing frequent releases whilst keeping the rest of the system available and stable.

This said, Microservices are not a free lunch!

I am currently involved in architecting a system based around Microservices, and whilst the individual services are very simple, a lot of complexity exists at a higher level level in terms of managing these services and orchestrating business processes throughout them.

Microservices one of these ideas that are nice in practice, but all manner of complexity comes out when it meets reality. For this reason, I wanted to write this article to capture some of these and redress the balance.

Significant Operations Overhead

A Microservices architecture brings a lot of operations overhead.

Where a monolithic application might have been deployed to a small application server cluster, you now have tens of separate services to build, test, deploy and run, potentially in polyglot languages and environments.

All of these services potentially need clustering for failover and resilience, turning your single monolithic system into, say, 20 services consisting of 40-60 processes after we've added resilience.

Throw in load balancers and messaging layers for plumbing between the services and the estate starts to become pretty large when compared to that single monolithic application that delivered the equivalent business functionality!

Productionising all of this needs high quality monitoring and operations infrastructure. Keeping an application server running can be a full time job, but we now have to ensure that tens or even hundreds of processes stay up, don't run out of disk space, don't deadlock, stay performant. It's a daunting task.

Physically shipping this plethora of Microservices through your pipeline and into production also needs a very high degree of robust and release and deployment automation.

Currently, there is not much in terms of frameworks and open source tooling to support this from an operational perspective. It's likely therefore that a team rolling out Microservices will need to make significant investment in custom scripting or development to manage these processes before they write a line of code that delivers business value.

Operations is the most obvious and commonly held objection towards the model, though it is too easily brushed aside by proponents of this architecture.

Substantial DevOps Skills Required

Where a development team might have been able to bring up, say, a Tomcat cluster and keep it available, the operations challenges of keeping Microservices up and available mean you definitely need high quality DevOps and release automation skills embedded within yourdevelopment team.

You simply can't throw applications built in this style over the wall to an operations team. The development team need to be very operationally focussed and production aware, as a Microservices based application is very tightly integrated into it's environmental context.

Idiomatic use of this architecture also means that many of the services will also need their own data stores. Of course, these could also be polyglot (the right tool for the job!), which means that the venerable DBA now needs to be replaced with developers who have a good understanding of how to deploy, run, optimise and support a handful of NoSQL products.

Developers with a strong DevOps profile like this are hard to find, so your hiring challenge just became an order of magnitude more difficult if you go down this path.

Implicit Interfaces

As soon as you break a system into collaborating components, you are introducing interfaces between them. Interfaces act as contracts, with both sides needing to exchange the same message formats and having the same semantic understand of those messages.

Change syntax or semantics on one side of the contract and all other services need to understand that change. In a Microservices environment, this might mean that simple cross cutting changes end up requiring changes to many different components, all needing to be released in co-ordinated ways.

Sure, we can avoid some of these changes with backwards compatibility approaches, but you often find that a business driven requirements prohibit staged releases anyway. Releasing a new product line or an externally mandated regulatory change for instance can force our hand to release lots of services together. This represents additional release risk over the alternative monolithic application due to the integration points.

If we let collaborating services move ahead and become out of sync, perhaps in a canary releasing style, the effects of changing message formats can become very hard to visualise.

Again, bckwards compatibility is not a panacea here to the degree that Microservices evangelists claim.

Duplication Of Effort

Imagine that there is a new business requirement to calculate tax differently for a certain product line. We have a few choices in how to deliver this.

We could introduce a new service and allow the other services to call into this where needed. That does however introduce more potentially synchronous coupling into the system, so is not a decision we would take lightly.

We could duplicate the effort, adding the tax calculation into all of the services that need it. Besides the duplicated development effort, repeating ourselves in this way is generally considered a bad idea as every instance of the code will need to be tested and maintained going forward.

The final option is to share resources such as a tax calculating library between the services. This can be useful, but it won't always work in a polyglot environment and introduces coupling which may mean that services have to be released in parallel to maintain the implicit interface between them. This coupling essentially mitigates a lot of the benefits of Microservices approaches.

It seems to me that all three of these options are sub-optimal as opposed to writing the piece of code once and making it available throughout the monolithic application. The teams I have seen working in this style tend towards option 2, duplicating of business logic, which goes against many principles of good software engineering. And yes, this even takes place in well decomposed and designed systems - it's not always a sign of bad service boundaries.

Distributed System Complexity

Microservices imply a distributed system. Where before we might have had a method call acting as a subsystem boundary, we now introduce lots of remote procedure calls, REST APIs or messaging to glue components together across different processes and servers.

Once we have distributed a system, we have to consider a whole host of concerns that we didn't before. Network latency, fault tolerance, message serialisation, unreliable networks, asynchronicity, versioning, varying loads within our application tiers etc.

Coding for some of these is a good thing. Backwards compatibility and graceful degradation are nice properties to have that we might not have implemented within the monolithic alternative, helping keep the system up and more highly available than the monolithic application would be.

The cost of this however is that the application developer has to think about all of these things that they didn't have to before. Distributed systems are an order of magnitude more difficult to develop and test against, so again the bar is raised vs building that unsexy monolithic application.

Asynchronicity Is Difficult!

Related to the abovei point, systems built in the Microservices style are likely to be much more asynchronous than monolithic applications, leaning on messaging and parallelism to deliver their functionality.

Asynchronous systems are great when we can decompose work into genuinely separate independent tasks which can happen out of order at different times.

However, when things have to happen synchronously or transactionally in an inherently Asynchronous architecture, things get complex with us needing to manage correlation IDs and distributed transactions to tie various actions together.

Testability Challenges

With so many services all evolving at different paces and different services rolling out canary releases internally, it can be difficult to recreate environments in a consistent way for either manual or automated testing. When we add in asynchronicity and dynamic message loads, it becomes much harder to test systems built in this style and gain confidence in the set of services that we are about to release into production.

We can test the individual service, but in this dynamic environment, very subtle behaviours can emerge from the interactions of the services which are hard to visualise and speculate on, let alone comprehensively test for.

Idiomatic Microservices involves placing less emphasis on testing and more on monitoring so we can spot anomalies in production and quickly roll back or take appropriate action. I am a big believer in this approach - lowing the barriers to release and leaning continuous delivery in order to speed up lean delivery. However, as someone who has also spent years applying test automation to gain confidence prior to release, anything that reduces this capability feels like a high price to pay, especially in risk averse regulated environments where bugs can have significant repercussions.

To Conclude

These are some of the difficulties I am seeing during the early phases of building and running a Microservices based system.

I am still a fan of the approach and believe that on the right project with the right team it is a wonderful architecture to adopt where the benefits outweigh the costs above.

However, when considering Microservice like architectures, it's really important to not be attracted to the hype on this one as the challenges and costs are as real as the benefits.

 

Related Articles

• On Hacker News

Todd Hoff | 20 Comments | Permalink | Share Article Print Article Email Article

Tweet

Reader Comments (20)

I am moving to a microservices architecture because I'm moving beyond things that are affordable to do with a monolithic architecture. A big issue is that I find there are a very queries that have high latency and to get the latency down with a monolithic system means expensive overprovisioning or alternately, carefully teasing the system apart to optimize the data structures for this query without breaking any part of the system.

Here are my take on the challenges.

Even though people will celebrate the benefits of decoupling here, in the sense that one server could be written in LISP and another in COBOL and another in FORTRAN, this is only a low-cost, high-quality answer if there is a lot of standardization. For instance, if you are working in Java, you may plan to deploy all of the microservers as Tomcat applications or even be more specific and standardize on using Jetty.

You need to have out-of-the-box answers for all of the cumbersome things as to how you package microservers for deployment, how serialization and deserialization work, how you configure database connections and things like that. You really need a template of some sort so a developer can create a new "Hello World" service that is immediately deployable into production in 5 minutes. If it isn't that easy, people are going to find it easier to do things in random ways and then life will be hell for devops.

A deeper architectural problems is minimizing coupling, not in a cargo-cult way, but in a real way. One problem I have seen with SOA apps is often a change to an app such as adding a new field to the data model requires changes to a large number of different servers. One person might be tasked with this "simple" change, but making it involves tracking down everything affected by the change and understanding a number of code areas that were written by different people, probably deploying something that is only 95% to production and then the stressful problem of fixing the last 5% in a hurry.

I was talking with a Ruby on Rails guy and he was like "What's the problem? Can't you just add a field?" and the point is that you can in RoR because it is architected that way, and a microserver architecture should be also. There should be centralization of serialization-deserialization, for instance, so you can make a change in one place and not have to do a lot of tedious and error-prone stuff.

Connected with that is the issue of data updates and data integrity across the system. If you tried to put multiple microservers inside a distributed transaction boundary I think you would lose the benefits of microservers. Without transactions I think it's inevitable that minor data corruption will occur in the system. It's got to be built to sweep those problems under the rug and to clean up problems over time. (You can't ever throw a 500 error because data you wanted was missing) Connected with that is some systematic approach to updating. If you change one data item, maybe that means you update 20 microservers. Unless you have some strategy for updating that is provably correct, I think it's almost impossible that you'll get it right, particularly in a system that is always changing to meet new requirements.

April 9, 2014 | Paul A Houle

Being one of the tech leads on transforming a monolithic Java application to a SOA implementation, I've come across everyone of the issues you raise but instead of seeing those as problems I see them as opportunities to build software better.

First of all, as an aside, no one can clearly explain the difference between a micro service and a service. My service might be your micro service and your service might be my micro service. They are both services and should adhere to the same engineering principals. At our company we go with a service definition that says a service is a "business capability".

You say "Substantial DevOps Skills Required". I see that as a good thing. It gives the people writing the code the responsibility of how it runs in production. Going to a SOA implementation almost forces you to a de-centralized devops where the service team devs do the devops as compared to the old school "throwing it over the wall" to the centralized operations team. It's a big positive to have the dev team be responsible for the operations of their code.

I'd like to address your issue of increased operational complexity. Yes, there are more services compared to a monolithic application to build, test, and deploy but in today's world those things should all be automated. Having two vs. twenty that follow the same automated patterns should not be that much more work.

I agree with many of your points on "Implicit Interfaces" but I would point out that most of your points apply to any API whether in a service or a monolithic application. It's never a good thing to change the shape of an existing interface. That will end up bad. Most companies tend to create a new API version for interface shape changes. With a good API design, you want those to not happen very often and you should design your service and supporting systems in a way that you can still do daily code commits into production without changing the interface contract.

You mention that duplication of code goes against many good principals of software engineering. I used to be a purest in this area in that any and all code duplication is bad but then realized this purity sometimes resulted in much more costly and complex solutions and no one won but idealism. I'm now more practical in this area and simplicity has to be a part of the equation as well. I really like what Richard Gabriel wrote in The Rise of Worse is Better about this.

Testing in a SOA world can be complex due to other service dependencies but thinking about how you are going to test your service up front may end up saving you lots of time in the end. You will not have a good time if you design a service that has many other services that you require to be up and running while you test your service.

Asynchronous programming can be difficult but it doesn't have to be. Things like functional reactive programming and different async libraries like the Clojure core.async and Node.js async help developers be more productive in writing async code. But as a programmer, whether or not you use services, bets are you need asynchronous behavior somewhere in your application. So I don't see this issue as unique to services.

Thanks for addressing the subject of micro services since many of us are eager to learn more about this and can share our experiences.

April 10, 2014 | Steve Willcox

I recently wrote about this topic at peopleandcode.blogspot.in. It is not a free lunch, but you always need to know the trade offs. The decision to partition the system into parts (services or otherwise) needs Agility at the heart of the organization.

April 11, 2014 | unmesh joshi

Hi Ben

Thanks for a great insight into the realities of building a microservice architecture from the ground up. I'm a big fan of the microservices approach - particularly for Continuous Delivery - but there is definitely a sweet spot in terms of service composition.

I think of a microservice as an endpoint hosting a single business capability, whereas a lot of people go further and think of an endpoint hosting a single infrastructure capability. I want a service to have a single reason to change - a business reason to change - with minimal dependencies and minimal impact upon the rest of the estate. If this is accomplished then enormous benefits can be delivered. Once you drill down to infrastructure-centric microservices, you will face a lot of trade-offs in terms of operations/monitoring investment, distributed transaction management, etc.

Your point about microservices operability being dependent upon DevOps is a good one. I've certainly see monoliths broken up into microapplications or microservices by development teams, with the net result being an unhappy operations team swamped with new services to monitor and manage.

A final point - there is a cheap solution to your concern about Implicit Interfaces, and the answer is Consumer Driven Contracts (http://martinfowler.com/articles/consumerDrivenContracts.html and http://www.alwaysagileconsulting.com/application-pattern-consumer-driven-contracts/). Consumer Driven Contracts enable interfaces to rapidly evolve with minimal impact upon consumers, and that is crucial in a microservices architecture.

Cheers

Steve

April 12, 2014 | Steve Smith

For anyone reading this in London we have organised an event around the subject of Micro Services at the Stack Exchange offices in Old Street. Rackspace, Zeebox and British Gas connected homes are doing presentations on how they are organising their development teams around many small services.

The address for the meetup is http://www.meetup.com/DevOps-Exchange-London/events/169675532/

April 13, 2014 | Steven Acreman

I'm one of those enthusiasts about micro-services architecture. I agree with you regarding almost all mentioned points but I believe this architecture as many others shouldn't be taken as a whole if you don't have all resources needed for that.

However, even if your team members don't have DevOps knowledge to deploy and monitoring micro-services you can use the concept of the architecture. You can develop your services as if they would run distributed but end up packing them together into a big monolithic application, until someone from your team manage to automate things, making the deployment of each micro-service easy for all team members.

Take it slowly, get your team familiar with it, I'm doing it now, I hope it works. As soon as I get some results I will add more comments.

April 13, 2014 | Eduardo Nunes

Nice post. I was working on a project moving towards micro-services from a monolithic application and we ran into a lot of the hurdles mentioned in your post. We ended up having a lot of code duplication (since the services were built on different languages and frameworks) - lots of little "implicit contracts", for example mapping User data from one service to another (one service not necessarily needing all the same data as another). While there are some clear benefits of the approach, probably not something you want to jump into without some careful planning. Our approach in the end was to modularize the monolithic application (so we can share code repository, deployments, and code between modules - but still have nice, loosely coupled components) and pull out the modules into their own independent micro-services that can be deployed/managed independently only if really necessary.

April 13, 2014 | Brady

Of course, we all know there is no such thing as a free lunch. Trade-offs are unavoidable in ours and most other enginerering disciplines. It all depends on the problem you are trying to solve. Sometimes a monolithic style may work best. I've been researching the topic for over a year after over 12 years of being frustrated with J2/Java EE apps. My thoughts are shared at microsvcs.io.

April 15, 2014 | Jason Chambers

Quality post and commentary. I'm only just discovering SOA as I bounce between refining requirements and architectural planning for a project of mine. It's really small compared to the projects I usually see referenced in discussions like this, but it will definitely benefit from such informed criticism. Please continue to share!

April 17, 2014 | Kirby

Great post Ben - we're currently implementing CD pipeline automation framework for a client that has over 450 developers working across 50 services (or microservices). To me one the most fascinating aspects of this architecture is none of those 450 developers will ever write a single line of code to support a customer facing user interface. All customer facing UX work is performed by a different group entirely.

While the level overall flexibility, risk reduction and cost savings this client currently enjoys is significant and a direct result of moving away from a monolithic architecture, there is no doubt a very real "microservice tax" paid due to many of the factors you very articulately outline in your post!

-

April 17, 2014 | Dennis Ehle

Sounds like the same old, same old, issues the software industry has been grappling with since I came aboard in the late 80s. All that's happened is that the issues have been relabeled, renamed, and kicked up a few layers / degrees of abstraction. Your microservices are equivalent to 80s subroutines and functions. The issues with interfaces and the propagation of changes HERE to changes THERE and EVERYWHERE, are exactly the same as we faced "back then" in trying to change a function signature. Your multi-disciplinary team is the 80s' one-man-one-vision programmer who knew all the languages and the entire architecture of an entire system -- or maybe that's your hard-to-find-and-hire DevOps-knowledgeable developer (I'm a little handicapped by not actually knowing what "DevOps" IS; but I'm sure if I did, the 80s analog for that, too, would be easy to spot). We "solved" these 80s problems by developing Object-Oriented technologies -- so, clearly (to me), what the world needs now is the higher-level-of-abstraction analogue of OO. Call it whatever you want; what you're looking for is a hyper-object that somehow encapsulates all the little microservice contracts/APIs into an abstraction that hides all that. LEARN FROM THE PAST, dudes.

April 23, 2014 | Chris Chiesa

As the author of SwarmESB, I agree with many points that the author and others in comments made about micro-services but I want to tell you about a new programming model ( I call swarming) that is related with older academic experiments (mobile code, an inverted perspectives of actors,etc).

For SwarmESB I use Node.js, because node is good for implementing the swarming concept but enterprise people (eg Java ) could easily use SwarmESB as an integration point for services exposing REST APIs. The main point is that service orchestration can be programmed in java script plus an abstract model of moving execution in nodes as required (swarm scripts execution). Transactions, monitoring and other aspects discussed here can be part of the swarm level of the ESB.

April 25, 2014 | Alboaie Sînică

Great post. I'm working at an organisation that has faced all the challenges you mention in their microservices implementation. And like some of your other readers have mentioned, it has provided opportunities for devops and design skill development, which has greatly has improved the culture and capabilities of the organisation.

The impact of several of the problems you mentioned (integration testing, setting up test environments, the inability to easily visualise the impact of changes, versioning and backwards compatibility) has been mitigated by our use of consumer driven contracts, specifically the use of the open source "pact" library, which I have had the pleasure of contributing to. It gives us the ability to run standalone CI builds, without extra environments, and know immediately if any change to our services will break anything. The ability to mine the relationship data to create visualisations and autogenerated documentation is also particularly useful, as conceptualising the interactions of 50+ microservices can become an overhead in itself.

May 22, 2014 | Bethesque

The concept of miroservices is not really new, the term is. Good software engineers think/design modular code. I have been in polygot environments with "microservices" deployed that had all the above challenges you speak of. Netflix makes a big deal of microservices but most people skim over the fact that they have a homogenous java based environment for the most part.

I wanted to see how other people were tackling the challenge while still staying true to SOA ie. how do I keep my service contract clean/consistent and yet build small microservice functionality, how do I inculcate the DevOps mentality and meet business demands in an agile fashion. Through my (re)search, I came across this github project 'lightblue' - https://github.com/lightblue-platform/lightblue/wiki. These guys are putting a different spin on the microservice concept, building pluggable modules that I can use for backend interaction (SQL, NoSQL, libraries) while keeping the contract for applications to call the service consistent and search based. With the world moving towards big data, what better way to ensure your APIs meet changing demands than building search based APIs. Standardizing on the CRUD layer and then building modules seems to me a very intuitive way to approach the problem, and I get to decide if I want a monolithic layer or a microservice layer.I know I'm going to keep a watch on project lightblue.

June 20, 2014 | hitchhiker

About this:

"We "solved" these 80s problems by developing Object-Oriented technologies -- so, clearly (to me), what the world needs now is the higher-level-of-abstraction analogue of OO. Call it whatever you want; what you're looking for is a hyper-object that somehow encapsulates all the little microservice contracts/APIs into an abstraction that hides all that."

I would put the metaphor somewhat differently. Micro-services can be compared to objects. But then the question comes up, what is the equivalent of dependency injection for micro-services? The difficulty of wiring objects together was solved with dependency injection. What is the metaphorical equivalent for wiring together the micro-services?

June 29, 2014 | Lawrence

The glue has to be a standard language or "protocol" - and it has to be designed to accommodate failure. That's the fun part about designing languages, in general. :)

I have limited tech experience, so do pardon the abstraction of my comment.

July 27, 2014 | Jerng

Hi,

we started to divide our monolith into some business oriented services. A service comes around a business unit - so it is not micro but comprehensible.We will have a kind of structure for communication, that a service can poll data but did not push directly, and have no dependencies between the services. So far theory.
But we have always again discussions, where to see the boundaries or home for some specific data. Is there a recommendation, how to deal with data and belonging operations that have dependencies to two or more services. (formerly there were tables with several foreign keys that points to other parts of the application).

Thanks

December 3, 2014 | ahs

Great post and valid points on the hurdles to making microservices a reality when moving from a monolith or even starting from scratch. Like some above commenters said developing and thinking in microservices is not necessary something new, but I think we now finally have a word for it and thus are able to communicate it better and discuss about what a true microservice architecture should look like. I'm seeing more and more discussion on fat vs. thin services, inclusion vs exclusion of DBs, "SOA = microservice or only a predecessor" and many other parts of the microservice topic being discussed nowadays. We are also seeing an increase in Google search volume, so more and more people are actually interested in this.
There are not only new problems arising with this, but also a lot of new tools and such that need to be learned and deployed. We also see some other trends (or hypes) getting combined into the microservice trend. For example the revival of container technologies, especially with Docker getting more and more widely known and stable. And it's easy enough for people to actually use it (at least for dev and testing). I think Docker and similar (or future) container technologies can help with at least some of the problems mentioned (especially when it comes to decoupling dev and ops dependencies). However, many of the mentioned problems still remain, especially when it goes into production and deployment. Handling your operations yourself is already quite some work without microservices, but with them as you also mentioned, the work just increases manyfold. Suddenly you have to take care of several servers, environments, message queues, load balancers, clustering,...We ran into the same problems and we tried a lot of stuff. Sure Netflix, Twitter, Spotify, Shopify and the likes can pull this feat off. But even those have a lot of self-developed stuff and "hacks" running in between to make it bearable for dev and ops alike. Thus, we thought maybe we could build something for the rest of us and got a running alpha now on Giant Swarm. It's a simple infrastructure, where you describe your microservice architecture (based on Docker containers) in a simple JSON file and we take care of the rest, incl. clustering, scaling, load balancing and such. With time we will also add more specific microservice "enablers" like a ready to use message queue for example. Seems like a lot of you here are interested in or already working with microservices, so we would be happy to let you try out our system and tell us what you need from a perfect microservice infrastructure, so we can build it for you.

December 18, 2014 | Puja Abbassi

This all micro services idea has roots in UNIX commands which than have to use shell to have meaningful application.
Another words access to micro services has to be simplified so another layer has to be in place. Is it performance hit ?

December 22, 2014 | John Smith

First, very interesting post.
As someone says, nothing really new with microservice. We had similar problems and questions with corba. Why ? Because the problem is not technology but is about business and urbanism. 
The most important is not really the size of the application but the duplication or not of responsibilities. In an IS the most important is to clearly define who is responsible to do what. And this must be organized around domain models and use cases. We often forget the domain model in SOA like approaches. And this is the difficulty, what are the rights, just sub-context that allow me to define the good sized application; with enough responsibility and not to much dependencies with other applications ?
So the problem is more about urbanism and domain model sharing than the size of your application or service

December 24, 2014 | erik gollot