Moving to “Micro-service” and “Serverless”

Abstract

“Micro-service” and “Serverless” might be most popular words on software architecture in recent years. Amazon is definitely the pioneer on these areas. I’ve constructed or reconstructed several systems with “Micro-service” and “Serverless” architecture during my 7 years in Amazon. The followings are what I’ve learned from these projects.

Why Micro-service?

Although “Micro-service” is very poplar today, it is not a right reason to adopt the technique just for its popularity.

In Amazon, my team had adopted “Micro-service” architecture very naturally. Frankly speaking, the situation had pushed us to “Micro-service”architecture while our system and team becoming large.

According to my experience, the moment you detect the following bad smells is the time to think about moving to “Micro-service”architecture.

 

Bad smells

In Amazon most of the projects are started from a simple (monolithic) architecture and then scaled by duplicating directly.

The monolithic architecture normallyrelates to single continuous deployment pipeline.

Some bad smells as the following would be detected while the system and organization growing more and more larger. These smells would appear in the different stages of development process.

 

Plan

Nowadays, when starting new projects, engineers like me get used to start from “Google” to check if there are some existing implementations can be leveraged. Unfortunately, sometime the existing implementations might be not compatible with your current technique stack. For example, the existing implementation and the monolithic system are not written in same programming language.

 

Code

When submitting the codes, you always get the integration conflicts. More and more code branches were created by the different teams for the different purposes. And these branches are hard to merge or integrate for the conflicts.

 

Build

When building the version set, the dependency conflicts drive you crazy.  A lot of packages with different versions are introduced to the same versionset,the potential version conflicts introduced by the dependency chains are impossible to avoid.

Deploy

The deployment’s quality is out of control.Each deployment is going to be uncertain due to the different changes from thedifferent teams being carried out together. Shit always happens.

 

Operate

When shit happens, “rollback” is no longer a simple solution. So many unware changes have been rolled out, it is impossible to estimate theimpacts caused by your “rollback”. For example, the new created data might not be used by the previous version deployment.

Move to Micro-service

1 Divide the original system into the propermicro-services. The dividing solution would be defined at least by considering on the following aspects:

- Business domain model (for the details,please, refer to the book "Domain-Driven Design" by Eric Evans)

- OO Design principles. For example, make sure the services high cohesion and loose coupling.

- Team Organization. This aspect is easy to be ignored, but it is very important. It is better to let your organization division align with the services’ division.

 

2 Encapsulate the legacy implementations into the local components. The structure of the components is consistent with the micro-service structure defined in step 1.

3 Wrap the components as a remote service.As the following, to make the process smoothly, proxy pattern would be used this step to transform the local components to remote services.

4 Separate the data to the multiple storages. After taking the above steps, the business logics have been divided into multiple individual services, but thedata is still coupling in runtime. Because all services access the same database they can potentially interfere with one another. For example, if long running CustomerService transaction holds a lock on the ORDER table then the OrderService will be blocked.

 

5 Split the single continuous integration/deployment pipeline to multiple pipelines for different services. If this step is ignored, most of the bad smells would be still there even the monster service are divided into many micro services.

Micro-service common problems and solutions

Network failure

Compared with local invokings, remote invokings might bring extra network failures. Retry is the common solution tohandle network failure.

To avoid of scattering the retry logic everywhere, it is better leverage some library/framework (such as: Spring-retry)to separate the retry logic from the main logics.

public class InventoryProxy Implements Inventory{

       private InventoryClient client;

 

    @Override

    @NeedToRetry(recoverableExceptions= {ConnectionRuntimeException.class },

           retryTime = 3, retryInterval = 1000)   

     public InventoryInfo getInventory(AsinInfo asin)throws InventoryException {

                                    client.getInventory(asin)

        }

       ...

}

 

Hard to test

The system being composed of the remote services is much harder to test than the one being composed of the local components.

By leveraging proxy pattern as mentioned in the previous section, it is easier to test the system with local mocks.

 

Data inconsistent

The data storage separation would break the data consistent, which was guaranteed by database transaction. Considering on the scaling problem, normally 2PC transaction is not an option in Amazon. And adopting compensation transaction solution would introduce the change in the different parts.

The following solution is about implementing eventual consistent by leveraging pub-sub MQ.

The data changes would be published to pub-sub MQ as notifications, and the services, which care about these changes, can subscribe these notification and sync the changes to the related data storage.

Why Serverless?

One important trend in software architecture is that the scalable unit’s granularity is getting more and more fine, which are from monolithic system to micro service, and then to function.

Serverless architecture is getting popular.You can get some basic ideas about what serverless is and why use it fromhttps://cloudncode.blog/2017/05/15/why-serverless-architecture/.

In my Amazon experience, almost half operation efforts relate to the infrastructure maintenance. No matter how simple your implementation is, you would face the same maintenance problems introduced by the infrastructure stacks (e.g. OS, virtual machine, SDK, service frameworks)

Scaling is another benefit brought by serverless architecture. By leveraging FaaS (e.g. AWS Lambda), the system can be scale more dynamically and finely (per request scaling).

Move to Serverless

You can find some serverless architecture templates from https://d0.awsstatic.com/whitepapers/AWS_Serverless_Multi-Tier_Architectures.pdf.

The following are some key points summarized from our real practices.

 

Stateless

By leveraging FaaS to scale your application horizontally, the function implementations should be stateless.

In most software engineers’ minds, to be stateless is normally about leveraging the external storages or caches to store the state outside. But actually, we can do more than that based on other AWS backend services. Program logics about processing pipeline or controlling flow can also be moved out of the function implementations by leveraging AWS data pipeline and function steps. This would not only help us make the function stateless,but also help us keep the function small.

 

Small

In AWS serverless solution, the implementation of AWS Lambda function is better to be small. There are some limits of AWS lambda as followings. You should be careful about the function’s execution time and resource to make sure it not beyond its limits

   

Asynchronous

Serverless architecture and FaaS are still in its early stage. Its capabilities still need to be proved in real practices,and many areas are still need to be improved. For example, AWS lambda implementation runs in the containers. When scaling out, the running environment would be initialed firstly before running. That would be challenging to handling the largeamount requests in real time by AWS lambda.

Today, leveraging these serverless foundation in asynchronous way is more scalable and safer. Also, AWS Lambda has well supported the different kinds of asynchronous solutions (e.g. trigger the function by SNS notification, knessis messages and DynamoDB/S3 data events)