Last month, I attended the workshop on Microservices by Fred George at Agile India 2015, Bengaluru. I must say it was really a good one and learnt quite a lot. I really like Fred’s Rapids, Rivers and Ponds metaphor describing the Microservices architecture. Below is what I jotted down during his session.
Why Microservices now?
We are in a world where its no more about requirements, but its about ideas and the faster we can validate ideas, the better we learn, its about validated learning and micro services fits very nicely here.
Also, Confluences of technologies
- cloud computing
- languages (and supporting frameworks)
Also, Confluence of biz needs
- Accelerating business needs
- Reduction of entry barriers
The concept of application is dead where we had a single database, we need to think in terms of system. But, now we have to optimise for deployment and we need to shard that database in to tables, such that a micro-service deals with small set of data in an encapsulated way. Don’t share data in micro-services. We want to go the Netflix route, where we deploy very very frequently, where Chaos Monkey kills machines at random and yet the system is functional. You can have 100s for micro services, all running at the same time.
Micro-services are language agnostic. With services we need to design events, not entities. With events, we design history, not current (like with DBs that store the current state of the object, and we need long audit trails to keep track of data). With events, we can replay through history to build up to current state.
- Don’t denormalise too quickly like in the case of RDBMS
- Operational DBS Vs Reporting DBs. Operational DBs have been replaced by event buses (like Kafka, quarter million events per second, started as linkedIn project and then open-sourced).
- Reporting DBs are for the static view of the system.
Don’t chain micro services, instead enrich messages to be feed to the next one. Each micro services is listening to the bus, so they see both, the original message and the processed message. Always echo back the request parameters in the response, so that it is helpful for other micro-services consuming that message and also a co-relation id to be able to co-relate
Real feedback – quickly
By pulling in stream of events, one can generate data that helps understand traffic patterns, flow patterns and helps generate insight from that data. This insight is great to feedback in to the system.
Test automation in terms of acceptance tests, exploratory tests etc…in micro services almost goes away as you are move from tests to active monitoring and you are testing it in production. You invest in active monitoring tools, this is because systems are getting very complex. One of the first services to write is a monitoring service that monitors the one you plan to putting in production. So that it shouts when your production service goes down. A micro-service is like human cell, replace dead cells with new cells.
Write tests for individual cells (micro services) if your service is processing many messages etc…or as a general rule if any failure is a likelihood, then write a test. So, in general, self-monitoring replaces unit tests, business monitoring replaces ATs.
Copy-Paste is valid!
Copy-paste in micro services becomes a coding pattern. Duplication does not matter, because refactoring away in to a common library makes that library a suspect when code in micro-service breaks. Because micro-services are really small creating a new version of the micro-service when a bug is fixed is cheap.
Debugging in Micro-services
It is much easier to debug micro-services than to debug a big program. Even in an event flow, you are at most dealing with 5-7 micro services at a time.
From implementation perspective, node.js is really gorgeous to implement micro-services.
Principles of micro services
- Very small.
- Loosely coupled.
- Multiple versions of services are encouraged. For older versions, if no one is calling, then don’t take the trouble to find and kill them, and if they crash they are dead anyway.
- Publish interesting stuff anyway, regardless of whether anyone is listening or not.
Micro Services Challenges
- Synchronous Vs Async services
- Go for Async as default implementation for it and not sync version, but the jury is still out there, don’t take this as a rule.
- Deconstructing the database
- Entity Oriented DBs
- How many Dbs?
- 1 DB per microservice
- Ploy-glot (various NoSQL, SQL) + event bus
- 10% writable; fewer transactional
- Microservices or Clojure
- Services are like OO classes
- Every services has one job, if it does 2 jobs, make separate services instead.
- Choosing Architecture and Frameworks
- No Design patterns book yet.
- Taxonomy maybe useful
- Ask how they provide primary access APIs to service (Synchronicity)
- Ask about average size/ number ratio of services?
- Ask how may DBs they have?
- Operational Challenges (my addition)
- Infinite loops
- Lost messages
Rapids, Rivers and Ponds Metaphor