Microservices Monitoring and Alerting using Prometheus, Grafana and AlertManager.
There is a huge industry wide shift towards Microservices replacing monolithic application stacks to provide simplified and quick application deployments, scale up/down capabilities and accelerated development. But with all the functional and delivery advantages, it also increases the operational complexity.
Monitoring and troubleshooting microservices is more complex due to its modular and distributed nature. Lets look at Monitoring and Alerting microservices using Prometheus, Grafana and Alert Manager. These are the most widely adopted monitoring tools as of today and so have wide support.
There are 4 components in the above architecture:
- A microservice developed using using spring boot, actuator, micrometer etc to expose metrics as an endpoint like /prometheus.
- Prometheus service with configurations necessary to scrape the /prometheus endpoint and integrate with AlertManager.
- Grafana service which will be pulling data from Prometheus and use it for data visualizations.
- AlertManager service which will be purely used for alerting based on alert rules defined in Prometheus.
We are not going to go into details of these services as there are numerous blogs and articles on these services.
Download or clone the below GIT repository. It has all the necessary services shown in the architecture document for getting started.
There is a docker compose file provided in the project which will spin up 4 containers and setup the services.
mvn clean package docker build -t techmanyu-springboot-prometheus . docker-compose up -d
Four containers will spin up exposing 4 services-
Techmanyu-Service – http://localhost:8080
Prometheus Service – http://localhost:9090
AlertManager – http://localhost:9093
Grafana –http://localhost:3000 (Creds – admin/admin)
Health Checks in Prometheus:
In Grafana, go to DataSources and add prometheus endpoint.
Note: In alertmanager.yml file replace the slack webhool url and channel name to start receiving alerts. Update the alertmanager.yml file to swicth to any othe notification channel like email, webhook etc.
All set. Please comment in case of any questions or issues.