Using CheatSheets To Apply Best Practices

Cheatsheet Prometheus

Cheatsheet Prometheus

1.1 Prometheus Commands

Name Command
Run prometheus server with docker docker run -p 9090:9090 prom/prometheus, http://localhost:9090/graph, http://localhost:9090/metrics
Run cadvisor to get local containers’ metrics docker run -v /var/run:/var/run -v /sys:/sys -p 8080:8080 google/cadvisor, http://localhost:8080/metrics
Query metrics by api, instead of web console curl http://localhost:9090/api/v1/query?query=container_memory_usage_bytes
List all alerts of alertmanager curl http://localhost:9093/api/v1/alerts
Prometheus tech stack footprint prometheus(350MB RAM), node-exporter(10MB), kube-state-metrics(20MB), alertmanager(15MB), grafana(30MB)
Example of client libraries Link: prometheus-python-example.py
Prometheus Online Demo Live demo from CloudAlchemy
Prometheus Config file /etc/prometheus/prometheus.yml Sections in conf: global, rule_files, scrape_configs

1.2 Prometheus Components

Name Command
Prometheus server Scrapes and store time series data. It uses mainly pull model, instead of push.
Special-purpose exporters Get metrics for all kinds of services. e.g, Node Exporter, Blackbox Exporter, SNMP Exporter, JMX Exporter, etc
Client libraries Instrument application code.
Alertmanager Handle alerts.
Push gateway Support short-lived jobs. Persist the most recent push of metrics from batch jobs.
Reference Link: Exporters And Integrations, Link: Default port allocations

prometheus-overview.png

1.3 Prometheus Metric Types

Name Command
Counter It only goes up (and resets), counts something. e.g, the number of requests served, tasks completed, or errors.
Gauge It goes up and down, snapshot of state. e.g, temperatures or current memory usage, etc
Summary It samples observations, espeically over a sliding time window. e.g, rate(http_request_duration_seconds_sum[5m])
Histogram It samples observations and counts them in configurable buckets.

1.4 Prometheus Concepts

Name Summary
Target A target is the definition of an object to scrape.
Job A collection of targets with the same purpose.
Instance A label that uniquely identifies a target in a job.
Exporter Expose metrics from a non-Prometheus format into a format Prometheus supports.
Collector A part of an exporter that represents a set of metrics.
Handler  
Rule  

1.6 Prometheus PromQL Query

Name Command
Reference Link: query
Find metric by name+job+group somemetric{job=”prometheus”,group=”canary”}
  rate(apiserverrequestcount{verb=”GET”, code=”200″}[1m])
The avg network traffic received per second, over the last min rate(nodenetworkreceivebytestotal[1m])
topk query Link: query-topk.txt
join  
cut  
slice  
count  
predict  
sum  
min  
max  
avg  

1.7 Prometheus Alerts

Name Command
How full will the disks be in 4 hours?  
Which services are the top 5 users of CPU?  
What’s the 95th percentile latency in EU datacenter?  


Leave a Reply

Your email address will not be published. Required fields are marked *