Storing Metrics with Gnocchi on CentOS

class: middle, center, inverse

# Storing metrics with Gnocchi on CentOS

by @hguemar/number80 and his numerous aliases
      
      <img src="img/centos-logo-dark.svg" width="60%" />

---
      class: middle, center
      <img src="img/gnocchi-logo.png" width="30%" />

---
      class: middle

## Whoami

.right-column[
      <img src="img/me.jpeg" width="110%" />
      ]

.left-column[
      * CentOS/Fedora developer (Cloud SIG)
      * RDO Release Wrangler @ Red Hat
      ]

---
      class: middle, center, inverse

# This is not about Pasta

---
      class: middle

## Gnocchi is

... a service to store metrics as time series originally developed within OpenStack, now independent

Gnocchi is
      * Scalable
      * Pluggable
      * Simple to use

---
      class: middle

## History

* Started in May 2014 within OpenStack Telemetry project
      * Designed to be usable standalone from the beginning!
      * No doc no merge policy => thoroughly documented
           
      ---
      class: middle
      
      ## What about X ∈ {Graphite, InfluxDB, OpenTSDB} ?

* Alternatives are often complex to deploy
      * Scalability promises scarcely met

---
      class: middle

## Did you say Features?

* REST API
      * Multi-tenancy
      * Efficient storage compression using LZ4
      * Archive policies
      * Various backends: file system, redis, ceph, write your own
      * Metric aggregation
      * Measures batching
      * Did I mention scalability? I think not!
      
      ---
      class: middle, center, inverse
      
      # Dissecting Gnocchi

---
      class: middle

## Data model

---
      class: middle

## Architecture

---
      class: middle

## Authentication

Gnocchi has 3 authentication providers:

* Basic: default using HTTP Authentication Header (no password)
      * OpenStack Keystone (can be used standalone!)
      * remoteuser: looking at HTTP server REMOTE_USER variable (same model as Basic)
      
      ---
      class: middle

## Grafana support

* Grafana has a gnocchi plugin to use it as a data source
      * Can be used in both direct and proxy mode

```bash
      $ grafana-cli plugins install gnocchixyz-gnocchi-datasource
      ```
      
      <img src="img/grafana-screenshot.png" width="40%" />
      
      ---
      class: middle

## Prometheus support

* Prometheus can use Gnocchi to store data
      * Gnocchi has to be installed with the prometheus flavor
      * Gnocchi takes care of mapping prometheus data model to its internal one
      
      Sample prometheus config file
      ```yaml
      remote_write:
      - url: "http://localhost:8041/v1/prometheus/write"
        basic_auth:
          username: "admin"
          password: "whatever"
      ```

---
      class: middle

## And also

* Nagios
      * Incinga (both through gnocchi-nagios utility)
      * Collectd
      
      ---
      class: middle, center, inverse

# Eat it!
      
      ---
      class: middle
      
      ## Serve it on a plate erm deploy it

Prerequisites: have a postgres server running somewhere
      ```bash
      $ pip install gnocchi[file,postgresql] gnocchiclient
      $ emacs /etc/gnocchi/gnocchi.conf
      $ gnocchi-upgrade
      $ gnocchi-api&
      $ gnocchi-metricd& # That's it!
      ```

---
      class: middle

## Do U Ansible ?

The Opstools SIG has developed a Gnocchi role!
      
      See https://github.com/centos-opstools/opstools-ansible/tree/master/roles/gnocchi

```yaml
      ---
      - hosts: all
        vars:
          database_host: localhost
          database_service: mariadb
          manage_packages: true
          manage_repos: true
        become: true
        roles:
          - gnocchi
          - httpd

tasks:
      - name: install gnocchiclient
        package:
          state: present
          name: python-gnocchiclient
      ```

---
      class: middle

## Coz it's not a real tech talk without mentioning Docker

You can use gnocchi-docker to run a small Gnocchi cluster

https://github.com/gnocchixyz/gnocchi-docker
      
      ---
      class: middle

## List archives policies and create a metric

```bash
      $ gnocchi archive-policy list
      $ gnocchi metric create --archive-policy-name low
      ```

---
      class: middle

## Send and retrieve measures

```bash
      $ gnocchi measures add -m 2016-05-16T12:00:00@42 -m 2016-05-16T12:01:03@45 -m 2016-05-16T12:06:07@22
      95fdc8ff-1aed-4dd3-b65b-bfb53f91081b
      $ gnocchi measures show 95fdc8ff-1aed-4dd3-b65b-bfb53f91081b
      $ gnocchi measures show --aggregation min 95fdc8ff-1aed-4dd3-b65b-bfb53f91081b
      $ gnocchi measures show --aggregation 95pct 95fdc8ff-1aed-4dd3-b65b-bfb53f91081b
      ```

---
      class: middle

## Create resources

```bash
      $ gnocchi resource-type create --attribute name:string --attribute host:string server
      $ gnocchi resource create --attribute name:www-42 --attribute host:compute1 --create-metric
      cpu:medium --create-metric memory:low --type server `uuidgen`
      ```

---
      class: middle

## Update resources

```bash
      $ gnocchi resource update --attribute host:compute2 --type server
      e4c2eab7-52ed-4447-bbcb-48cb04f12015
      ```

---
      class: middle

## Review previous changes in history

```bash
      $ gnocchi resource history --format json --details e4c2eab7-52ed-4447-bbcb-48cb04f12015
      ```
      ---
      class: middle

## Search resources

```bash
      $ gnocchi resource search --type server host=compute2
      ```
      
      ---
      class: middle, center, inverse

# Performances (coz we all like numbers and fancy charts)

---
      class: middle

## Write throughput

Hardware: 24 core Xeon, 20 OSD, 256 GB RAM
      
      Capacity: incoming: ~1.35 million measures/s
      
      <img src="img/write-throughput-gnocchi.png" width="70%" />

.footnotes[Full report: https://medium.com/@gord.chung/gnocchi-4-introspective-a83055e99776]

---
      class: middle

## Aggregration performances

Hardware: 24 core Xeon, 20 OSD, 256 GB RAM
      
      Test case: Send 10K metrics with 720 measures each every 10 minutes

.footnotes[Full report: https://medium.com/@gord.chung/gnocchi-4-introspective-a83055e99776]

---
      class: middle

## Mandatory lego slide

.footnotes[Credits to the Italian Vogue and photographer Fabrice Fouillet and stylist Enrica Ponzellini.]
      
      ---
      class: middle, center, inverse

# Dolci (Conclusions)

---
      class: middle

## Where to get help?

* Awesome doc is at https://gnocchi.xyz
      * Nice people are on #gnocchi @ freenode

---
      class: middle

## Thanks

* The Gnocchi team
      * Italian cuisine for the tasty names