Tags: ansible data-visualization data-management back-end

Martin Berger

Installation Guide for OCI Monitoring

1 Introduction

An Installation Guide for OCI Monitoring

Note: This is an experimental environment. Feel free to try it out, extend it, and have fun with it!

In this walkthrough you’ll install a basic OCI monitoring solution with these components based on Ansible in Oracle Linux 8. The setup is tested for:

  • OL8 running in ESXi
  • OL8 running in local VMware Workstation with NAT
  • OL8 running in Oracle Cloud Infrastructure

Installed components by Ansible roles:

  • Docker
  • Steampipe
  • Grafana
  • Prometheus
  • Push Gateway
  • PostgreSQL
Begin »

3 How it works

Architecture Overview
Architecture
  1. Execute the Python script
  2. Steampipe gathers the information from Oracle Cloud Infrastructure
  3. The return value is pushed to Prometheus Push Gateway
  4. Prometheus scrapes the metrics from the gateway
  5. Grafana reads the metrics from Prometheus
« Back Continue »

4 Prerequisites

  • root access by password
  • /etc/hosts configured
  • Ansible and Git configured
  • Internet access
  • Oracle Cloud Infrastructure user with inspect permissions, including SSH PEM key and configuration

Software Installation OL8 ESXi / OL8 VMware

As user root:

$ yum -y install yum-utils
$ yum -y install oracle-epel-release-el8
$ yum-config-manager --enable ol8_developer_EPEL
$ yum -y install ansible git

Software Installation OL8 Oracle Cloud Infrastructure

As user opc:

$ sudo dnf upgrade
$ sudo dnf -y install oracle-epel-release-el8
$ sudo dnf config-manager --enable ol8_developer_EPEL
$ sudo dnf -y install ansible git

Ansible SSH Configuration for Oracle Cloud Infrastructure

  • Upload the opc’s SSH private key to /home/opc/.ssh temporarily for installaton purposes

  • Change the Ansible checked out hosts file to:

      [all:vars]
  ansible_ssh_private_key_file=/home/opc/.ssh/<your_ssh_key_file_name_here>

  [monitoring]
  <your_oci_compute_private_instance_here> ansible_user=opc ansible_python_interpreter="/usr/bin/env python3"

After the installation, it’s a good practice to remove the opc private key from your compute instance

« Back Continue »

5 Steps

  1. Login to Oracle Linux 8 as root
  2. Clone the repository to a local folder such as /root/git
  3. Change to subdirectory oci-monitoring
  4. Update the Ansible hosts file with your IP and root password. ansible_ssh_pass is required for local connections
  5. Run ansible-galaxy collection install -r roles/requirements.yml
  6. Run ansible-playbook install.yml

As root, verify that all Docker containers are running:

$ docker ps
CONTAINER ID   IMAGE              COMMAND                  CREATED             STATUS             PORTS                    NAMES
f7f2e137f4a1   prom/pushgateway   "/bin/pushgateway"       About an hour ago   Up About an hour   0.0.0.0:9091->9091/tcp   pushgateway
c6ecc72065c9   prom/prometheus    "/bin/prometheus --c…"   About an hour ago   Up About an hour   0.0.0.0:9090->9090/tcp   prometheus
3485de8cc1f9   grafana/grafana    "/run.sh"                About an hour ago   Up About an hour   0.0.0.0:3000->3000/tcp   grafana
8e821aa0044b   turbot/steampipe   "docker-entrypoint.s…"   About an hour ago   Up 30 minutes      0.0.0.0:9193->9193/tcp   steampipe

Network Security

The Ansible playbooks also open these ports in the VM for troubleshooting access:

  • 3000 - Grafana
  • 9090 - Prometheus
  • 9091 - Prometheus Push Gateway
  • 9093 - Steampipe Service
« Back Continue »

6 OCI Configuration

  • After the successful Ansible execution, put your personal OCI configuration and SSH key into the directory /home/steampipe/.oci
  • Replace the dummy values
  • Update the file /home/steampipe/config/oci.spc with the correct SSH key file name

Take care that owner and group of the OCI configuration file is steampipe

Example:

$ pwd
/home/steampipe/.oci

$ ls -l
total 8
-rw-r--r--. 1 steampipe steampipe  307 Aug  9 09:01 config
-rw-r--r--. 1 steampipe steampipe 1730 Aug  9 09:01 jurasuedfuss-20210809.pem

Restart the Docker container for Steampipe:

$ docker stop steampipe
$ docker start steampipe
« Back Continue »

7 How to create the user for OCI access - based on OCI CLI

Next we create an OCI user for monitoring. An existing OCI CLI setup for an tenant administrator is required to execute these steps. The required SSH key in PEM format can be downloaded from the OCI web interface. The user, group, and policy can be created in web interface as well.

All we need for Steampipe is the OCI config file for the new user and their SSH key in PEM format.

Create User

$ oci iam user create --name oci_user_readonly --description "OCI User with inspect all-resources."

Create Group

$ oci iam group create --name oci_group_readonly --description "OCI Group with inspect all-resources."

Add User to Group

$ oci iam group add-user \
--user-id <your user OCID from created user above> \
--group-id <your group OCID from created group above>

Create Policy

$ oci iam policy create \
--compartment-id <your tenancy OCID> \
--name oci_policy_readonly \
--description "OCI Policy with inspect all-resources." \
--statements '[ "allow group oci_group_readonly to inspect all-resources on tenancy" ]'

Add API Key

  1. Add your API key

    OCI API Key 01

  2. Download the created private key in PEM format

    OCI API Key 02

  3. Copy the configuration file preview. The values are used for the Steampipe OCI configuration

    OCI API Key 03
« Back Continue »

8 Steampipe

OCI Regions

To filter your regions, just edit the file /home/steampipe/config/oci.spc.

For example:

connection "oci_tenant_kestenholz" {
  plugin                = "oci"
  config_file_profile   = "DEFAULT"          # Name of the profile
  config_path           = "~/.oci/config"    # Path to config file
  regions               = ["eu-frankfurt-1" , "eu-zurich-1"] # List of regions
}

Here are some commands to verify if Steampipe is working as expected. Execute as root:

$ docker exec -it steampipe steampipe plugin list
+--------------------------------------------+---------+-----------------------+
| Name                                       | Version | Connections           |
+--------------------------------------------+---------+-----------------------+
| hub.steampipe.io/plugins/turbot/oci@latest | 0.1.0   | oci_tenant_kestenholz |
+--------------------------------------------+---------+-----------------------+

$ docker exec -it steampipe steampipe \
query "select display_name,shape,region from oci_core_instance where lifecycle_state='RUNNING';"
+-----------------------------------+------------------------+----------------+
| display_name                      | shape                  | region         |
+-----------------------------------+------------------------+----------------+
| Instance-DB-1                     | VM.Standard1.2         | eu-frankfurt-1 |
| Instance-AS-1                     | VM.Standard1.1         | eu-frankfurt-1 |
+-----------------------------------+------------------------+----------------+

$ docker exec -it steampipe steampipe \
query "select key,title,status from oci_region where is_home_region=true;"
+-----+----------------+--------+
| key | title          | status |
+-----+----------------+--------+
| FRA | eu-frankfurt-1 | READY  |
+-----+----------------+--------+
« Back Continue »

9 Python Example Scripts

In the subdirectory /home/steampipe/py, there are two basic examples of how to get the data from the Steampipe PostgreSQL service to Python3. Feel free to adapt the queries and files. Returned values are pushed to the Prometheus Gateway on port 9091 for further usage.

Script Purpose
pgsql-query-bv-zurich.py Summary of Block Volume in OCI Region Zurich
pgsql-query-ci-running-zurich.py Summary of running Instances in OCI Region Zurich

Note: You’ll need to restart the Docker container before executing Python3 according this error. This is something I’m working on!

Manual execution and upload of the query result:

$ python3 pgsql-query-ci-running-zurich.py
$ python3 pgsql-query-bv-zurich.py

Something went wrong: no connection config loaded for connection 'oci'

Restarting Steampipe as root:

$ docker stop steampipe
$ docker start steampipe
« Back Continue »

10 Prometheus Push Gateway

According to the Python script, new data is loaded in Prometheus Push Gateway to port 9091 and scraped by Prometheus port 9090.

Checkout this example for the Protheus Gateway where data is loaded by jobs oci_blockvolume_ / _oci_compute.

OCI Prometheus Push Gateway 01
« Back Continue »

11 Grafana

Grafana is reachable by address your-machine-ip:3000.

  • Username: admin
  • Password: welcome1

The Prometheus data source and a basic dashboard are deployed during the Grafana Docker setup process. Here’s an example for dasboard OCI Demo - eu-zurich-1:

Prometheus data source
Prometheus data source
Sample dashboard OCI Demo
Sample dashboard OCI Demo

Here you can see the pushed metric from the Python script by name:

Metric from Python script
« Back Continue »

12 Troubleshooting

Docker Logs

To verify that Steampipe is running properly:

$ docker logs steampipe

Steampipe Access Logs

The foreign data wrapper logs are stored locally — not in the Docker container — in the directory /home/steampipe/logs:

drwx------. 11 steampipe steampipe     173 Aug  9 17:18 ..
-rw-------.  1      9193 root       756701 Aug  9 19:57 database-2021-08-09.log
drwxrwxr-x.  2 steampipe root           68 Aug 10 02:00 .
-rw-------.  1      9193 root      3411203 Aug 10 07:19 database-2021-08-10.log
« Back