Labels

Aws (32) MySQL (20) Aurora (12) Rds (7) DynamoDB (6) Cloudformation (4) Python3 (4) Unix (3) MongoDB (1)

Friday, 17 September 2021

Telegraf basic configuration for an Operating System and MySQL

 This tutorial is for configuring the basic metrics and settings for Telegraf on Ubuntu 12.04.

What?! Ubuntu Precise? That is an almost 10 years old release. Yes indeed but sometimes you still find some old ancestor Linux versions running on legacy systems that it is not a simple process to upgrade due to the missing development of the applications, lack of knowledge about the drivers. 

Despite the oldness of your OS, it still needs to be monitored and you need to gather metrics. 

Back in the time, Telegraf was one of the most used daemons to collect this information.  

$telegraf --version Telegraf 1.9.4 (git: HEAD 4da8d0a4)


Basic configurations

Agent settings. You can initiate the configuration file and leave the default setting for the local agent. The --sample-config tells Telegraf to output a config file. 

sudo telegraf --sample-config --output-filter influxdb > /etc/telegraf/telegraf.conf

cat /etc/telegraf/telegraf.conf


[agent] interval = "20s" round_interval = true metric_buffer_limit = 1000 flush_buffer_when_full = true collection_jitter = "0s" flush_interval = "10s" flush_jitter = "0s" debug = false quiet = false hostname = "" omit_hostname = false

The config file will assume that the InfluxDB cluster runs locally and you need to edit it if is not the case. 

Output InfluxDB. InfluxDB receives the metrics. Telegraf output plugin is configured to send the metrics over the network to the database
 

cat /etc/telegraf/telegraf.conf

....

# Configuration for influxdb server to send metrics to

[[outputs.influxdb]]

  ## The full HTTP or UDP endpoint URL for your InfluxDB instance.

  ## Multiple urls can be specified as part of the same cluster,

  ## this means that only ONE of the urls will be written to each interval.

  # urls = ["udp://localhost:8089"] # UDP endpoint example

  urls = ["http://influxdb.metrics.com:8086"] # required

  ## The target database for metrics (telegraf will create it if not exists).

  database = "mysql.applications" # required

  ## Retention policy to write to.

  ##  retention_policy = "default"

  ## Precision of writes, valid values are "ns", "us" (or "µs"), "ms", "s", "m", "h".

  ## note: using "s" precision greatly improves InfluxDB compression.

  precision = "s"


  username = "xxxxxxx"

  password = "xxxxxxx"



  ## Write timeout (for the InfluxDB client), formatted as a string.

  ## If not provided, will default to 5s. 0s means no timeout (not recommended).

  timeout = "5s"


  ## Optional SSL Config

  # ssl_ca = "/etc/telegraf/ca.pem"

  # ssl_cert = "/etc/telegraf/cert.pem"

  # ssl_key = "/etc/telegraf/key.pem"

  ## Use SSL but skip chain & host verification

  # insecure_skip_verify = false


Output plugins. Telegraf supports different input plugins, a plugin defines the type of output that you want to gather.  

  • https://github.com/influxdata/telegraf/blob/release-1.19/plugins/inputs/cpu/README.md
  • https://github.com/influxdata/telegraf/blob/release-1.19/plugins/inputs/disk/README.md
  • https://github.com/influxdata/telegraf/blob/release-1.19/plugins/inputs/diskio/README.md
  • https://github.com/influxdata/telegraf/blob/release-1.19/plugins/inputs/mem/README.md
  • https://github.com/influxdata/telegraf/blob/release-1.19/plugins/inputs/system/README.md

$ cat /etc/telegraf/telegraf.d/os_mysql.conf

[[inputs.cpu]]

  percpu = true

  totalcpu = true

  collect_cpu_time = false


[[inputs.mem]]


[[inputs.disk]]


  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]


[[inputs.diskio]]


[[inputs.system]]


[[inputs.procstat]]

    exe = "mysqld"


[[inputs.mysql]]

    servers = ["mysql_check:mysql_check@tcp(127.0.0.1:3306)/?tls=false"]

    table_schema_databases = []

    gather_slave_status = true


The basic plugin configurations are OK for most of the cases, you can edit and make it more specific.

The daemon has to be rebooted every time the configuration file changes. 

$service telegraf restart telegraf process was stopped [ OK ] Starting the process telegraf [ OK ] telegraf process was started [ OK ]


Global tags. InfluxDB supports tagging the metric. In this way, it is easier to group and aggregate the metrics when you want to retrieve and visualize them. You defines both the key and the value.

$ cat /etc/telegraf/telegraf.conf

....

[global_tags]

   dc = "sart"

   project = "iprod"

   role = "database"

   environment = "test"



Mysql input. Base on my experience I want to share some further details regarding this plugin

 [[inputs.mysql]] # specify servers via a url matching: # [username[:password]@][protocol[(address)]]/[?tls=[true|false|skip-verify]] # see https://github.com/go-sql-driver/mysql#dsn-data-source-name # e.g. # root:passwd@tcp(127.0.0.1:3306)/?tls=false # root@tcp(127.0.0.1:3306)/?tls=false # # If no servers are specified, then localhost is used as the host. ervers = ["tcp(127.0.0.1:3306)/"] ervers = ["mysql_check:mysql_check@tcp(127.0.0.1:3306)/?tls=false"] # the limits for metrics form perf_events_statements perf_events_statements_digest_text_limit = 120 perf_events_statements_limit = 250 perf_events_statements_time_limit = 86400 # ## if the list is empty, then metrics are gathered from all database tables table_schema_databases = [] # ## gather metrics from INFORMATION_SCHEMA.TABLES for databases provided above list gather_table_schema = false # ## gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST gather_process_list = true # ## gather auto_increment columns and max values from information schema gather_info_schema_auto_inc = true # ## gather metrics from SHOW SLAVE STATUS command output gather_slave_status = true # ## gather metrics from SHOW BINARY LOGS command output gather_binary_logs = false # ## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE gather_table_io_waits = true # ## gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS gather_table_lock_waits = true # ## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE gather_index_io_waits = true # ## gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS gather_event_waits = true # ## gather metrics from PERFORMANCE_SCHEMA.FILE_SUMMARY_BY_EVENT_NAME gather_file_events_stats = false # ## gather metrics from PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST gather_perf_events_statements = false # ## Some queries we may want to run less often (such as SHOW GLOBAL VARIABLES) interval_slow = "30m"




Thank you for reading it. I hope you can still find joy by using Telegraf and InfluxDB.