Labels

Aws (32) MySQL (20) Aurora (12) Rds (7) DynamoDB (6) Cloudformation (4) Python3 (4) Unix (3) MongoDB (1)

Friday, 21 January 2022

Monitoring a Mamcached server

 As a database administrator, you time to time want to monitor or to do any maintenance on a Memcached server. As long as the cache works, we consider the server in a good state and is there nothing that needs to be done. 

However, it is always a good practice to review the server used to be able to understand if the cache is suffering from a lack of resources or if the cache is oversized based on the workload.

There aren't any official Memcached tools that pull out stats continuously. There are a few choices between using third-party clients or developing a wrapped script using telnet.  

What you want to monitor

The essential metrics to look for the state of the cache are:

  • Current connections
  • rejected_connections
  • get_hits
  • get_misses
  • curr_items
This information can be useful to get the correct level of cache and memory usage.

Stats interface through telnet protocol

The quickest way to obtain the statistics is through the telnet protocol where you can use the stats interface.

$ telnet cache-seb.exx.0001.euw1 11211

Trying 10.112.49.208...

Connected to cache-seb.exx.0001.euw1.

Escape character is '^]'.

stats

STAT pid 1

STAT uptime 74658714

STAT time 1642680707

STAT version 1.5.16

...


The full list of statistics are available in the official documentation here 


Third-party clients

In the Debian distribution, the netcat command can be used to send commands to the Memcached cache.

$ echo stats | nc cache-seb.exx.0001.euw1 11211 STAT pid 1 STAT uptime 83584 STAT time 1642775824 STAT version 1.5.16 STAT libevent 2.0.21-stable STAT pointer_size 64


The libmemcached-tools provides the memcstat command, which displays the statistics

$ memcstat --servers=cache-seb.exx.0001.euw1 11211 STAT pid 1 STAT uptime 83584 STAT time 1642775824 STAT version 1.5.16 STAT libevent 2.0.21-stable STAT pointer_size 64


A tool that hasn't been supported for years but still does a very good job is mcstat because it prints every 5 secs the aggregate stats in real-time and returns only the stats that really matter.

$ memcst cache-seb.exx.0001.euw1

Memcache version 1.5.16

Max Memory Size:  6.0G  Pointer size: 64 bit


    time  cnct    used    gets    hits    miss    sets    evic   miss%   evic%      in     out

14:41:56    23    2.7M    7.8M    7.8M    2.9k  435.8k       0    0.04    0.00  901.7M    2.3G


14:42:01    23    2.6M    1.1k    1.1k       0      45       0    0.00    0.00  119.6k  328.5k

14:42:06    23    2.6M     618     618       0      25       0    0.00    0.00   69.2k  193.0k

14:42:11    23    2.6M     678     678       0      30       0    0.00    0.00   76.9k  210.7k


Session Statistics:

             Statistic             Total           Session

                uptime       23h 17m 56s               15s

              libevent     2.0.21-stable                 0

           rusage_user          2m 36.1s             0.06s




AWS Elasticache monitoring tool

AWS through the Cloudwatch service, easily provides some of the most relevant metrics which enable you to make sure the cache cluster is healthy and running.
The most important metrics are:
  • CPUUtilization
  • SwapUsage
  • Evictions
  • CurrConnections
Which metrics should I monitor? here 


Conclusion

As you can see there are tools that are no longer supported but still, they do enough to monitor the cache. However, in my opinion, Memcached doesn't provide enough interface and commands to collect more internal metrics and you still need to work out on your own is the server is suffering or is doing OK.

I'm leaving you with other resources if you want to know more.