Disable Filesystem space checks

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
computerfr33k
Posts: 1
Joined: Thu Mar 16, 2017 8:40 pm

Disable Filesystem space checks

Postby computerfr33k » Thu Mar 16, 2017 8:45 pm

Hi, I was wondering if there is a way to disable zimbra space checks for certain mount points, because I am using Ubuntu 16.04 with kernel livepatch service which is using snapd and mounts the livepatch service into a mountpoint which is then shown as being 100% used space causing Zimbra to then constantly send warning emails about filesystem being 100% used.

Thanks


kilton_jmiles
Posts: 2
Joined: Thu May 10, 2018 8:20 pm

Re: Disable Filesystem space checks

Postby kilton_jmiles » Tue Jun 25, 2019 5:43 pm

For reference to others who encounter this issue:
https://bugzilla.zimbra.com/show_bug.cgi?id=109125

This was my scenario::

Where this happens: EC2 instances deployed using the AWS Ubuntu 16.04,18.04 AMIs

Why this is a problem: 4320 critical disk space alerts will be send per day which may exceed any smarthost mail relay limits.

How the problem appeared to happen:
The AWS Ubuntu images include the amazon-ssm-agent to communicate upwards into EC2 and Systems Manager for automation, reporting, management.
This is facilitated using snapd.
In a Ubuntu EC2 instance three loop devices are created:
/dev/loop0
/dev/loop1
/dev/loop2

'df' reports these mounted devices as having filesystems who are 100% full

The default zmstats alerting interval for critical disk space issues is once every minute.
This results in 3 alerts per minute (one for each of those loop devices) and 4320 critical disk space alerts per day.

These begin immediately after completing the installation of ZCS.

One must manually exclude the /dev/loop# devices after installing ZCS and then restart zmstats



This is how I checked and then stopped the issue from happening:
Run as the Zimbra user:

Check your existing df excludes:
zmlocalconfig | grep zmstat_df_excludes


If you have none excluded already, run this, or modify it with the paths of the devices you already had:

zmlocalconfig -e zmstat_df_excludes="/dev/loop0:/dev/loop1:/dev/loop2:/dev/loop3:/dev/loop4:/dev/loop5:/dev/loop6:/dev/loop7:/dev/loop8:/dev/loop9"
zmstatctl restart
myzamri
Posts: 20
Joined: Tue Mar 28, 2017 2:46 am

Re: Disable Filesystem space checks

Postby myzamri » Thu Jul 25, 2019 1:45 am

It worked for me on Ubuntu 18.04 with zimbra 8.8.9 patch 4.

I want to exclude monitoring of loop devices, thus:

Code: Select all

zmlocalconfig -e zmstat_df_excludes="/dev/loop0:/dev/loop1"


Restart after making changes:

Code: Select all

zmstatctl restart
kilton_jmiles
Posts: 2
Joined: Thu May 10, 2018 8:20 pm

Re: Disable Filesystem space checks

Postby kilton_jmiles » Thu Jul 25, 2019 4:20 pm

myzamri wrote:It worked for me on Ubuntu 18.04 with zimbra 8.8.9 patch 4.

I want to exclude monitoring of loop devices, thus:

Code: Select all

zmlocalconfig -e zmstat_df_excludes="/dev/loop0:/dev/loop1"


Restart after making changes:

Code: Select all

zmstatctl restart


Indeed.
The local configuration to exclude those devices appears to work without issue.

The issue I wanted to raise and present and experience was regarding /dev/loop# devices and Zimbra's default set of monitored devices/filesystems.

For syadmins with Zimbra installed in AWS it would have been worth a note in the installation guide about this being a known issue.
(I will go back and double check the release notes and installation guides to see if I missed it)
It would be a good idea for Zimbra support to include loop devices by default in the exclusion list.
The rationale = AWS Is a large platform for hosting Ubuntu EC2 instances and their Ubuntu AMIs (and if you install ssm agent) are using snapd,

Summary:
Zimbra hosted within AWS on Ubuntu
- the standard Ubuntu AMI uses snapd to install it's Systems Manager agent
- this creates a /dev/loop# device for each snap package that is installed
- the aws ssm agent pacakage will result in at least on /dev/loop# device existing
- Zimbra's free disk space monitor will see these /dev/loop# devices as 100% full and email an alert on each of them every minute by default
- following the installation of Zimbra in this platform you need to manually add an exclusion of the existing /dev/loop# devices
- potentially also need to ensure that if you install a package using snapd you need to update the exclusion list

Return to “Administrators”

Who is online

Users browsing this forum: Google [Bot], MSN [Bot] and 5 guests