Monitoring

From BitFolk
Jump to navigation Jump to search

BitFolk provides a free monitoring service which can be enabled on request, and is required for some of BitFolk's other services.

Disclaimer

Warning Warning: Please note that no guarantees are made of the accuracy of this free service; if you have anything critical you may wish to monitor it yourself!

Web interface

The monitoring web interface is at https://mon.bitfolk.com/. You log in to it using your usual BitFolk credentials, and it should show every configured check against all VPSes that you have with BitFolk.

If for whatever reason you do not wish to use your normal BitFolk credentials for this, it is possible for BitFolk to set up some different credentials for you to use. Please contact Support about this.

Setup

Monitoring checks are free but are not added by default. This might change in the future but for now you have to ask Support for them to be added.

Usually just an IPv4 ping check will be added, which will suffice for checking that your VPS is up. Almost any service that you run can be monitored though, and common requests include:

  • IPv6 ping
  • SSH
  • HTTP / HTTPS (including TLS certificate validity)
  • SMTP
  • MySQL

These sorts of checks can work without an agent (i.e. without anything installed on your VPS). More complicated checks such as disk space, load or anything else that you can check with a script will need some sort of agent such as an NRPE daemon or SNMP daemon.

NRPE

NRPE is a typical agent you would run that would allow BitFolk's monitoring system to execute health checks on your VPS. On Debian/Ubuntu systems it can be installed from the package nagios-nrpe-server. This will normally pull in the package monitoring-plugins-basic which contains the check plugins.

Check plugins end up in the /usr/lib/nagios/plugins/ directory. NRPE can run any of these when asked and feed the info back to BitFolk's Icinga. All of the existing ones should support a --help argument to let you know how to use them, e.g.

$ /usr/lib/nagios/plugins/check_tcp --help

You can run check plugins from the command line:

$ /usr/lib/nagios/plugins/check_tcp -H 85.119.82.70 -p 443
TCP OK - 0.000 second response time on 85.119.82.70 port 443|time=0.000322s;;;0.000000;10.000000

There are a large number of Nagios-compatible check plugins in existence so you should be able to find one that does what you. If there's not, it's easy to write one. Here's an example of using check_disk to check the disk space of your root filesystem.

$ /usr/lib/nagios/plugins/check_disk -w '10%' -c '4%' -p /
DISK OK - free space: / 631 MB (11% inode=66%);| /=5035MB;5381;5739;0;5979

Once you have that working, you put it in an NRPE config file such as /etc/nagios/nrpe.d/xvda1.cfg.

command[check_xvda1]=/usr/lib/nagios/plugins/check_disk -w '10%' -c '4%' -p /

You should then tell BitFolk (in a support ticket) what the name of it is ("check_xvda1"). It will then get added to BitFolk's Icinga.

By this means you can check anything you can script.

Alerts

Your first interaction with BitFolk's monitoring will probably be when you receive an email alert. There are two kinds of alerts: Host and Service. Host alerts happen when the host check fails; this is usually an IPv4 and IPv6 ping against your VPS. The other checks are for individual services on your VPS.

Host alerts will repeat every hour unless they recover on their own. Service alerts will repeat every 4 hours unless they recover on their own.

Stopping the flow of alerts

If you intend to fix the root cause of the alert, but not just now, you should go to the web interface and acknowledge the problem. This will prevent more alerts being sent until the state of the host or service changes. There is a handy link in the alert email itself that sends you to the right place.

If the check is no longer relevant to you, or if its thresholds need tweaking, please contact Support to ask for it to be removed or adjusted.

Controlling where alerts go to

In the Contacts section of BitFolk's web Panel you can add additional contact records, and then assign them to the "Alerting" role. When you do this, these will be the only email addresses that receive alerts from BitFolk's monitoring. If you do not make use of the "Alerting" role then the email address in your main customer record will be used. You can add multiple contacts to the "Alerting" role and they will all receive the alerts. The monitoring configuration will update within 5 minutes of you making a change here.

Pre-emptively disabling alerts and/or checks

If you know you're going to be doing some work and don't want to receive alerts for it, you can mark hosts and/or services as being in "downtime". The periods of downtime can be set to specific times, or just "until the state changes".

Finally, you can indefinitely turn off notifications and/or checks against hosts and services. When viewing the host or service, scroll to the bottom and find the section Feature Commands. Here you can uncheck "Active Checks", "Passive Checks" and "Notifications".

Monitoring required for BitFolk services

Certain services that BitFolk provides require monitoring to be set up because they rely on something on your VPS, and BitFolk wants to know that is working when diagnosing any problem with the service. So far this includes:

This service works by SSH, so an SSH check will be added. Also the disk space used by your backups and the age of the last successful backup will be monitored. See the dedicated article on the backups service for more information.
This services requires that your primary DNS server is correctly serving your DNS zone(s). Convenience monitoring of each of your zones on each of BitFolk's authoritative DNS servers is also added. See the dedicated article on the secondary DNS service for more information.

In addition, if you opt in to suspend and restore then at least a basic ping check will be added so that BitFolk has some confidence that your VPS has been successfully restored.

Frequently Anticipated Questions

http

Why do I have a http-v4- and http-v6- check for every web site?

This is to provide separate checks for HTTP over IPv4 and IPv6 when your VPS has both address families configured. Otherwise your web site being reachable over either protocol would provide a success and you might miss breaking it on one of the protocols.

Why do I have some http checks that are just against an IP address?

In the old system some customers had a simple HTTP check which would have been using the main IPv4 address as a vhost. Therefore the new checks are doing exactly the same thing. It would be better to specify a vhost and a URL path if applicable. Please contact Support to do so.

Does the https check have to verify the certificate?

By default BitFolk's https checks do verify that your certificate is not expired, but it is possible for BitFolk to amend the checks to not care about this if for some reason you do not intend to renew it.

Can you add http checks on IPv6 as well?

The short answer is: Yes. Contact Support to ask for it. However, it should just be working…

If BitFolk's monitoring system knows that your VPS has an IPv6 address then it will automatically generate http checks on both IPv4 and IPv6. If this isn't happening that generally means:

  • You haven't asked for a ping6 check yet, so the monitoring doesn't know that your VPS has a working IPv6 address.
  • BitFolk noticed that IPv6 http checks were failing while IPv4 ones weren't, so assumed that was intentional and disabled IPv6 http checks for you.
  • There is a config error on BitFolk's side.

Why does my https check show a different certificate name to the vhost name?

The most likely reason is that you have one TLS certificate with multiple SubjectAltNames. This is what happens for example when you specify multiple names on one Let's Encrypt certificate. Only the first (CN) will be shown by the check.

If you instead have multiple certificates being served on one IP address using SNI then this should be working because BitFolk's https checks do use SNI by default. If it's showing the wrong name in this case, please contact Support to investigate.

Can my https check fire when the certificate is within something other than 28 days of expiry?

Yes. 28 days is only the default. It can be customised per-host or per-check if necessary. Please ask support.

Other

I don't want to use my main BitFolk credentials for this. Is there any other way?

Yes. A separate password database can be consulted just for this service.

There isn't a web interface for this yet, but if you'd like to email a password hash to Support then this can be added (you can encrypt the email with PGP if you like). Here's how to generate the password hash:

$ php -r 'echo password_hash("your_top_secret_password", PASSWORD_DEFAULT) . "\n";'
$2y$10$h3tvhPGLw0QBRp/X0pHBaO/982Br0Uvc2hwUgM1wiQNmjHEQGLGtS

Or:

$ htpasswd -nBC 10 foo
New password:
Re-type new password:
foo:$2y$10$NQ/L.XXXFi8sM6DzAi6MQOhvAVcaHboDuJovG0dBOGuI3AczzeeMi

(It is normal that these two hashes of the same password do not match)

Limitations of this approach
  • Your username has to remain the same.
  • The BitFolk LDAP directory is consulted first so if you accidentally provide the normal BitFolk password then it will still work.
  • For now you can't change the password without contacting Support.

Which IP addresses will BitFolk's monitoring checks come from?

Checks may come from any of:

  • 85.119.80.238
  • 85.119.80.244
  • 2001:ba8:1f1:f25d::/64

Why don't I have different host objects for IPv4 and IPv6 like I did with Nagios?

It's much easier with the new system (Icinga) to have both IPv4 and IPv6 on the same host object.

Why don't I have a ping6 check?

Your monitoring setup has been copied over from the old system. Many customers only had ping4 checks even when they had working IPv6 (all BitFolk customers have IPv6, but not all of them use it!). If you'd like a ping6 check added, please contact Support.

Can you alert me by some other means than email?

Possibly. Please contact Support to discuss your needs. One customer does have Pushover notifications set up.

Also bear in mind that anything which can turn emails into another kind of alert can be used by adding them as a contact in the Alerting role. There are several customers who have alerts sent to an email-to-SMS gateway service that they have provided themselves.

Do I have to receive notifications every 4 hours?

Initially that is what is configured but it can be changed. If you'd like a different notification interval then please contact Support. It can also be set to only notify once.

Note that if you've seen a notification and don't want to be bothered by it again you can also log in to the web interface and Acknowledge it. You then won't see it again until the state changes.