Archive storage: Difference between revisions

From BitFolk
Jump to navigation Jump to search
m (→‎No performance guarantees: IOPS description)
(→‎Frequently asked questions: Why you can't use this for BitFolk backups)
Line 30: Line 30:


There's no reason why any of those won't work inside a BitFolk VPS. BitFolk would be very interested in any reports of attempts to do this; they would make for an excellent wiki article.
There's no reason why any of those won't work inside a BitFolk VPS. BitFolk would be very interested in any reports of attempts to do this; they would make for an excellent wiki article.
==Can I use archive storage for my [[Backups|BitFolk backups]]?==
At the moment not. [[Backups|BitFolk's backups]] currently use [http://rsnapshot.org rsnapshot], which works like this:
# <tt>rm -r hourly.5</tt>
# <tt>mv hourly.4 hourly.5</tt>
# <tt>mv hourly.3 hourly.4</tt>
# <tt>mv hourly.2 hourly.3</tt>
# <tt>mv hourly.1 hourly.2</tt>
# <tt>cp -al hourly.0 hourly.1</tt>
# <tt>rsync […args…] root@your-vps:/path hourly.0/your-vps/</tt>
That happens at every backup run, so potentially 6 times a day. Also
once a day a "du" is needed against each customer's backups in order
to work out exactly how much has been used and the differential
usage between iterations.
All of this is obviously a large amount of IO and in fact when the
backup VMs were on hosts with HDDs it was getting difficult to have
a backup run complete within 4 hours (the maximum available run time
at 6 times per day).
So at the moment BitFolk can't risk putting backups on HDDs again as they
will end up running too slowly.

Revision as of 03:31, 28 January 2017

All about the cheap, low-performance archive storage on offer at BitFolk.

What is it?

From late 2015 BitFolk went all-SSD for VPS storage, primarily for scaling reasons. The excellent random IO performance of SSDs has solved that performance issue but has not allowed for any reduction in storage prices, which is something those using their VPS for mass storage have been looking for.

Therefore archive storage is being introduced which can hopefully serve that requirement. It's a much cheaper optional extra with no performance guarantees.

Whereas regular storage is backed on enterprise SSDs in RAID-10, archive storage is backed on 2.5" SATA HDDs in RAID-10.

No performance guarantees

A typical SATA HDD can provide only about 80 IOPS of random IO and is not suitable for anything which has a performance requirement. You should expect data on archive storage to be reliable but not fast; ideally it should only be used for things which will do streaming reads and not many writes. For example, as a destination for backups. Even then, if you need to read the filesystem back at each backup run (for example to compare the current run with the previous) then that might require a lot of random IO which will not be speedy.

How to order

Just contact Support and ask for a chunk of archive storage. They're sold in multiples of 50GiB at the moment with prices as per the price list. It will be added as a separate block device (e.g. starting at xvdc). If this is for an existing VPS then you may need to have your VPS moved to a different host as not all hosts support archive storage.

Frequently asked questions

Can I run my whole VPS on archive storage?

BitFolk does not want to sell VPSes with less than 10GiB of SSD storage because they would be unacceptably slow to boot and manage and this could affect other customers.

If you want to purchase 50GiB of archive storage, move most of your VPS to that and then use the 10GiB of SSD for something else then there is nothing stopping you doing this, however. For example you could have /boot on a small xvda1, / over on the archive storage on xvdc1, and xvda2 being the rest of the SSD which you use for something else.

Can my root filesystem be partly SSD and partly archive storage?

No. Each block device (xvda, xvdb, etc.) can only be either regular (SSD) storage or archive storage.

Can I accelerate archive storage with regular storage?

Various projects exist to accelerate a slow block device with a smaller, fast one. For example:

There's no reason why any of those won't work inside a BitFolk VPS. BitFolk would be very interested in any reports of attempts to do this; they would make for an excellent wiki article.

Can I use archive storage for my BitFolk backups?

At the moment not. BitFolk's backups currently use rsnapshot, which works like this:

  1. rm -r hourly.5
  2. mv hourly.4 hourly.5
  3. mv hourly.3 hourly.4
  4. mv hourly.2 hourly.3
  5. mv hourly.1 hourly.2
  6. cp -al hourly.0 hourly.1
  7. rsync […args…] root@your-vps:/path hourly.0/your-vps/

That happens at every backup run, so potentially 6 times a day. Also once a day a "du" is needed against each customer's backups in order to work out exactly how much has been used and the differential usage between iterations.

All of this is obviously a large amount of IO and in fact when the backup VMs were on hosts with HDDs it was getting difficult to have a backup run complete within 4 hours (the maximum available run time at 6 times per day).

So at the moment BitFolk can't risk putting backups on HDDs again as they will end up running too slowly.