| Recent Comments |
| Categories |
| Archives |
| Tags |
Backup strategy
A frequently returning question concerns best practices for computer file backup to prevent loss of your (audio) files and projects. Designing a good backup strategy involves many aspects and is not easy. There are even companies that advice on this. If you do a Google search you will find many resources providing useful insights about computer backups. Below I describe my strategy and approach that is based on many years of experience (and failure to restore backups). This works for me. I am happy with it. But you may need a completely different strategy. Think about it.
3-2-1 Backup Strategy
I have always followed a 3-2-1 backup strategy, which implies having at least three total copies of my data, two of which are local (inside my home) but on different hard drives, and at least one copy off-site in the cloud. In practice this implies that I have all (audio) files, many Tbytes, on my SATA disks inside my studio computer. I don’t use RAID for data redundancy, although this could further improve reliability it also increases backup costs. In the last 10 years I only once had problems with a disk but even in this case I could clone 99% of the files to a new disk without having to rely on my backups. In addition, I have a backup of all files on two local Synology NAS (Network Attached Storage), currently a DS218 with 6Tbyte and a DS218 with 14Tbyte of disk space. These NAS drives are also local inside my home. Finally, I have a copy of all files in the Cloud for which I use StratoHiDrive, thus located on a computer server somewhere in Europe. In case my computers/NAS are stolen or my house burns down, I can still access all files from the Strato server.
Perhaps Dropbox is an alternative for HiDrive but I never looked into this in detail. Dropbox is also not free if you need more storage space than offered by the free variant, your files may reside on US computer servers (which is not a problem per se), and your backup software should be able to access Dropbox.
In the end it is a balance between investment (costs for NAS, Cloud service, power consumption) and risk for loosing data.
Image 1: Synology NAS (DS218). Unexpectedly, the second disk of the second NAS crashed (yellow led).
Backup format
Once you decided where the backup your files you need to decide about the format of your backup. For me there were always two considerations:
- Backup format
- Encryption
Backup format
Based on my experience I know that I hardly ever need any of my backups (but we want to be safe, right?) but when I need them it sometimes turned out that the backup could not be restored (e.g., because it became corrupted without me knowing). Therefore, in general, I don’t trust backups. However, this mostly applies to backups that were made and stored in some kind of special format (blocks) by the backup program. A block is a raw storage volume filled with files that have been split into chunks of data of equal size (see image 2 below). Although there are advantages for storing backups in a specialized format (e.g., tib files), I generally prefer to backup all files and folders in there native file-based format such that I can see what I have backup’ed and such that I can directly access any file or folder without the need of dedicated backup/restore software to restore (or inspect) the file (image 3).
Image 2: Backup of files from my Windows computer to Strato HiDrive with use of Acronis True Image. These .tib files contain the files and folders that were backup’ed. These proprietary tib files are binary files (blocks) that can only be read (restored) by the Acronis software. Note, that I do not longer use the Acronis software but currently use the GoodSync backup program.
Image 3. Backup of files from my Windows computer to Strato HiDrive, which I made manually. The screen dump shows one of my Cubase mixing projects in their native file-based format. I can see what has been backup’ed (which gives me trust in the backup), and I can restore any individual file without the need of dedicated software. However, a manual backup requires some discipline.
Encryption
Regardless of the backup format (blocks or file-based) you may or may not encrypt your backups. Modern encryption scrambles data using a secret value or key known only by the person who makes the backup. There are different types of encryption and the best systems balance safety and efficiency. Sometimes the possibility to encrypt is offered by the backup software (e.g., GoodSync) but otherwise there are stand-alone solutions that you can use such as VeraCrypt. For reasons explained above, I generally do not encrypt my backups since it will be impossible to directly access the files without decrypting. I only use Encryption for files that contain sensitive data.
Backup frequency and scheme
There is a lot to say about frequency and scheme. The backup frequency determines how often you make a backup of your files. This is basically up to you and it should balance the risk between loosing files and consumption of CPU/network/disk resources. However, in general, you don’t want to have frequent backups of folders (e.g., sample libraries) that don’t change regularly. On the other hand, you may want to backup your current mix project on a daily basis. Thus, you need to setup different backup frequencies for different files and folders on your system. These frequencies may even differ between your local (NAS) and Cloud backup. I tend to more frequently make backups on my NAS since it is fast compared to a Cloud backup (although this is no longer a issue since I have a gigabit glass fiber upload connection from my studio computer). Backing up large sample libraries (e.g., Kontakt or EWQL) may easily take many days or even weeks if you are on a slow network (but once you have a backup, you don’t need to do it again unless you want backup updates to these sample libraries).
Another consideration is the backup scheme. Dedicated backup software such as GoodSync, Acronis True Image or Bvckup allows you to specify if you want to make full or incremental backups, how many copies you want to keep, etc, etc. See [here] for more information. My strategy is to keep a very limited number of incremental backups to save expensive (cloud) disk space. The drawback of this is that I cannot retrieve lost or changed files from, for example, a year ago. For me it is not worth the investment in Cloud storage to increase my diskspace 10 fold or more to keep older versions. Files that I delete on my computer will eventually, due to my backup scheme, also be deleted from backups made with GoodSync or Bvckup. However, when doing a manual backup you can control this yourself. Rationale being that if I delete a file on my computer then I really don’t need it any more). Of course, this brings the risk of accidently removing a file (without realizing it) and finding out after one year you still need it…..
Anyway, backup frequency and scheme are crucial and it is worthwhile to learn more about these schemes and the possibilities offered by backup software before starting a backup at all.
Backup software
There exist many (free) backup programs to help you with your tasks. Duplicati is considered to be one of the better ee backup programs. I have used it for a long time until I needed to restore some files and, for unknown reason, the backups turned out to be corrupt. Nowadays, I use three different programs for making backups
- Bvckup (for backup to my NAS)
- GoodSync (for backup to the Cloud)
- FileZilla (for manual cloud backups)
Note that I used the Acronis software for a long time but too often I ran into problems with this software. Once in a while Acronis could not make a incremental backup because it was ‘missing’ files. I never found out the cause of this but in such case you need to redo a full backup again (instead of an incremental). Although this is not a problem for backups that are relatively small, few of my backups (e.g., the sample libraries, videos) are very large and it would take days again to backup during which I could not shutdown my computer or infer (e.g., pause) such backup, or even could see its progress. Moreover, HiDrive no longer support the regular ftp protocol but only SFTP or FTPS, which are still not supported by Acronis (see [here] for the different ftp protocols).
Advantages of GoodSync are: cheaper, faster and easier to setup. GoodSync backups are copies of the individual files, i.e., in their native file format (instead of binary blocks) and therefore can be manually retrieved from the (cloud) server when needed. On the other hand, Acronis offers more extensive options to setup backup schemes.
Bvckup and GoodSync are dedicated backup programs that can automatically backup your files and folders with any frequency and backup scheme you wish. The main advantage of this is that you only do the configuration once and then (theoretically) you never have to think about your backups again until you need them or until you reorganize and add disks to your computer. On the other hand, FileZilla is not a backup program but and application to transfer files between computers (e.g., to the NAS or Cloud) using the (secure) file transfer protocol (ftp).
I use Bvckup to backup all files from my studio computer to my local NAS in its native file-based format. These backups run with a pre-specified frequency but often I use GoodSync to make backups to the cloud (HiDrive). I also run part of my cloud backup’s manually using the FileZilla ftp client FileZilla. Since FileZilla is not a backup program, I cannot set backup frequencies nor backup schemes automatically. I have to manually start FileZilla and then select the files I want to backup, and finally copy everything to the cloud. Although this seems a hassle, it is not and it gives much control over the backups, and all files are copied in there native format. But…..it is my responsibility to backup since the process does not run automatically.
Administration
Finally, personally I find it convenient to have a good administration of my backups. I keep an Excel file with all disks and folders that I backup and their backup locations. If I exclude files/folders I also make a note of that. Few files/folders are not backup’ed to the NAS or Cloud to save space and since it would not be the end of the world if I loose them. However, I do keep a separate copy of these files/folders on external hard drives, which I also document in the Excel file.
See also
