I would like to make my own NAS.
And basically I would like a lot of TB – for me that’s probably somewhere between 50 and 100 TB for now. And Perhaps between 100 and 200 TB within 5 years.
I would very much like, but I might not actually *need* some kind of deduplication, – to save space.
But I want the hardware to be cheap and not to use too much power.
I am thinking to build my own server, and install the needed software myself. I.e. I don’t think I want FreeNAS or another turnkey solution.
I am thinking a normal PC. I have an old on with a i3 processor. It only have 3 SATA, but I am thinking to install a 4- or 8-port SATA adapter. It just have 12 GB of memory.
Right now I am thinking in the direction of FreeBSD on ZFS. Probably 6×18 TB hard drives in RaidZ1 or Raidz2. That would give me 108 TB RAW, but somewhere around 70-90 TB depending if I choose Raidz1 og Raidz2.
For backup I am actually thinking a Raspberry Pi in a different location. And I am thinking to let it be turned off for 14 days. And then every 14 days it will get some incremental backup files. I am thinking to make one every day, perhaps even more often than that and then store it on a 3rd computer, or perhaps in the cloud, until day 14. The reason for this is a) To make it use just around zero power, it’s not going to be in my place, but also b) to make it a little bit safer. If I get hacked in a bad way, chance is that I’ll find out and stop the backup process and then have a good copy and some incremental files that I will be very careful about installing or perhaps discard.
But I am very much in doubt about the Raspberry setup. I don’t want to make a Raidz or striping or anything like that over several USB discs on the Raspberry. It sounds a bit like a disaster waiting to happend! But can I backup a 50 TB dataset in a good way to several smaller filesystems/datasets?
For now I think this is what I can afford to do.
First priority for improvements would be backing up locally, and hence get to 3 copies.
Second priority would be to get a better fileserver. Hopefully I would be more informed and know better what I want. But for now, I am thinking a faster and especially to get lots of ECC memory and two mirrored NVM SSDs. I’m thinking 128 GB memory, 2×2 TB Nvm, I guess at least 2,5 Gbps NIC and at that time probably 8, 10 or 12 drives of 20 TB each in raidz2 or perhaps even raidz3. Yes, I know, 20 TB isn’t out yet. But I’m not ready to buy yet either. I guess 20 TB would be the sweet spot in 1-2 years which is likely to be the time.
I don’t know if it makes to much difference. But my hoarding is party videoes. They do not change. They do not require much speed. They don’t compress. But also a lot of HTML files, some databases that I populate myself. I am probably not neurotypical and I like to collect data. And it takes up a little more space than most people imagine 🙂 RSS files and some screen scraping that I process and put into databases. And then of course personal files, python-, java-, php- etc source. As well as the boring standard stuff, some personal images, some mails, some letters, excel spreadsheets etc. etc.
I’m not really sure how and exactly what I want. But I suppose that I’ll learn that it would be better to have two or more different filesystems with different rules. Snapshots of my videoes are actually not important at all, I suppose. And there’s no reason to compress those files. My source-files is the exact opposite. I would like compression, I would like snapshots. And I suppose speed is of some importance, especially to be able to open/read hundreds of small files fairly fast.
Is FreeBSD, ZFS, Samba/NFS the right solution? Or is there something better?And do you have some ideas about how to back it up. I would prefer the Raspberry with external USB-drives. But how to make that work is still unclear to me.