A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
It's viable, but when you're buying a DAS for the drives, figure out what the USB chipset is and make sure it's not a flaky piece of crap.
Things have gotten better, but some random manufacturers are still using trash bridge chips and you'll be in for a bad time. (By which I mean your drives will vanish in the middle of a write, and corrupt themselves.)
I've had "trash controllers" in Orico units which rewite the drive details which makes them annoying to work with but I've never heard of
your drives will vanish in the middle of a write, and corrupt themselves
That sounds likes its underpowered and when the draw is up the supply can't handle it, which could happen for internal drives if your supply isn't up to powering enough drives.
Use a reputable brand like TerraMaster and you'll not have those sorts of problems.
I’ve personally gone with an N100 Mini PC running Proxmox and two of these daisy-chained (purchased on sale). https://www.amazon.com/MAIWO-Enclosure-Cooling-Storage-Expansion/dp/B0D28Q187R/
The MAIWO DAS uses garbage JMicron firmware by default, and there are significant issues with their sleep functions. Because of that, it took me forever to figure out why SnapRAID kept failing mid-sync. Fortunately, new firmware seems to have fixed their issues and they’ve been rock solid ever since. I specifically had to update the firmware for all 4 of the USB controllers on each DAS.
Direct link to firmware that worked for me. https://gbatemp.net/attachments/bin-16028_jms578_std_v00-04-01-04_self_power_odd_20190611-zip.230929/
JMS578_STD_v00.04.01.04_Self Power + ODD.bin
MD5: 7701fb7a968e3ad4ca926dd7854806ff
Firmware updater tool for Windows found here. I ran this from a Virtualbox Windows 10 VM inside my Arch install: https://gbatemp.net/attachments/jmicron-jms578-sata-crystal-enclosure-fwupdate-zip.216335/
FwUpdateTool_v1_19_16_24.exe
MD5: 735ec8d9f99c457ce793739480c55706
Mirrors for posterity:
https://files.catbox.moe/e4121s.zip
https://mega.nz/file/OJAX2KhQ#67kIDJun92nqi56mFur_9vALSi2yTJXXv7ew5pYSJVY
Blog post detailing firmware update procedure for an external drive: https://ralimtek.com/posts/2021/jms578/
Detailed post on JMS578: https://gbatemp.net/threads/how-to-update-firmware-of-jmicron-jms578-usb3-0-sata-enclosure-black-screen-lock-music-stop.569158/
Alternate FOSS software for flashing I found later, but never used. https://github.com/BertoldVdb/jms578flash
Wow thank you for these links, you're insane. And if you would start this homelab process again, then would you go with this mini PC + DAS layout again, or with desktop pc as a NAS?
I have the QNAP TL-D800S. It’s an 8 bay DAS but there is also a 4 bay variant. Works well for me. It uses SFF cables to connect to the PC and comes with the appropriate PCIe card which seems more robust to me than anything USB for this.
I built my own and it honestly wasn't that expensive (at the time back in 2018). I just started with the basics but built it to be expandable. I used a Define 6 case which gave me room for 12 HDDs, a mobo with the highest number of SATA ports, processor, RAM, etc and then just added drives 1-2 at a time as they filled up. My only regret is that I didn't and still haven't learned Linux well enough to rely on it because it runs Windows, the PC is showing its age now, and I need to think about the replacement solution and how I'll be able to migrate 70+ TB of media and all my configurations to the new machine.
If you do run Windows, Drivepool and SnapRAID are useful for pooling everything into a single virtual disk and setting up a software RAID that will protect from drive failures without locking your data away.
You can also use snapraid (and mergefs) on Linux to do the same thing! I'm excited to recognize this because I recently turned an old PC into a crappy nas with Open Media Vault and used these two.
If you use a reputable brand, such as TerraMaster you'll avoid all of the scare stories you hear.
Almost everything bad seems to be along the lines of
I bought "off brand" XzzYyG from Amazon/EBay/Wish and it failed
Something like this:
https://www.terra-master.com/uk/products/homesoho-das/d6-320.html
It'll be faster than your spinning rust anyway, as long as you have high speed ports on your mini PC.
Consider the machine being on 24/7 and cooling.
Furthermore, depending on the current power supply, you might need to upgrade it to keep everything running.
I had a setup similar to this for a year or two that ended with an hdd destroying itself one night. Probably because of the drive and not the usb enclosure I was using. Until then it worked fine, it’s definitely a viable route.
If you can swing a desktop pc case it’ll probably end up easier and cheaper and have some headroom for upgrades, that’s the route I went down after trying an escalating series of mini-pcs and running into their limits one too many times.
Sounds like a good idea, since according to the comments it seems I really shouldn't be using DAS as a primary storage (not with USB specifically).
I use an M1 Mac Mini running Asahi Linux with a USB 3.0 4-bay enclosure. Works great so far.
I have used mini PCs as a servers for years with file serving being a major duty of them. Granted my storage needs aren't excessive, but most NUCs or Nuc-likes can hold two drives, some can have a third if you include 2.5" drives. My AsRock A300 can hold 4 drives (two of each), but its m.2 support sucks so that's not as much of a boon as it sounds. If you need significant storage, there is no replacement for something that can hold 3.5" drives though since those can now reach 20+ GB a drive.
Did that for 3 years. Not pretty but it worked
Get old HP thinclient T510, or Igel M340C. Got a few of those practically free online. Has Via Eden 1.2Ghz 2 core. Not powerful at all. But cold. Mine runs on hot summer days approximately between 40-50C. HP has I think 19V power source, Igel runs on 12V brick.
Get a USB-C DAS (enclosure) for your disks, those use their own power supply. Since it is USB-C performance will be very good and stable and you'll be happy with it.
Yes, and you might want to ask in the datahoarders community.
While I dont use a mini-pc, I have a server with 48TB in it on spinning disks, and I've built a hybrid DAS/NAS that I back up to.
I use this 4-bay DAS: https://www.amazon.com/gp/aw/d/B078YQHWYW I chose it because it supports USB 3.2 Gen 2 and I've been pretty happy with it.
It's usually plugged into my server directly, and I use ZFS to snapshot and send to it. However, I also can plug it into a Pi5 and use ZFS send over SSH to treat it like a NAS. The Pi can of course run Samba/CIFS and SSH for sshfs.
The biggest downside to this structure is probably the metadata speeds for ZFS over USB (looking up snapshot names), but you could always use a cache drive with ZFS.
I highly, highly recommend ZFS and figuring out your software requirements before picking hardware.
Happy to answer any specific questions, too.
I am currently getting a replacement for my old win10+drivepool setup using hand me down parts from my old main pc. I decided to go with UNRAID this time and other than a few headaches, its been pretty simple to get going. I am already a big fan of the container/docker/app setup. My setup is 3 12tb seagate iron wolf nas drives recertified/refurbished from ebay, the nvme ssd I used as my prior boot drive as a cache drive, and a r7 1700 and ab350m pro 4 mobo with a pny xlr8 1070gpu. It was mostly free since I was replacing it with a newer gaming build. I think I spent $280 on the hdds?
I used intel nuc j5005 for 4 years iirc with 2 bay hdd enclosure, Orico brand. Never had any problem that i hear about such as random diaconnecting. Granted i used it to host my media. So i dont pratically mind if the data is gone.
I used openmediavault with usb boot drive. Internal 2.5’ drive as docker appdata. All my media files is stored on the external enclosure. The only problem i noticed is the temperature but thats because of high ambient temp.
If youre planning to use ext enclosure for critical data. Might wanna think twice. But, if you have proper backup plan. Sure why not.
I use a couple of mini pcs in exactly the setup you are talking about. Only downside is throughput. Anything built in is gonna be faster for read write ops, but usb3 is plenty fast for most things including media and data sync. I run Ubuntu with ZFS, and created raid arrays for data redundancy. It works really well. I virtualize using Incus with docker inside of those tiny VMs. It’s awesome.
I would either get a used HP Proliant microserver Gen 8 (I own one. Pretty sweet) or get a mini ITX nas case, 4 or more bays. Depending on USB doesn't seem like a good idea to me. You can always sell your mini PC. I have slowly learned to avoid the sunk cost fallacy.
Doing it now with a not so mini second hand pc
I used USB enclosures for my RAIDs for over 20 years. The turning point has been usb3 and then usb-c even better, but I found really no difference as in the bottleneck where the mechanical drives.
Moved to an all internal sata setup a few months back because I upgraded the space and moved to a desktop form factor.
Can still recommend the USB approach tough.
BUY A QUALITY EBCLOSURE.
I always used Linux software raid, but purchased a 4 slots USB raid/jbod enclosure to keep the number of used USB ports down.
I never ever had issues with the setup, but I purchased a known-brand enclosure, one with also e-SATA, which unfortunately was/is more a fad than even been really used.
There are small SATA backplanes that allow you to fit 3 HDDs into two 5.25" slots (or 4 HDDs in 3 slots). You can find used ones for cheap (mine was 30€), and with some cheap tower case you could get something NAS-like with hot-swap drive bays for way cheaper
You may not be able to do RAID or other redundant/performant arrays with USB. You can definitely achieve a big JBOD array but it will be less resilient and slower than a RAID array. Enclosures often don’t cool as well so heat may degrade your disks faster as well. I did this for a while with some old disks and some $30 HDD toasters. I only put data on there I could afford to lose. I wish there was a standalone hardware RAID solution… like a NAS without the network. That would have a huge draw for hobbyists that don’t want to buy an expensive NAS. I’ve searched for this but haven’t found anything. Message me if you know of such a product! Maybe consider building your own NAS with an old PC. Way cheaper than a prebuilt and fun to build! I had an old Dell Optiplex 990 that is now a 32 TB NAS. Had to get a new case but it’s a decent backup to my Synology.
This is completely untrue.
You can get plently of performant arrays over USB. You do know how much USB 3.1 or 4 can transmit?
Enclosures often don’t cool as well so heat may degrade your disks faster as well.
DAS enclosures can do a great job of cooling by separating and not sharing the air inside a single case.