this post was submitted on 21 Jan 2024
21 points (95.7% liked)

Selfhosted

40173 readers
963 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
21
How to backup object storage for NextCloud (lemmy.procrastinati.org)
submitted 9 months ago by SeeJayEmm@lemmy.procrastinati.org to c/selfhosted@lemmy.world
6 comments fedilink hide all child comments
 

I'm experimenting with running NextCloud (AIO) on a VPS with a B2 bucket as the primary storage. I want to compare performance compared to running it on my home server (esp. when I'm remote) and get an idea of the kinds of costs I'd rack up doing it.

As part of the setup I have configured the built in borg backup but it has this caveat:

Be aware that this solution does not back up files and folders that are mounted into Nextcloud using the external storage app - but you can add further Docker volumes and host paths that you want to back up after the initial backup is done.

The primary storage is external but I'm not using the "external storage" app. So, I have 2 questions.

  1. Does it backup object storage if it's primary (my gut says no)?
  2. If no, what's a good way to backup the B2 bucket?

I've done some research on this topic and I'm kinda coming up empty. I would normally use restic but restic doesn't work in that direction (B2 -> local backup).

It looks like rclone can be used to mount a B2 bucket. One idea I had was to mount it, read-only, and let AIO/borg backup that path with the container backups.

Has anyone done this before? Any thoughts?

top 6 comments
sorted by: hot top controversial new old
[–] morethanevil@lemmy.fedifriends.social 6 points 9 months ago (1 children)

Be aware your backup is useless, if you don't backup nextclouds database when using a bucket as primary storage ☝🏻

I use nextcloud with local storage and I use rclone for backup to a S3 bucket (MinIO).

You can use rclone to directly backing up your bucket. There is no need for restic. It is simple like rsync

An example would be:

rclone sync b2:mybucket otherprovider:otherbucket

You can use local storage too:

rclone sync b2:mybucket /my/path/here

Rclone can mount and backup almost everything. It is a swiss knife and I love it.

Performance is really good with nextcloud, if configurated well. Look at this thread

[–] SeeJayEmm@lemmy.procrastinati.org 2 points 9 months ago* (last edited 9 months ago) (1 children)

Be aware your backup is useless, if you don’t backup nextclouds database when using a bucket as primary storage ☝🏻

Understood. My hope was to mount the bucket locally (ro) and have it backed up with the container backups using the built in borg backup option.

rclone sync b2:mybucket otherprovider:otherbucket

I'd prefer to have proper incremental backups not just a warm copy of the data.

Rclone can mount and backup almost everything. It is a swiss knife and I love it.

It seems to be very capable but I cannot make it work for my purposes. I fought with rclone/aio for a few hours yesterday trying to make it work.

I was, quite easily, able to mount the B2 bucket to a local path. I used the --allow-other option to make it available to the whole system. Everything was accessible via the CLI, but the Nextcloud AIO admin refused to allow me to add that path to the backup job. I was unable to find any logs that indicated why. If I could get this working, I think, it would be ideal as the backups would be consistent.

I also tried using a couple of the serve options. The nfs option would launch but mounts would fail with protocol errors. I couldn't get the docker plugin to sync up properly with docker. I haven't tried the restic serve option yet. I can provide logs if requested.

Thanks for the help.

[–] morethanevil@lemmy.fedifriends.social 1 points 9 months ago (1 children)

Did you try to mount your bucket on your host system via rclone?

rclone mount b2:bucket /path/on/host --daemon --vfs-cache-mode full

I would mount it on the host system and add an additional volume in your docker-compose.yml

        volumes:
         - /path/on/host:/myvolume

You could give this a try, if you want to use it in your container 🤔

[–] SeeJayEmm@lemmy.procrastinati.org 2 points 9 months ago (1 children)

Did you try to mount your bucket on your host system via rclone?

I did. That's where I ran into the problems.

I would mount it on the host system and add an additional volume in your docker-compose.yml

I'm embarrassed to admit I didn't try this. I think I was too far into the weeds the other night. I'll give this a try.

If it works, I'm thinking I'll need to setup a systemd service to auto-mount the path on boot and set it as a dependency in docker so docker doesn't start before it?

[–] morethanevil@lemmy.fedifriends.social 1 points 9 months ago (1 children)

Your way should work with a systemd service. I do it the same way but I just need one bucket mounted in my file system, so I don't need dependencies

[–] SeeJayEmm@lemmy.procrastinati.org 2 points 9 months ago

I was able to get the rclone mount on boot via a systemd unit without much trouble.

I even managed to drag it kicking and screaming into a docker volume that I mounted as an external volume to the Nextcloud AIO stack. It still refused to allow me to add it as a backup directory.

I think I'm throwing in the towel with getting Nextcloud to back it up via the built in mechanism. I'll just schedule a separate job (cron/systemd) that runs shortly after the Nextcloud backup. It should be close enough for my purposes.