Messed up folder structure for arrs (lemmy.dbzer0.com)

submitted 1 day ago by ExcessShiv@lemmy.dbzer0.com to c/piracy@lemmy.dbzer0.com

2 comments fedilink hide all child comments

I just realised yesterday that the size of my media library was growing significantly faster than anticipated. The culprit seems to be that hard linking doesn't work in radarr and sonarr because I messed up the folder structure of my docker containers, so they've been copying everything to my media folders instead.

Now, the real issue is that my media folders are a mix of and older collection (manually sorted) and the new parts that has been added through radarr/sonarr, so I can't just nuke that and re-import everything from my torrents folder once I've fixed the folder structure. I don't want to nuke the torrents folder because I want to keep seeding all my downloads (most through private trackers, so I need the download credit it creates), but the imported library in my media folders has all the files renamed and many rearranged.

What's the easiest way to fix this? There's about 8tb of actual data, without duplicates, I have 6tb of unused disk space left.

top 2 comments

sorted by: hot top controversial new old

[-] Chewy7324@discuss.tchncs.de 9 points 1 day ago* (last edited 23 hours ago)

fclones is fast and supports hardlinking/softlinking of duplicates instead of removing them.

I've used it successfully to deduplicate my documents folder (and "archive").

As its quite the amount of data, I recommend using the --cache option to make subsequent runs way faster, if you want to dial in the options. This directory can be deleted at any point and isn't necessary.

https://github.com/pkolaczk/fclones

[-] Berstrrs@lemmy.dbzer0.com 5 points 19 hours ago

+1 for fclones

Btw. Always wondered if the name possibly comes from Fucking Clones

I'm using this in my unraid. Scheduled weekly, just in case.

#!/bin/bash
#noParity=true
fclones group --one-fs --follow-links --min 256MB --name '*.mkv' --name '*.mp4' --cache /mnt/user/data/torrents/ /mnt/user/data/media/ --max-prefix-size 128MiB --max-suffix-size 64MiB --skip-content-hash  > /tmp/dupes.txt
fclones link < /tmp/dupes.txt