Very good points, Thank you!

I guess the most canon-compliant way to make this idea work is that they were a cleric of a deity that died. That does happen in DnD settings sometimes and I would expect that would remove their access to divine magic. Of course I would expect that rules would let you substitute a different deity with similar domains, and there are definitely skills and feats you wouldn't lose with your magic, but it would be an interesting backstory.