Calldav sucks and ActivityPub is really good. I would probably implement both - use caldav for compatibility with other software and ActivityPub for your own internal representation of the data.
But there's another option, also an industry standard, which is structurally very similar to ActivityPub only it provides more than just a network protocol to transmit data, it also provides a storage method and tools to work with it.
I am of course talking about git.
Put all your tasks/etc in a directory, use git to track and sync/backup/share changes. If you want a kan ban board, just have todo/in progress/code review/done directories and move your stuff between them.
Personally I do my note taking in VSCode with an extension that automatically (and almost instantly) commits changes and pushes them to the cloud/my other devices. Plus a few other extension like Foam which which supports linking documents/etc.