Emacs

I'd agree but it goes a bit beyond. Each file has pointers to the source code that is locally checked out. Sure it's not dynamic (e.g. my comments and understanding of what's going on don't sync with the codebase as it changes), but it still allows me to keep a mental map of what's going on and where each piece of the pipeline resides.

It's a mixture of roam, literate programming and an IDE for me. E.g. I can easily spin up a vterm, ssh into a spark node and test some of the code if I don't understand what it's doing thoroughly or to check if something isn't doing what is intended.

In my use case, collaboration is not required. I'm doing it for me so there's no real cost to the documentation and understanding getting out of date. I find emacs an invaluable tool to deepen my understanding and test things in the codebase. An IDE just doesn't fit the bill in that regard.