I'm talking about having the right to never release a work to the public in the first place (replying to another comment on that). This has nothing to do with scarcity.

The simple argument is: you can choose to create something and never give it to anyone. Nobody is entitled to take it (that is a basic privacy principle). But if you do release something to the public, either for free or for sale, then there should be rules protecting the public's access to that work.

This doesn't mean it has to be the end of copyright. Yes there's no scarcity, but there still needs to be a function incentive to create the work in the first place, so a little artificial scarcity creates that incentive. But once the work has had a reasonable lifetime under copyright, or is no longer legally available, then yes we absolutely should be able to access it as part of the public domain.