There's tons of ways to make a space unsafe while it maintains a perfectly civil appearance.

Those ways would fall under "plausible physical threats." Absent that, we're talking about people being uncomfortable (which can also be bad and may warrant addressing), but that's not the same as unsafe.