I've had that problem. The craziest thing about this is the pain. Most of the time, the reason I discover my flashlight turned on is I feel a dull burning sensation on my thigh. I always think "huh, that's annoying. Must be a random aberration on my skin like a zit or a bug bite". But the burning grows hotter until eventually I either pull my phone out to feel the spot on my thigh or I go to the bathroom to check. I swear it's happened to me a half dozen times or so by now.
It's not a big deal. The burning isn't burning. Just uncomfortable having a bright LED on my skin for that long.