It seems like the benefits are having the device lock/wipe itself after a set amount of attempts in case of a brute force attack and not having to run software to decrypt the drive on the device you plug it into.
I included a picture of the IronKey Keypad 200 but that's just because it's the first result that came up when I was looking for an example. There seem to be a few other manufacturers and models out there and they probably have different features.
I am curious what do you think of them? Do you think they are useful? Do you find it more a novelty?
It was an ExplainingComputers video titled Very Useful Small Computing Things that made me think of them.
I'd stop short of saying "easily" since you have to get the epoxy potting off of the chip. But you are right that there doesn't seem to be any active tamper reactance. The numeric key is apparently 8 digits. Since it's a 10 digit keypad, at least 2 of the digits are unused, and you might be able to recognize those from the comparative lack of fingerprints and wear on those specific keys. So that narrows down the search range some more.