If you've been eating broccoli throughout the whole selective breeding process, then the flavor change would have been subtle enough that you don't realize there's been a change at all. If you ate them side by side, the difference would be noticeable.
It's not too different from Jim adding nickels to Dwight's phone, then suddenly removing them.
You wouldn't notice slight changes over decades. You'd need to do a side by side comparison to say for sure.