I've always been told the best thing to do is stick to complimenting things that are their choices.

Not great: That outfit makes you look good!

It's you coming off as being interested in their physical appearance, not the outfit.

Better: That's an awesome T-shirt! Where did you find that?

It's you thinking they picked out something cool or stylish and you like their taste in outfits. You're putting the attention on something they did, not anything about them appearance-wise.

Especially if they don't know you, odds are they have no desire to hear a stranger's opinion on their looks. That's too personal. But a stranger agreeing with their decision on something like buying something cool generally isn't.

Of course, some people are more or less open to any conversation with someone they don't know, so if you still get ignored or get looked at like a creep, you don't know their background and you respect that and don't persist.

A good rule to go by is if you're a guy, think of a guy coming up to you and saying the same thing or you saying what you're going to say to another guy. If you wouldn't tell another bro that he looks good wearing that, maybe don't do that to a girl. If you see a guy wearing a band shirt of a group you like though, you'd probably be ok saying "whoa, I love that band too!" or you'd be cool with some random dude telling you the same.

You shouldn't be afraid to talk to people, but you should always be respectful and keep in mind how well you know them and keep conversation at that level of appropriateness.