I can second Ascendance of a Bookworm that has been mentioned. It's a very good series.

Escaflowne was mentioned too, which I found great back when I watched it, but it's been too long for me to remember. I want to rewatch it at some point. (Like I did with The Twelve Kingdoms which I found much less good today than back when I first watched it.)

I'll list a few more and add my rating assessment (ranking as exceptional, great, very good, good, decent, average, bad, …). I ended up going through the anidb isekai category, so I guess these are all I know of, and it's quite a lot of them considering it feels - as you describe it - like all or most isekai end up with lewd harem fantasies. As for the reason, I think it is primarily business people / production management producing for easy baits and the most generic middle audience - young males. Because of the success of isekais they produce more, and end up with the standard tropes of lewd, harem, and over powered, which very often ruins it. But as the following list shows, it's not always like that - even for an over-saturated filled category like isekai.

Handyman Saitou in Another World is another very good one that's not harem. Great comedy.

Grimgar: Ashes and Illusions is very different and very good.

The Magical Revolution of the Reincarnated Princess and the Genius Young Lady is very good until the ending, where it falls short and forces itself into something bad. The early and mid of the series is great though, with exceptional visuals and vibrancy.

I'm the Villainess, So I'm Taming the Final Boss is good. (Paired romance.)

In the Land of Leadale is good. Female protagonist.

Kuma Kuma Kuma Bear is simple but heartwarming, overall good within its simple formula. Female protagonist.

Life with an Ordinary Guy Who Reincarnated into a Total Fantasy Knockout is very good, great comedy with a gender bend/swap.

Now and Then, Here and There (2000) is great.

No Game No Life has a sibling pair as protagonists. (Prequel movie is great but not an isekai, the very good series is. Although it has its own share of "lewd service" IIRC.)

My Isekai Life: I Gained a Second Character Class and Became the Strongest Sage in the World! is only average, but a non-harem isekai as far as I remember and from what I can see skimming through.

Reincarnated as a Sword is good.

Saga of Tanya the Evil is good.

Saving 80,000 Gold in Another World for My Retirement is good. Female protagonist.

The Devil Is a Part-Timer! is very good.

The Great Jahy Will Not Be Defeated! is good.

The Twelve Kingdoms (2003) is decent.

Uncle from Another World is great.

Kagami no Kojou (2022, movie) is great.

If it's about the boy harem, there's those series with a female protagonist and reverse harem:

My Next Life as a Villainess: All Routes Lead to Doom! is good. (Otome-based.)

The Most Heretical Last Boss Queen: From Villainess to Savior is good. (Otome-based.)

Why Raeliana Ended Up at the Duke`s Mansion is good. May be reverse harem, or paired romance - I don't remember in particular. (Otome-based.)

Bofuri: I Don't Want to Get Hurt, So I'll Max Out My Defense. is not isekai transported, but videogame playing in world.

When broadening to fantasy, I rewatched Moribito - Guardian of the Spirit (2007) a little while ago, which has a strong female protagonist, and is exceptional. Definite recommendation.