Games like Ultima Underworld
 

I'm looking for games like Ultima Underworld, have beaten UW1+2 twice.

Swinging a sword in first person together with some magic, inventory management, difficult puzzles, some RPG statistics... those were good old school games which I remember until today. But I haven't found any games which are like this. If you know it then tell me?

One game I can recommend to you - Arx Fatalis.
It's pretty similar to Ultima Underworld series, it's even said to be an unofficial successor to the series. It's also taking place underground, you're using runestones to cast magic, the entire feeling is there. Unfortunately, just like the Ultima Underworld games, it's also a bit too short and is finished before you notice it. Also there's no music unlike in UUW games, which makes for a bit more gloomy mood.

There's also System Shock, from the same company, Origin. It's not a fantasy RPG like UUW though but more of an Adventure / Action / Horror mix, but very to recommend. Once you try it out you'll surely love it.

The best has already been mentioned Arx Fatalis is the closest to a modern UW you're going to get.

I wouldn't recommend System Shock though, it's not an RPG and it doesn't have the same element of exploration that's so important in UW.
You can try the Elder Scrolls games, but as a UW lover, I have to say that I personally despise those games (except for Daggerfall).

Try your hand at Arx Fatalis and perhaps try the Gothic series (1 and 2 specifically), while not first person like UW they derive heavily from ORIGIN's masterpiece and share the same feeling of exploration and danger.

Elders scroll is much more and the puzzles are different. But i would recomend them.

Arx Fatalis is a must if you liked ultima. Indeed there is no music, but that is great for me. It also has plenty of sound effect that fill that voide making you feel like you are in Stygian Abyss again...

I third Arx Fatalis, it basically is UW3.

Blood-Pigggy is correct in that Elder Scrolls 1 + 2 are heavy RPG games (and are huge games), but can get very repetitive. However, the best game in that series, Elder Scrolls III: Morrowind, is a masterpiece. If you can get your hands on it, I highly recommend it.

System Shock 1 + 2 are more FPSs with an RPG element in them. System Shock 2 is a good one to try though.

In my opinion, having played all the aforementioned games, Morrowind is likely the closest modern equivalent of the UW experience that you're likely to find. Barring that, Arx Fatalis is your game.

I hardly consider Morrowind to be more like UW than Arx Fatalis, aside from the perspective, they are nothing alike, but can see where you're coming from. UW did essentially establish the free form movement of most first person RPGs.

Arx Fatalis has a similar combat system, setting, writing, and it even aludes to some of the plot used in UW, it is the quintessential modern UW conversion.
As I mentioned, I utterly hated Morrowind and the rest of the TES series and only consider Daggerfall worth playing, that is one rocky RPG series.

Do FPS RPGs count?

If so, try Deus Ex. It's a masterpiece.

there is First-Person view in Gothic 3, ya know? ;)

other then that there is the Wizardy series, or Wizard & Warriors, although those are party based.

They're all not like Ultima Underworld series, though, unless you mean the ancient Wizardry 1 through 5 games. Taking it strictly, Ultima Underworld games are Dungeon Crawl RPG's, while Wizardry 6 or 7 are general, outside world RPG's.
Same with Gothic or Elder Scrolls series, they're all general RPG's, but Arx Fatalis is Dungeon Crawling since it's playing strictly "under earth's surface", so it is the same as Ultima Underworld if not for all the rest that makes it a successor.