Possibly my favourite gaming machine after the PC is the PS1. The PS2 is also good, but many of the notable releases are available on the PC anyways. I'll list PS1 games only, at least for now.
Many overlook the PS1 out of nothing but ignorance. If you want to get into the machine, I'd say start with Castlevania: Symphony of the Night, a genuine classic still frequently discovered by many today. After that, your guard should be down and you'll be more susceptible to other potential gems.
I've tried to remain exclusives-only, versions that were best on the PS, or games that were originally PS exclusive but later saw ports.
(J)RPG
Castlevania: Symphony of the Night (ARPG)
Koudelka (Turn-Based)
Numerous FF games (Turn-Based)
Vagrant Story (RTwP)
FFTactics (sRPG)
Action/Adventure
Medievil
Tomb Raider series
Blade
Silent Bomber
Nightmare Creatures
Legacy of Kain: Soul Reaver
Survival Horror
Resident Evil 1 & 3
Silent Hill
Parasite Eve 2
Martian Gothic
Platformers:
Spyro the Dragon (the first one only)
Crash Bandicoot series
Tomba
Stealth
Metal Gear Solid
Tenchu 1 & 2
Shooters
Shooters on the PS1 was essentially equivalent to playing keyboard-only shooters from the pre-mouselook days. If that doesn't bother you, then there's:
Duke Nukem: Time To kill
Duke Nukem: Land of the Babes
Alien Trilogy (probably better off playing the later-released PC version).
Syphon Filter 1 & 2
South Park (one of the weirdest shooters I've ever played)
Exhumed
The prime thing the N64 did better than the PS1 was shooters, namely Turok and Duke Nukem: Zero Hour. Many of the above wont hold up for most while those N64 shooters certainly do.
Strategy
Hogs of War
Worms Armageddon
Puzzle
Kula World
Abe's Oddesey
Other (Shoot Em Ups, Racing, Simulation, Beat Em Ups etc)
G-Darius
Vigilante 8 series
Grudge Warriors
Driver
Overboard
Bloodlines (not Vampire the Masquerade, but a pretty cool game nonetheless)
Tony Hawks Pro Skater series
SCARS
Einhander
A Word on Ports
An absolute crapton of PC games were ported to the PS1. Sometimes with big changes, sometimes with next to none. Many of these games can be worth experiencing, and many are not if you have the PC version available. An example of big is Doom's sound design, music and lighting changes. Doom ends up with a very heavy atmosphere as a result. This can be replicated with mods/wads on the PC version.
A word on emulation
Don't be a faggot by using save states, play the games as they were originally designed.
In the case of games that were originally designed too easy (this is rare, but Castlevania: SotN is one example), there are ROM hacks (mods) for certain PS games that increase difficulty, rebalance, and sometimes add worthwhile content as a bonus.