Anyone know much about IBL?

'Image Based Lighting'
Iray's Environment Maps allow the prospect to drive lighting based on an image map. For various reasons, you want as much bit depth as possible, so the lighting can be 'accurate.'
But I've had terrible results trying to drive good, well-lit scenes using it -- at best, I use a map for background and to provide some basic ambient light, and then have to resort to spotlights for any decent sun-like effects.
Even if I make a simple white disk on black, the shadows I'm seeing are _really_ anemic.
Anyone have any tips?
I've played with the Luxo HDR, the shadows are the best I've seen... and still very faint. I've also jumped through lots of hoops trying to make IBL-level HDR maps, to no avail (I mean, they work as well as the other stuff I have -- poorly)