One thing about great exhibition places is their ability to balance the interaction of the exhibit with the space around it. In some cases, the surrounding space can be cleverly used to enhance the effect the exhibit has on the observer, whereas in others the surroundings can be minimized to let the exhibit take full prominence. Chihuly Garden and Glass museum in Seattle, WA uses lights and reflections to highlight some of the glass exhibits. In the picture below, the reflection on the bottom makes the glass sculpture seem even bigger and more impressive.
Another thing I like to see in exhibition places is the connection between inside and outside spaces, and the ways the outside permeates the inside space. Whether it is letting natural light in, or positioning windows in interesting ways, seeing a little bit of the outside environment often enhances my experience. One example is the water wall at the National Gallery of Victoria in Melbourne, Australia. The fact that the glass wall is covered by water minimizes distractions from the outside but still brings in more daylight.