We have also been finishing up our study of earth, and with it, caves and caverns. We read a few books on the subject, but my favorite was Caves and Caverns by Gail Gibbons. I especially appreciate her explanation of how limestone caves are formed -- a topic I found a little confusing in Caves (A True Book), which we also read.
Another favorite -- especially for the younger crowd -- will be Caves: Mysteries Beneath Our Feet. We started off with it.
Finally, we looked at postcards from Luray Caverns which we have visited several times.