In my limited experience, most of the walls would have been covered by wood panelling, tapestries, paint, plaster, or antlers, or some combinations of the above. Very little bare stone would have been left visible in the public areas and living quarters. The only places I've seen with all bare stone are ruins.
But in Skyrim, every single castle, tower, keep and fort has pretty much bare stone walls all the way through ... it's been bugging me for a while now. All the effort into making the game pretty, but somehow they got their ideas for the interiors by only looking at a few ruins ? Bit lazy, really.
 Sorry :)
Original post on Dreamwidth - there are comments there.