I don't hear any voices at all when I read (which makes sense, as I read so fast that they'd all sound like one of those fast-talker commercials on helium!), although I can definitely go for the idea of Dr. Doom as James Earl Jones, as mentioned previously. So when I hear an actor or voice-actor (in a cartoon) taking on one of those voices, it very rarely clashes terribly for me (although there have been exceptions, where I had no idea what the character sounded like, only that they couldn't possibly sound like that actor!).
The most jarring one in recent memory was Viggo's voice as Aragorn. He's indelibly burned into my mind as the voice of Lucifer, from the movie Prophecy, and when he first spoke up in the Prancing Pony my spine crawled and I wanted to urge Frodo to run away as fast as he could! But that's nothing to do with a pre-conceived idea about what Aragorn would sound like and everything to do with Viggo's past associations.