Gahhh. I FINALLY got it out with pliers (I found some which actually fit inside the thing, barely). Pulled out the old one, put the new one in. It was tight getting back through the hole but I got it firmly locked in.
Put it back into my car and... The light doesn't work. ARGH!
To make it all worse, it's now beginning to rain, so I can't pull it and try to fix it today.