To get the leather/plastic handle portion off of the lever, well, pretty easy (if you're lucky). Some slide right off, others require a lot of effort. They aren't glued on or anything, just slid on to the metal shaft. It helps to pry from the bottom with your fingers to break the seal.

 

Going by your description it sounds very similar to the way the gearknobs on manual transmission cars are removed.
Very easy on paper, quite difficult and maybe harmful in reality

Will give it a try, otherwise my local dealer gets to deal with it.

 

Unclip the plastic portion of the handbrake trim base from the console; pull the cover over the handbrake lever - it will be inverted, then you can twist the cover to free it from the base of the handbrake lever & slide it off (inside out, effectively)

 

@lookforjoe
Thanks for the input, but I'm talking about the other cover. Sorry if this wasn't clear from my first post

I've uploaded two pictures to show what I'm struggeling with:

I've taken apart the complete handbrake mechanism - not a fun thing to do, the snap rings are annoying. Right now I am unable to pry the leather part from the lever, it feels like I'm going to damage it. I'll try my best and report back.

PSA for all those that like to buy cheap parts:
Don't buy parts from a water damaged vehicle! The "slight amount of rust" is the result of the car catching fire, being extinguished and sitting for two weeks. The bushings were seized to the lever, making it harder to move than usual. Also one bushing showed some wear, so if anyone ever restores a P80 car those might be a good thing to replace. Handbrake assembly came from a '04 C70 convertible with 165.000KMs.

 

Burnout was right.
The leather handle does simply slide of, it just takes A LOT of strenght. Unlike the shift knobs on manual transmission cars you probably won't damage the handle unless you pry at it. I've got no reason to suspect that you can't reuse the handle.
Given the amount of effort it's easier to just buy a flawless handbrake assembly and put that into your car.

 

Does anybody know how to remove the plastic hand brake sleeve? I want to replace it with the leather one. As I can see the leather one has a soft plastic / rubber base and you can twist it and pull it off. But the plastic one that is currently on... that's a though one because it's all hard plastic. I have tried to twist it and pull it off, but it just doesn't move. I don't want to put to much pressure on it and break something.

Found this on another forum. This is the base and I have indicated the groove where the plastic pin sits.

I assume it would be the best to just carefully cut the plastic one off. But that would probably be a big mess and I don't want to destroy it because it's in nice condition.

Any ideas would be appreciated.