Starlings in Woodpecker Nest

Just bought a house with a big old weeping willow in the back, looks like 20-40 starlings live there (maybe a slight exageration but there is alot). Not a paritcular fan of starlings they are to me a junk bird like Seagull (my state bird) they mimic other birds (and my cat) so it sounds like I have more of a variety back there then I do. But I would really rather have a woodpecker, so my questions is short of killing them is there any way to get them to leave (eg some kind of baffle? placed outside the hole) so they could leave but not return. Or am i stuck with them,and maybe the woodpecker would not return anyways. What normally happends to an old woodpecker nest? Trying to build a more bird freindly habitat so that I can get some variety have more then my share of doves, wrens, starlings occasionally see some quail. No hummers yet this year. Thanks for any & all info.
Transplanted Canadian