Dead Branches, Lilly Pilly

Hi,

I recently moved into a new property where the previous owner planted a beautiful "hybrid dwarf Lilly Pilly" hedge. It is around 2.2m high although some of the trees have not grown so well leaving small gaps in the hedge. The previous owner suggested this was due to those trees being planted too low?

Unfortunately they also erected a tall fence (1.8m) in front of part of the hedge (west facing..we live south east Melbourne close to the beach..I believe the fence was to protect some of the slower growing trees from the wind).

Anyway, we want to get rid of the fence but I can see that behind it there are a lot of what appear to be dead branches (no leaves).

Is there some way to restore these branches and leaves before we decide to do anything further as it would influence what we choose?

Also, what recommendations are there for promoting the growth of the smaller trees?

Thanks
kkcc