Ylang ylang dying? To prune or not to prune?

Hi everyone, hoping to get some insight before chopping branches off my Ylang ylang tree. I live in Miami and was out of town for 4 weeks. When I returned my young Ylang ylang tree had dropped almost every leaf and has some black branches! Before leaving the tree had been small but full of leaves and very happy looking. I don't have a pre-leaving picture, but there's one of it as a happy tree, just shorter than when I left. It's been living happily in the same spot now for a few months. I added some top soil to the pot a week before leaving to replace some of the washed away dirt. The weather was a little dry these last few weeks, but there was some rain. Here are some pictures of the tree now. Any thoughts? Advice to save my poor tree? Should I prune off the dead looking branches? Thanks for any input!!

I use to have a Michelia Alba which is a tropical and cannot withstand any coldness at all, I don't know what's your temp in Miami when you have your Ylang ylang outdoors for 4 weeks, mine would have died with winter climate. I think yours still look ok, I would have cut off the brown stems/part since the main stem still looks pretty good, hopefully it's not damaged in the roots.

sherrytam, thanks for the input! It did get down to 50 a couple nights here, but generally we've got pretty warm weather (back to high 70s now). I'm inclined to chop off the dead looking stuff too, but figured I'd do better to check w/ the community here first.