how much does it really take over the garden?
phlox divaricata
I found some P. divaricata growing in a very tough terrain last spring. I've moved it to a shady spot with amended soil and so far it's doing fine, and has even stayed green all winter. Your question kind of threw me for a loop because I've never thought of it as invasive. A web search doesn't indicate it's a rampant thug like loosestrife or some primrose species, and I'd think in zone 4, it's not going to spread too rapidly becuase it'll get nipped back each year.
I think it's a very pretty plant, and I'm hoping mine will spread some to fill out some areas :)
Mine's taken three years to form a clump that's maybe a foot round. I don't consider it invasive myself. In fact, I wish it would spread more.