Yarrow
Believe it or not, in many temperate climates yarrow can become naturalized and survive in fields on it's own. I never plant any perennial inside the garden since the garden is for annuals and I do not want to rotate around them. Some of the plants I listed are not as weedy as others so you can take your chances if you only have a perennial garden. It depends on your climate.