"Compost" is a specific term that, at leaat in the U.S., describes the plant material that has been broken down by certain process, so I don't think vegetation that stand around through the winter to be weathered and basically dry rot can be called "compost" though they will eventually return to earth. (BTW bagged potting soil/mix is called "compost" in the U.K. and presumably other British English speaking countries.)
Sometimes, pest bugs overwinter in the stalk or in the ground under or near where the plant was grown. So if you are not composting in a pile, (chopping up and) using the plant material as mulch for a different plant that won't be bothered by the bug and planting the same plant elsewhere in the garden (crop rotation) is a better idea than to leave them standing. Also, close contact with the soil/ground promotes their breakdown because it provides access to all the ground detrivores/microbes and keeps the debris moist(er) which encourages and nourishes their activity.
That's why mechanically tilling in the plant debris and residue in place -- which chops up the material, especially stalks, disturbs the soil and exposes hibernating phases of pests, and incorporates the debris into the soil is successful. Tilling has its detriments and no-till methods have other advantages. ...I agree with Eric about benefits of leaving the roots undisturbed in the ground.
-- It's something to think about.
It depends on the plant and likely pests so it's also a good idea to learn about garden pests their life cycle in your area.