What I do depends on the plant and how I plan to use it--fresh? dried? cooked? or even...frozen?
Examples:
Italian flat-leaf parsley--I cut off the thicker end of the stems and chop the rest of the cutting (leaves and small stem) prior to saute-ing for a dish. I would probably dry the same parts. For fresh, non-cooked use, I prefer the texture of the leaves only.
Basil--I use the leaves only, no matter what: tomato dish, pesto whether for fresh use or frozen in ice-cube tray for later use, etc. I put the stems into my compost bin.
Rosemary--Leaves only, most definitely. The stems of my rosemary, which is now a landscape-sized plant, are much too woody for use as anything except maybe skewers. The stems join basil in the compost bin.
I hope my examples give you some ideas; there aren't any specific rules, and no culinary herbs that I'm familiar with have "do not eat" segments.
Enjoy, and bon appetit!
Cynthia H.
Sunset Zone 17, USDA Zone 9