I don't think it makes any difference to the health of the plants, really. What you do want to remove is any flowers. Once they flower, the leaves will die back. At least, that's what happened to mine.
If you cut them, cut them a few inches above the ground so they can grow more leaves. I've been harvesting mine like that all summer and they keep growing back like crazy.