I pick mine off. But I've found that you have to do it with care, and pinch off the whole shoot that has the buds, right down to the stem, otherwise you get funky subsequent growth.
It gives the plant some time to develop better roots & foliage. The years that I've pinched produced better than the years that I didn't.
Had a soil test lately? Peppers need magnesium to do well. And neutral soil, so they can access calcium. The conventional spacing is 12-18 inches, but I bump mine out to 2 feet apart or more.
I've also found that mulching them will actually have a negative effect. They really need as much heat as possible, and straw or leaf mulch cools the root zone. Black plastic keeps the root zone toasty, but then you will have to irrigate in some fashion.