Wow that's a lot of pots to undo.
But yes, better to fix it now, especially with peppers which are very susceptible to root rot due to soggy soil that doesn't drain well. You can remove the plant, dump the contents of the pot and thoroughly mix in 1/4-1/3 perlite or mixture of sand/gravel or pumice gravel. (Perlite will keep the mix less heavy as in weight to pick up and move the container, but the sand/gravel would weigh them down and help from falling over if the plant grows big and tends to become top-heavy).
Depending on how dense the potting soil is, you could also mix 1/3 shredded coir (or peat moss if you have to, but if you do, then you'll need to add a little lime to balance the pH).