It does look like mites. There are other kinds of mites besides spider mites. spider mites are easy to see since they are red and if you tap a leaf over white pepper you can see them moving around. Other mites are even smaller but cause similar damage. Broad mites and cyclamen mites will also attact peppers. I don't have a lot of problems with cyclamen mites but I have had broad mites before. I usually dip the plants, it gets better coverage than spraying. My peppers are mostly in pots. The ones in the ground just get cut to the ground and I just bag and toss the tops. It is easier to start fresh.
You have already used an oil. That can work, but dipping would be better.
Mites can be controlled with hot water (actually warm water 111 degrees. Use a candy thermometer. You will have to hold the plant under water for about 10-15 minutes so you need to rig something up. I use a 5 gallon bucket. I put newpaper on the top of the pot and tape it over the top of the pot so the soil won't spill. I turn the pepper pot over the 5 gallon bucket and I use a couple of boards to keep the plant from falling in the pot. It also helps to use a kitchen timer. 15 minutes can seem like you are waiting an eternity, especially if you are actually holding the pot over the bucket.
I don't use soaps on peppers, they make the leaves curl if you don't wash it off after 10 minutes.
Miticides work best, but not many are readily available to the home gardener. Using insecticides kill off predators as well and you end up with a rebound in the pest because the predators don't recover as easily as the pests.
https://entnemdept.ufl.edu/creatures/orn/broad_mite.htm