It sounds like your compost could be too wet. This attracts fungus gnats. Not a good thing to have flying around your kitchen.
Make sure that when you water you tip any excess water out of the saucers/trays under the plants. This will be better for your herbs as they dislike sitting in wet conditions and should also get rid of the gnats.
You mentioned that the window gets full sun? This is not the best position for any plant to grow other than a cactus. The sun will be magnified through the window and will scorch them. Better to move to a window that is East or West facing or further back from the window they are in front of now. That way they will get plenty of light but not be cooked!