I have propagated and grown herbs under lights indoors and in a midwest greenhouse for many years. First.....the soil should be 5.6 to 6.5 ph as a general rule.
Overwatering is the big killer AND short daylight. A 10 to 12 hour light schedule is best....that may be why your herbs started to do better in the spring when the days become longer......
The bagged soilless medias work wonderfully well....Jiffy Mix or the Miracle Gro brands should all be ok.
Trimming overgrown herbs is a must to keep them going.
Some herb seeds benefit from soaking and / or exposing to light. (I have a chart I created for most common herbs in this regard and can try to reproduce it here if you like)
All this should be considered and don't get hung up on temps too much. 60 to 80 degrees will handle most things. 40's and 50's not so much. A night or two down there is ok, but not as a daytime temp.
Watch for insects......Organic pesticides ONLY!
Go forth and multiply!