I use vermicast, too. I mixed vermicast with baby worms and worm cocoons in it with my tomato and pepper seed starting mix. So far, every group-sown cup with at least one biggest seedling growing true leaves had one or more baby worm in it, and the baby worm was curled in the roots of the biggest seedling. In some cases, "vermed" seedlings had jumped ahead of "un-vermed" earlier sprouted seedlings.
I add more (and bigger worms) to the uppotting mix as well as UCG's (used coffee grounds). Greensand, rock phosphate, dolomitic lime.
My vermicomposting bins are not that big so I try to maximize and put the extra worms to work in situ
While the seedlings are inside, I water with UCG and used tealeaves steeped water, rinse water from finished beverage containers (milk, juice, soda, etc.), rice and beans rinsing water, soaking water. Vermicast tea. Diluted AACT (actively aerated compost tea) from vermicast.
I do use organic fertilizers when I have them or when I don't feel like mixing up stuff on my own but they usually smell bad so only very small amounts if added to soil mix. I also put small amounts in their water once they go outside to be hardened off. Also add compost from outside compost piles to the water, but usually don't get large quantities of AACT brewing until after temps get warm -- after first frost.
...maybe you should tell us exactly how you are growing your seedlings. It's hard to advise without knowing the details and what I do out of habit might not be obvious and not fully explained.