Chad,
I've had this experience when I first attempted to start some violas from seed. I would guess as Val did that you have some damping off problems. I've found that watering my seed trays from the bottom up helped prevent the problem from starting. I also used a small house fan and turned it on low to let some good air circulation flow over the seedlings, that really seems to help (and I think it strenghtens their stems, but that could just be in my head

).
Also...can I ask what kind of soil medium you're using? I too came from a very small town back in Michigan and getting the Vermiculite seed starting mix was difficult back then, but I found that any other kind of soil simply killed off every seedling I had. If you lose a lot of seedlings, I'd start over from scratch by emptying the seed trays, disinfecting them with a 1:10 bleach/water solution (afterwards giving them a good clean water rinse and letting them fully dry), then getting the right kind of seed starting soil mix with the right amount of light. I remember one year starting over 6 times until I got it right!
Hope this helps, and good luck!
-Paul-