Unfortunately I have never done well with chamomile either, so mainly I can tell you what doesn't work
. What doesn't work is too much shade. I have an herb garden that gets some morning sun, but very little afternoon. Not ideal, but you have to work with what you've got. Some things do fine there (mint, oregano, lemon balm), some things hang in (thyme, sage) and some things keep dying out of it (tarragon, chamomile)
I would say the chamomile needs full sun!