hopefully someone more experienced can offer more clues but without a picture it's hard to tell what condition it's really in, what kind of soil, etc. I see you are from SW virginia. If yours isn't flowering all the times, it's a sure sign that it's not healthy. It should have tiny white flowers year round, they last a day or 2 before withering.
ficus, maple, elm, juniper, pine