Lavender grows very long roots, like rosemary, so ultimately, it would be happiest planted in the ground. What species of lavender do you have? English lavender (L. angustifolia) is cold hardy, whereas French and Spanish lavender can be susceptible to cold, though that may not be so much of an issue for you in Maryland (USDA Zone 7?)
They like limey sandy/gravely poor soil in FULL SUN, and after some experimentation, my best lavender planting location has turned out to be what I call my "Desert Rock Garden" where I laid 4 layers of newsprint shipping paper on top of the lawn to block the grass (I have practically solid underlying clay soil), then piled 2 parts sand, 1 part topsoil, and 1 part compost on top. After planting, the entire area was mulched with medium and tiny white marble chips. Used unwashed, the marble chips washed off a lot of marble/lime dust into the bed. I have Lavender Hidcote (a short variety of English lavender), Provence (French) and xGrosso (hybrid). Provence and xGrosso were planted snuggled under/between the shelter of big rocks for extra protection in my Zone 6b garden. Spanish lavender is too tender to grow outdoors for me, and I usually have a lot of trouble growing lavender indoors.
I think what you are describing are new growth that are not getting enough light, plus without the weathering effects of wind and elements, remain weak and spindly.
I'll try to locate the thread -- someone with long-time lavender growing experience once posted lavender instructions for pruning twice a year, in addition to cutting the flower stalks. Outdoors in springtime, I trim the winter damaged shoots before new growth begins. When trimming/pruning any plant, it's a good idea to take no more than 1/3 of growth.
I just realized you posted this in the Container Gardening Forum (I thought it was in Herb Gardening). I guess my suggestions are not very applicable. People do grow lavender in containers, and I hope someone else can give you adivice that is more to the point.