The reason you've heard about the "loose" soil is because lavender is pretty picky about having too much water at the roots. The idea is to have a soil that drains well (hence, the "loose"), so the water doesn't sit in the soil and rot the roots (especially true since you mentioned a lot of clay in the soil).
I definitely would amend the soil before you try lavender. You need to break up the clay clumps, which hold water and also is very hard for roots to move through in order to get the nutrition the plant needs. What you're going for is a balance between what clay does provide (water retention so you're not watering 18 times a day!) and water drainage (so the roots don't get waterlogged. Some plants are a little easier to please in this regard than lavender.
If you live in a city and have access to the larger parks, most municipalities have free compost and mulch piles for residents. You can bag your own and amend (compost) and protect (mulch) your plants for very little money. Or, your local nursery will also be able to help you with things to help your soil. If you live in an area with a high clay content, believe me, the nurseries around are used to helping gardeners with this problem!
If you decide to try container lavender, start out with a variety that's more apt to do well, like Spanish lavender.
Hope this helps,