I quite sympathize with not wanting to spend a lot of money buying dirt. In general I am in the buy nothing, make / grow it all myself: soil, compost, wood chips, etc.
But when I was first starting out where I am, with terrible clay soil, I did build raised beds and I did have a truckload of good topsoil (2 cubic yards) delivered to fill the beds. It is WAY
cheaper to buy it in bulk like that, than in bags. Then I bought just a few bags of good stuff to mix in with it.
In your other thread about everything being stunted, you said: " I've tried adding some shovel fulls from a 20+ year old manure pile into it but I don't think its done much. With the compost pile started this year I'm really hopping to have my own compost for next year. I have very few worms in the garden but do have some ant hills.
For fertilizer I have some store bought 10-10-10 that I mix in at the start but I don't do much else throughout the year. "
A few shovels full is NOT going to make a difference. You need to be able to cover your whole garden plot several inches deep with compost and/or other good organic stuff and then mix it in. And you will have to keep doing that spring and fall for at least a couple years before you get your soil to the point where you are getting really good results.
And throwing a little bit of 10-10-10 on bad soil is not going to solve the problem, either.
Re: "I'm pulling weeds and tossing some of the cut grass into a 4' x 4' compost pile." PLEASE read the two composting basics stickies at the top of the Compost Forum. Just throwing grass and weeds and garden trimmings in a pile, is NOT the same as composting. They are "greens" (soft, moist, nitrogen rich). In order to compost successfully and end up with a product that is good for your soil, they need to be mixed with "browns" (hard, dry, carbon rich). Browns include fall leaves, straw, corn stalks (preferably chipped up), twigs, fine wood chips or sawdust, shredded paper etc. The stickies give a lot more info.
If you are not going to buy compost, then you are going to have to work very hard at composting everything you can get your hands on. All your kitchen scraps, left overs, waste etc. Everything paper that comes to your house (torn up junk mail, toilet paper tubes, etc). And go looking for more. Places that sell coffee will usually give you their used coffee grounds free. (They look brown, but they are a "green" nitrogen rich.) I collect the big yard waste bags of fall leaves that other people put at the curb for pick up and usually end up with at least a dozen of them to feed in to the compost pile through the season. You can buy a bale of straw at a feed store (cheap!) to gradually add to the pile. The volume of "brown" stuff should be at least equal to the volume of "green" stuff.
You want to have a LOT of compost, so you will have to work at it. You need to have one 4x4x4 compost pile filled now and settling through the fall and winter while you fill up the next one. Anyone that is seriously composting has at least two compost bins, often three.
Best Wishes! With a lot of work on the soil foundation of your garden, you can end up with a garden that is a delight, not a disappointment, way more productive and with less problems with diseases and pests.
And once you have invested all this work into building good soil, gardening will get a lot easier. You won't have to keep working so hard.
Keep updating us with how it is going for you!