You are basically right that it will take a few years of working on it, before it approaches really good garden soil. That doesn't mean you can't plant anything in the meantime. It means what you plant won't thrive as well and you are more limited in what you can plant (carrots for example are OUT).
(Used) Coffee grounds are a good addition to your soil, but it is a myth that it will acidify it. Most of the acidity goes in to the coffee when it is brewed. To start with add all the organic matter you can find. You can buy some bagged compost. It is probably the wrong season now to find fall leaves, but in the fall collect all the leaves you can (I drive around and snatch up the bags of leaves people put at the curb). They are really good for your compost pile or to add directly to the soil. Grass clippings, pulled and chopped weeds (as long as they haven't set seed), peat moss, aged composted manure, shredded paper, straw, all are "organic matter" for your soil. Pile it on several inches thick (or more or less as thick as you can), till it in. Wait 2-3 weeks until all the weeds sprout and then repeat the whole process. Then you can plant in it, probably still in time to plant stuff for a fall crop. Throw in some of the worms from your worm bin in to the soil, to help break down all that OM you added.
The peat moss helps to acidify soil, so is a good addition. If your soil is highly alkaline, you probably need to add some sulfur with the OM.
This is someone else who just wrote in about their clay soil:
https://www.helpfulgardener.com/forum/vi ... =4&t=63913