I guess the answer is somewhat dependent on what you are expecting from your garden. You mentioned that you "started my flower garden" so my answer may not exactly match.
I, too, have health issues that can be seasonaly debilitating and can cause problems. But for me, it's a matter of getting stuff done when I feel able, and when I do, I can get many of the heavy, strenuous stuff done.
I have tended to forego flower beds in the strictest sense. I have clearly identified the growing conditions in different parts of my garden, and plant plants that are suited to those conditions. I have sunny, well drained beds, sunny heavy clay beds, swampy wet areas, deep shade areas, etc.
By matching plants to their optimum growing conditions, you take a lot of effort out of maintaining them, except to trim when they become overgrown. I mostly don't plant flowers that need deadheading.
My garden has suffered the transition. My roses have died one by one -- fussy hybrid teas were the first to go -- until I now have only one David Austin English rose that bloom and sheds petals and grows hips without any help from me. But seeing those apricot colored blooms gives me a lot of pleasure, even if they only appear here and there, and not cover the canes.
I have put more effort into planting native plants, with emphasis on multi-season multi-purpose interest. Most provide fragrant or nectering butterflies/hummingbirds/insect flowers in season, fruits or seeds for wild birds, and/or colorful foliage in the fall. By attracting wildlife, the garden provides entertainment even when I can't be out there to tend to them.
I enjoy planting edibles for humans too. Annual vegetables are not difficult if you can get them planted at the right time. Even at worst with weather not cooperating and with health issues, I feel rewarded if I can get out there and harvest some things for the table, even if the amount or quality is not the very best they could have been had they been given full care. Forgotten or lost squash among the weeds, ripe tomato among the spoiled ones are like buried treasure.
I am slowly adding perennial vegetables to the mix.
I have planted a lot of fruit trees and berries as well -- only buying highly disease resistant varieties and planting them in their optimum locations to avoid need for heavy-handed care. Sometimes, I don't get around to protecting the harvest and the wildlife gets most or all. But usually I have some for the family to enjoy, and sometimes, I can do everything that is needed and have fantastic harvest.
Oh, the newer fruit trees are all pruned in espalier style to keep them within reach, and brambles are trained against a fence and wall to keep them within bounds.
Finally, if you haven't switched to organic growing methods, I highly recommend that you do. I really feel my garden is protected by the diverse living wildlife and microbe biosphere/ecosystem. There are plenty of predatorial insects, etc. that form my "Garden Patrol". I make use of my own compost -- which is not tended that often at all -- and make AACT (actively aerated compost tea) which cultures the beneficial microbes for the soil and for the phytosphere foodweb. That one remaining rose is treated to milk solution spray during the worst seasonal humidity as preventive against fungal disease.
Moreover, I strongly feel, though this is my opinion based on personal experience, that the chemicals that are used in the garden can exacerbate ill health. Avoidance of many chemicals commonly used outside and inside the home could make a difference, and it's something to be tried if you have not done so already.