But what Francis was talking about:
The flowers that do start to form immediately turn yellow and fall off.
doesn't sound like normal blooming and fading, it sounds like bud drop. Right, Francis? The flowers drop before they open up? Bud drop is common in gardenias:
"The most irritating problem encountered with gardenias is "bud drop,"when flower buds abort just before blooming. Common causes include low humidity, over-watering, under-watering, insufficient light high temperatures, rapid temperature fluctuations, cold drafts or change in plant locations. In other words, gardenias are temperamental!
Bud drop, the abnormal dropping of buds, occurs during periods of high night temperatures or during periods of low light intensity. Some bud drop is a natural condition. Every effort should be made to keep the soil uniformly moist, but not wet, during flowering."
https://www.uri.edu/ce/factsheets/sheets/gardenia.html
That same article mentions several fungal leaf spot diseases of gardenias including this one:
Rhizoctonia Leaf Spot (fungus--Rhizoctonia spp.)
Leaves infected with this fungal leaf spot disease have tan to brown spots up to 1/4 inch in diameter. Spots are circular and zoned. The disease begins on the older leaves and spreads upward when the plants are watered excessively or when air circulates poorly because of overcrowding. Diseased leaves should be destroyed and sterilized soil should be used. Disease-free plants should be used for propagation. Avoid wetting foliage when watering.
Your bottom picture the spot clearly looks "zoned."
I would treat it like a fungal infection.