RBG's comment about the tree being inappropriate for the space was not at all extreme. I am a licensed landscape contractor. I had an active landscaping business for 10 years and still occasionally do consulting work.
The most frequent mistake made by DIY landscapers is planting shrubs and trees that are inappropriate for the SPACE.
Jacksonville has a very long growing season. Your plants will quickly outgrow the space you have allowed. In 3 to 5 years you will be beating yourself up trying to keep your plants pruned.
Even if you manage to keep the overall size of your tree under control you can not control the root system. In a few years you will see damage to your fence and foundation from the roots.
Mature size is a critical factor in plant placement - especially trees. Even a dwarf avocado can have a mature spread of 12 feet. So figure a radius of 6' + 1/3 the radius. The tree should be planted no closer that 8' from any structure . I don't think your tree is a dwarf variety.
David - when members of the forum respond to a question they base their response on experience, both personal and professional, education and research. The responses you receive are all well meant and are never intended to sound critical. Frequently there will be varied opinions based on varied experiences. In your case the advice is fairly unanimous.
Sorry if you got the wrong impression.