When gardening, it’s true that you absolutely need to know the approximate first and last average frost dates, and they are not directly related to Hardiness Zones, but Hardiness Zone is also relevant for other reasons.
Hardiness Zone is exactly what the name implies — it indicates “winter hardiness” and is intended to differentiate which plants tend to be able to survive the winter. There are annual differences and borderline plants need/can be helped by adding a little winter protection — mulching, covering, etc. it also helps to know because plants that will become rampant weeds could be planted where they are not winter hardy to enjoy during the growing season and be merely winter-killed. It is also a good idea to buy locally adapted nursery plants.
There are food crops that are considered fall-winter season in warmer winter areas that simply cannot be grown where the winter low temperatures get too low, but some clever people have figured out that it’s possible to grow them double-protected by covering with fleece under high tunnel.
I tend to think in terms of winter survival from the freeze, but there are also plants that NEED certain amounts of cold dormancy — so there are plants that will not survive or fruit where the Zone is higher/hotter than its Hardiness Zone range. e.g. Sugar Maples grow well 25 miles north of here but not here (same for White Paper Birch).
Knowing the local weather patterns is important as stated. Because the zones are marked in broad sweeps across the map, it’s also important to know the local mini-climate surrounding your garden — urban heat pockets due to hard scape and population density, slope of the land, forested vs unforested, waterways, even underlying geological influences can create local differences — this is also true of frost dates. e.g. My MIL lives 23 miles north of here, but her last spring frost is 2 weeks earlier, and plants that won’t survive the winter in my garden without extra protection can survive in her garden.
My area is close enough to the Pine Barrens where the deep sand actually creates effects that lower the local temperatur in a vast enough pocket to affect the weather and climate, so even though my own soil is clay, my garden is affected by the general Pine Barren’s weather effect. I have also noticed that the weather systems which typically move from southwest to northeast are affected at the Delaware River — so we don’t usually get the deep snow winter cover that could make the critical difference in overwintering survival of some plants (or extra rain in the summer for that matter).