So I learned the hard way this is peach leaf curl. it is a fungus & the tree is supposed to be sprayed in the dormant stage before the Spring for prevention. I was told to remove the affected leaves even from the ground. Though I read here https://www.ipm.ucdavis.edu/PMG/PESTNOTES/pn7426.html
that it doesn't really help. This link also gives all the info we need to manage & prevent for next year.
Seems it is too late in the season for us to do anything about this except wait & hope the fruits aren't too affected. Hope this helps!