I have lots of hackberry (Celtis occidentalis) growing around me, in my yards over the last thirty years, from seed and in old growth and transplants given to friends. Would love it if the fruits were ever that big. They are more in clusters and only 3-5 mm roundish, or less than a 1/4 inch. The venation is wrong, the base of the leaf is wrong (should be uneven), the habit is wrong... Now same species can sometimes have different looks in different climes, but I'm pretty sure this is not hackberry. All my native plant books and tree field guides are packed away so I can't look it up there.
And Applestar, the fruits are not a staining problem, but critters like them. They dry very quickly to a red-brown leathery skin over a yellow sweet pulp that thinly robes a hard seed. I like to walk on the seeds on concrete, like playing hopscotch, making one or two at a time crack or snap with each step. The fruits would probably persist past leaf drop for you in Jersey. They are not a significant littering problem.
I'll think about this interesting tree some more. Something may come to me, but it doesn't look familiar enough to send me in any search direction.