As a hypothetical question, it's more then a little far-fetched. (This kind of thing happen a lot in your neighborhood?) I will play along just the same.
Now If a complete stranger had done this out of the blue, I would have to assume that he may be drunk, mentally deranged, or simply has me mistaken for someone else.
I could retaliate physically, but there is the possability he has a concelled weapon. In that case I would be at a distinct disadvantage being outgunned and having to defend myself as well as my family. Even if succesful in subduing him, there is a good chance we both end up in jail while the cops sort it out.
I could retaliate verbally, but the odds are good that the verbal confrontation would quickly escalate to the physical altercation discussed above.
I could walk away thereby demonstrationg to my wife my ability to keep my emotions under control. (a quality she greatly admires) as well as showing my child how to behave in a civilized sociaty even though not everyone else does.
Why would I justify an act so stupid with an equally stupid response?
Never argue with an idiot. Bystanders may not be able to distinguish between the two of you.