On Windows 10 (don't know about earlier versions), all you have to do is click on the image file in Windows Explorer and then edit the file details. The updated caption is written directly to the file.
I can't find anything this simple on Mac OS X. (I can view a caption with the Finder and with Preview, but can't change it.) I can add metadata in the Photos app, but there seems to be no way to sync it back to the original file. (I assume the new file written when you Export has the metadata, but that doesn't do what I want.)
I have my own app in the Mac App Store that can do this (ExifChanger), but I'm surprised that there's no way built into OS X to do it.
Am I missing something?