Windows Explorer has settings that display thumbnail pictures, but that setting doesn't change the application settings elsewhere in Windows Explorer.
I don't really know whether Windows will support the use of thumbnails for other images, when they're embedded. Maybe cross-referencing how the .jpg file format is registered in Windows Explorer, will help figure out what the other formats will support.