Oh, theres actually these really useful program that does exactly that, and has similar functions for all of that. It's called Czkawka, I use it all the time. It can compare by thumbnail simularity, and hashes to see if scenes are dupes or not. Here's a link:
You must be registered to see the links
Theres also Anti-twin, which is a similar but much older program that does the same thing. This one has the ability to compare all files with a simularity from 60%-100%, which Czkawka doesn't have as of now. But, anti-twin doesn't have multithreading, so its pretty slow.
I've used both of these, and they help a LOT. Although it isnt going to do the whole job because some people upload several duplicate scenes with different camera angles, which would be considered different. But at the very least, it certainly helps.
I havn't used this program for character card comparing, only scenes, but i'd imagine it still works for that too..