The best part of the fediverse is that anyone can run their own server. The downside of this is that anyone can easily create hordes of fake accounts, as I will now demonstrate.
Fighting fake accounts is hard and most implementations do not currently have an effective way of filtering out fake accounts. I’m sure that the developers will step in if this becomes a bigger problem. Until then, remember that votes are just a number.
This could become a problem on posts only relevant on one server, like community voting or updates on server issues. Such content would become very hidden.
Obviously, on the server the posts are from, you display the full vote count. There, the admins know the accounts, can vet them, etc.
This would be rather to detect and alert admin of a bad actors (instances) and then admin can kick it off from federation same for other tupe of offences.