Transferring account ownership: security vulnerability?

I’m looking at using accounts located at PDAs to store state values related to a program. For example, I might want to store yes/no votes for a proposal using accounts at PDAs with the stakeholder Pubkey and proposal address as seeds. But I was thinking, if you can transfer ownership of an account to a program, what would stop someone from creating an account holding a “yes” vote, transferring the ownership of that account to the vote program using an actual stakeholder’s Pubkey, and repeating this until “yes” votes win?

Am I thinking about this correctly? Or is there some mechanism to prevent this kind of attack from happening?