That's wise :)
You should check your "reports" section occasionally to see if anyone has reported a post, then consider: 1. Was this post made with intent to worsen the quality of the community? and 2. Does this post fit into the topic of the community? If you answer no to the second question, you should remove it, if you answer no to the first question, you should remove it and ban the user. One of the few things I like about the upvote/downvote system is it prevents the power of flooding and spamming, and makes communities almost self-moderating.
I currently moderate 10 communities (though I will give most of them away once this website becomes popular: not trying to be a powermod). It doesn't take much effort, and you can always add other mods to split the workload.
Lust for gold? Power? Or were you just born with a heart full of neutrality?
ubunut