One common type of bad answer is an answer that would really be more appropriate as a comment. If you flag an answer like that, I think it's possible for a moderator to convert it to a comment.
Of course, it's sometimes necessary to leave a comment as an answer, e.g. if the comment is much too long to fit in the comment space. Whenever I leave a comment as an answer, I preface it with a disclaimer to that effect. (See here for an example.)
Other kinds of bad answers should just be downvoted. One note: it's polite (but not required) to leave a comment whenever you downvote, indicating the nature of your objection, and perhaps a suggestion for how the answer might be improved. I think this is especially important at this early phase, when the definition of a "good answer" is still being determined. If someone else has already left a comment that summarizes the problems with the answer, you could just upvote the comment to indicate that you agree.
I agree that good answers should have justification and cite specific research or examples. I also agree that an objective tone helps, but I don't think an answer with a strong tone is necessarily bad. It's ok to feel strongly about things from time to time, and it's ok to make passionate, persuasive arguments as long as the discourse remains civil.
One more comment about downvotes: when the first post by a new user is of poor quality, it's not very welcoming to downvote them to oblivion with no comment. Wikipedia has a wonderful policy called Please do not bite the newcomers, and I think the same idea applies to this site. (See also Could we please be a bit nicer to new users? on meta stackoverflow). If a new user (easily recognized by their low score) makes a post of poor quality, the correct course of action is to first welcome them to the site, and then explain why their answer doesn't fit very well into this site's conception of a good answer. Many websites (e.g. reddit) are perfectly happy with users posting simple assertions of opinion, and new users here might not realize the distinction.