Without a downvote button, offensive or mean comments would easily fall under the "B" option and get flagged (and I believe they should now anyway even with the downvote button).

Obviously wrong or stupid comments would probably sink to the bottom pretty quickly anyway if people were upvoting the stuff that should be upvoted.