I used to think that, too. Then I actually tried Vim. I was wrong. Vim is significantly faster for text editing.

Go check out some screencasts from experienced Vimmers or Emacsers.