Your perfectly fine software(X11), lets windows spy on each other and read keystrokes to other windows. Any client can pretty much control the entire server.
This is a general problem in Linux. Non-root programs have too much power across the board. A more Android-like permission style solution would be much better.