That's mostly just because Tokyo is so immense: Greater Tokyo is the world's largest city on most measures. However, the skyscrapers are concentrated in a couple of districts, and the vast majority of the city is indeed low-rise (under 10 stories or so even in the immediate city center, under 5 outside it).

Incidentally, most urban planners figure that Tokyo should be more dense instead of forcing people to endure multi-hour commutes, but strict sunlight laws make it very difficult to build tall buildings outside a few designated areas.