They don't have to be in silicon valley. The only reason for Google to be anywhere is employees and to a lesser extent investors, lawyers and other service providers of the cluster. It's mostly employees though.
Why are they there, paying high salaries if they could be getting the same for less elsewhere?
Or, what might they be getting (or think they're getting) that they wouldn't (think that they would) be able to get elsewhere?
Maybe they're concerned about lead times for hiring new people (stealing someone from a competitor vs all new employees needing to move first)?
Supposedly one of the reasons that Silicon Valley did so well, is that California doesn't allow non-compete agreements. Perhaps there's actually some benefit to exchanging employees with your competitors more often? (But then, what about the no-poaching agreements that companies were getting sued over recently?)
Maybe they're concerned that being the only shop in town would make managers more reluctant to fire people who need it (does corporate location preference correlate to corporate culture regarding expected length of service)?
Maybe having offices in less expensive locations hurts the company image (for customers, or potential employees, or both) enough that the extra salary cost is worth it?
There's a benefit to everyone in the industry for exchanging employees with your competitors often, but there's a cost to your company. Hence, geographical areas where that happens do well overall, but the individual companies involved hate it and want it to stop. It's very much a prisoner's dilemma situation.
Similarly, technological developments that benefit everyone in the industry are good for the industry as a whole, but bad for each individual firm in the industry, which now face increased competition. That's why big companies often try to spread FUD around "open" alternatives to their core business.
They could if they were willing to leave the CS chauvinism behind. There are plenty of non-CS engineers, mathematicians, scientists and even non-STEM liberal arts majors who are not just capable, but competent or very competent computer programmers and software developers.
Granted the concentration is likely to be higher in the Valley, but it's not that much higher. Then there is the argument that "if you build it, they will come."
Why are they there, paying high salaries if they could be getting the same for less elsewhere?