The one near Sunnyvale Community Center is an apricot grove and you can buy their apricots in the early summer.
There's an orange grove here: https://maps.app.goo.gl/Tkycx4qdL8Wrq5zY6 near Intel HQ and the new set of SCUSD schools. The only thing from stopping you from picking an orange is a sign asking you not to.
The existence of citrus trees seems like a bit of a scourge to me at this point. Living in the Phoenix area -- indeed there was a wild real estate success with orange groves in our distant history -- there is a lot of ornamental citrus lining some streets. These fruits are, of course, inedible: very tough and bitter.
There are also citrus fruits in many, many yards of SFHs. And these are seemingly a scourge upon churches and other community centers. It seems that any Catholic who owns property is in the habit of harvesting their citrus and then toting huge bags to church, where they informally distribute it to all takers. Now you feel bad for not taking any, and there's always a ton left, and it's a common sight to see completely unlabeled bags of grapefruit or lemons with no hint of age or provenance. (Sometimes it's done right under the nose of St. Vincent de Paul, where they've got their own sources of good nutritious fresh fruit as well...)
The trouble is that unharvested citrus is likewise a scourge for the homeowner. The oranges can attract "roof rats" and other pests even while still on the branch. Falling to the ground, well you can imagine the mess and smell and pests that will come from that.
The homeowners often enjoy the shade and cachet enough to keep the trees around, though. I had another friend with fig trees, which was way more exciting when he offered them to us at church!
