you're right that it's not going to work out exactly equal to the brightness of the sun. another way to see this is to imagine an alien who lives on a (undiscovered) planet incredibly close to the sun - the sun would appear much brighter to the alien who would, presumably, expect a correspondingly brighter sky everywhere else (which clearly makes no sense since looking in other directions the alien should agree with us).
but the effect of variations is most important when you have fewest stars (so the largest variation should be the sun, which seems reasonable). at large distances, with many stars, things will even out despite the fluctuations (for physically reasonable fluctuations).
to make the argument rigorous, you need to work out how bright things would be "on average".
another (more convincing?) way to understand the problem is to see that if we have an infinite, static universe, full of stars, all burning away, it should get hotter and hotter over time... (and if it's infinitely old, should be infinitely hot by now!). clearly something is wrong with that model.
but the effect of variations is most important when you have fewest stars (so the largest variation should be the sun, which seems reasonable). at large distances, with many stars, things will even out despite the fluctuations (for physically reasonable fluctuations).
to make the argument rigorous, you need to work out how bright things would be "on average".
another (more convincing?) way to understand the problem is to see that if we have an infinite, static universe, full of stars, all burning away, it should get hotter and hotter over time... (and if it's infinitely old, should be infinitely hot by now!). clearly something is wrong with that model.