Exactly as you said, it's a decent approximation and with large tables it works well enough, but if there are clusters (long chains) in a bucket those elements have a smaller percentage of chances to be picked.
However there is a trick to improve this that I don't use, that is, instead of searching for a non empty bucket, and select a random element from the chain, it is possible to do this, choosing a small M (like M=3):
* Find M non-empty buckets.
* Pick the bucket among the M buckets, with a chance proportional to the chain length at every bucket.
* Finally pick a random element from the bucket.
The bigger M, the more accurate the algorithm becomes.
For instance in the pathological case you shown where a bucket as one element and another all the rest, this would find the two buckets and then pick the 1 element bucket with a much smaller probability that would adjust the difference in chain length.
However there is a trick to improve this that I don't use, that is, instead of searching for a non empty bucket, and select a random element from the chain, it is possible to do this, choosing a small M (like M=3):
* Find M non-empty buckets.
* Pick the bucket among the M buckets, with a chance proportional to the chain length at every bucket.
* Finally pick a random element from the bucket.
The bigger M, the more accurate the algorithm becomes.
For instance in the pathological case you shown where a bucket as one element and another all the rest, this would find the two buckets and then pick the 1 element bucket with a much smaller probability that would adjust the difference in chain length.