I have packs of 100 balls that should all be white. Occasionally a randomly distributed black ball appears in the population and I want to be sure...

Suppose the probability of a randomly-selected ball being black is P.

We want it to be so that after 100 samples, the probability that no more than 1 of those samples is black is less than 5%.

So, what is the probability of having more than 1 be black? It's easier, actually, to ask for the probabilities of 0 black balls and 1 black ball, and then add those up and subtract from 1.

The probability of having 0 black balls in 100 samples is (where n C k is the combination function, the number of possible ways to arrange k things from a sample of n things):

(100 C 0) * (P)^0 * (1-P)^100
1 * 1 * (1-P)^100
(1-P)^100

The probability of having exactly 1 black ball in 100 samples is:
(100 C 1) * (P)^1 * (1-P)^99
100 * P * (1-P)^99

These two things must add up to more than 95%:

(1-P)^100 + 100 * P * (1-P)^99 > 0.95
(1-P)^99 * (100 P + 1 - P) > 0.95
(1-P)^99 * (1 + 99 P) < 0.95

Because we have both a linear and an exponential term for P, this equation does not have an analytic solution. But all hope is not lost!

We can solve it numerically; by finding the intersection on a graph I estimate the needed P to be about 0.0035.

Let's try it:

(1-0.0035)^99 * (1 + 99*0.0035) >? 0.95
0.7067 * 1.3465 >? 0.95
0.95157 > 0.95

Sure enough, that works.

If you want a more precise solution, you could try some nearby values such as 0.00351 and see how high you can get P before it tips over and actually drops below 95%. But since they asked in terms of "balls per thousand", 0.0035 seems to be precise enough; that's 3.5 balls per thousand.

Round down to 3 and we're all set: If we ensure there are no more than 3 black balls per thousand total balls, we can insure that there is a 95% chance of each 100-ball set containing no more than 1 black ball.