Even if you pick out the outlier, a very, very well-mannered, intelligent person, your children are going to get genes that are closer to the average of the population / race, than the genetic-lottery-winning outlier

That's a misleading thing to say. First of all, how much the children will regress to the mean depends on the trait. For IQ with 0.6-0.8 heritability it'll be closer to the parents.

You'd also get a very poor estimation for a given couple if you use a race average. Family average would be much better due to assortative mating.

In practice I'd be very wary of high variance within the family, especially if you don't plan on having a lot of kids.