Genetics would play a part one would presume like almost everything else we inherit in our bodies or conditions/diseases we have that have been genetic such as types of diabetes, epilepsy, autistic traits, cancers etc etc etc all are in the family line and is why the doctors always ask about such ''has any men in the family died of a heart attack''.
Like Still said the male gets most (obviously not all) of his genetic material from the mother and the mother will have got it from her father, the sons grandfather. The grandfather got it from his mother who is the sons great grandmother and you can go on and on. Its interesting because if you think about it it really pans out into branches of the family you never would have thought you could have really had such a close attachment with, if that makes sense to you all and they say (Geneticists) that you will look very much like at least ONE of your ancestors from the past.
Daughters are the opposite. You may notice that sons sometimes look allot like the mother and the daughter allot like the father. It can be opposite but many times you see that occurring. So assuming Mr X has a big natural cock, one could hypothesise that his grandfather on the mothers side potentially may have had a larger penis and his grandfather on the mothers side (great grandfather for Mr X) may also have this genetic trait.
So if a girl brags about her man having a large fat cock than mention to her to thank the guys grandfather on the mothers side, he's likely to have passed on the genes. I've also read information by various human geneticists on evolution that one can actually tell if a girl will have large breasts! look at the grandmother on the fathers side and if she has big tits than chances are high but not guaranteed she will as-well.