In the Dark

Quite a few times on this blog I have acknowledged the tremendous amount of luck I have had all the way through my career, not least that the opportunity which led to my current position in Maynooth came up when exactly when it did. Another thing that has played a role has been privilege, defined not only in terms of race and social class but also educational and institutional background. Those of us who have benefitted from this are often blind to its influence, preferring to think we achieve things purely on merit.

This morning I read a piece by Izzy Jayasinghe that articulates similar thoughts from the point of view of the author’s own personal experiences. It’s a piece that’s very well worth reading and puts things better than I’ve ever managed to do.

The main point of this post is to draw attention to Izzy’s article, but I thought I’d take the opportunity to pass on links to another couple of pieces I have mentioned on this blog over the years.

The first is a paper on the arXiv by Brian Skinner, which has the abstract:

One of the major benefits of belonging to a prestigious group is that it affects the way you are viewed by others. Here I use a simple mathematical model to explore the implications of this “prestige bias” when candidates undergo repeated rounds of evaluation. In the model, candidates who are evaluated most highly are admitted to a “prestige class”, and their membership biases future rounds of evaluation in their favor. I use the language of Bayesian inference to describe this bias, and show that it can lead to a runaway effect in which the weight given to the prior expectation associated with a candidate’s class becomes stronger with each round. Most dramatically, the strength of the prestige bias after many rounds undergoes a first-order transition as a function of the precision of the examination on which the evaluation is based.

arXiv: 1910.05813

You can read the full paper here. The author acknowledges the role that blind luck played in his own career but also develops a simple mathematical model of prestige bias. It’s an interesting paper, well worth a read.

Luck plays a definite role in winning grant funding. Having been on grants panels I’m away that many very good proposals are not funded. A scoring system is generally used that introduces some level of objectivity into the process, but the fact is that a lot of proposals come out with similar scores and the ranking of these is a bit arbitrary. A slightly different panel would produce slightly different scores, but perhaps a large difference in ranking would result.

This is one of the issues discussed in a paper on the arXiv (by Pluchino et al) with the title Talent vs Luck: the role of randomness in success and failure that discusses the role of good fortune in scientific careers. This is the abstract:

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking. Sometimes, we are willing to admit that a certain degree of luck could also play a role in achieving significant material success. But, as a matter of fact, it is rather common to underestimate the importance of external forces in individual successful stories. It is very well known that intelligence or talent exhibit a Gaussian distribution among the population, whereas the distribution of wealth – considered a proxy of success – follows typically a power law (Pareto law). Such a discrepancy between a Normal distribution of inputs, with a typical scale, and the scale invariant distribution of outputs, suggests that some hidden ingredient is at work behind the scenes. In this paper, with the help of a very simple agent-based model, we suggest that such an ingredient is just randomness. In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result – although implicitly suggested between the lines in a vast literature – is quantified here for the first time. It sheds new light on the effectiveness of assessing merit on the basis of the reached level of success and underlines the risks of distributing excessive honors or resources to people who, at the end of the day, could have been simply luckier than others. With the help of this model, several policy hypotheses are also addressed and compared to show the most efficient strategies for public funding of research in order to improve meritocracy, diversity and innovation.

arXiv: 1802.07068

Postscript: I remember a conversation I once had with Lev Kofman – a far more significant scientist than me – during which he called me a “fucking lucky bastard” because of some guesswork that led to a result in a paper of mine that turned out to be right. For a moment I thought he was being abusive but then, with a smile, he added “Welcome to (the)* Club”.

*Lev, like many Russians, never really got the hang of articles; the definite article in parentheses is my addition.

P.S. My good fortune in surviving academia, of course, pales into insignificance when compared to this.