18 January 2013

Grant application result from incomplete data

Results for financing applications often arrive in two steps:
  1. One's personal score (confidential) and some overall statistical information (public).
  2. Final decision (accepted/rejected).
For projects close to the (unknown) cutoff value, the delay between 1. and 2. can be quite long; it is imposed by the negotiation process with those higher up on the list.

A real-life example I was confronted to are the Marie Curie Fellowships: There were 3300 valid applications for the 2011 Intra-European Fellowships (IEF), out of which 800 fell below a threshold T=70%. Knowing that about 600 projects will be financed and that your score is S (say, 95%), will you make the cut ?

Let us first search for the most probable distribution \( P(x) \) obeying the constraints:
  • \( P(x) \) is defined on (0,1) : \( \int _0 ^{1} P(x) \, \text{d} x = 1 \)
  • \(\int _0 ^{T} P(x) \, \text{d} x = F_1 = 800/3300 = 0.2424 \)
The problem is easily solved yielding a piecewise constant \( P(x) \), with \(P(x\leq T) = F_1/T \) and \( P(x \geq T) = (1-F_1)/(1-T)\) . Defining the success rate above the threshold \( M = 600/(3300 - 800)\), we conclude:
  • Your application is successful if \( \frac{1-S}{1-T} \leq M\). For the numerical values above, this corresponds to \( S \geq S_{\text{min}} = 94.5 \).
  • The number of applications below the threshold is in fact irrelevant. 
We limited the reasoning to one distribution (the most probable), for which there is a unique answer. However, we understand intuitively that the uncertainty is very large; it would be much more useful  to determine completely the success probability \(g(S)\). Only for \(g(S)\) significantly different from \(1/2\) can one draw any conclusion from such limited information.

No comments:

Post a Comment