Establishing a Threshold for Application Inclusion in Peer Review Committee (PRC) Discussions
What threshold would capture the largest percentage of applications that were ultimately approved following discussion at peer review committee (PRC) meeting, while minimizing the percentage of applications that would not be funded without PRC discussion?
Applications initial ranking percentiles within PRCs prior to face-to-face meetings was compared to success of the application following the discussions.
For all applications above a given ranking percentile, the percentage of ultimately approved applications found within the group was calculated.
Ninety-eight percent of all applications that were ultimately approved following PRC discussion had an initial ranking of 60 or above, while two percent of applications that would have received funding if included in PRC discussion are lost.
Figure 1. Application Rankings before Peer Review Committee Meetings and Application Success in Ten OOGP Competitions (2010-2015)
Figure 1 long description
|Pre-Meeting Ranking Percentiles||If a threshold for inclusion in PRC discussions is set at a specific percentile:|
|% of Applications Ultimately Approved after PRC discussion that are included||% of Applications Ultimately Approved after PRC discussion that are lost|
The blue graph line represents the percentage of applications that were funded after PRC discussions that are not captured within a bracket from the given percentile to the top ranked application.
The analysis demonstrates that using a threshold for inclusion on pre-meeting ranking percentiles would have better focused PRC application discussion meetings on those applications with a higher likelihood of receiving funding.
For a breakdown by OOGP competition of the above analysis see the Competition Breakdowns page.
Note: A similar analysis was conducted to determine if there was a threshold above which applications always get funded and do not need to be discussed by PRCs; however, there was no ranking percentile above which 100% of the applications were consistently funded. For analysis please see below.
Is there a threshold on pre-meeting rankings above which applications always get funded and do not need to be discussed by PRCs?
Figure 2 long description
|Pre-Meeting Ranking Percentile||Percentage of Applications Ultimately Approved after PRC Discussion|
The green graph line represents the percentage of applications that received funding after PRC discussions ranked above an identified percentile.
- Date modified: