Reply to Barry Allen: 'National Health and Medical Research Council grant funding: can the process be improved to achieve its objectives?'

Thank you for this opportunity to comment on the thoughtful article by Professor Barry Allen.

As the national agency responsible for supporting health and medical research, National Health and Medical Research Council (NHMRC) processes will always be of intense interest to members of the research sector. NHMRC processes have improved over time, in part due to the constructive feedback provided by members of the sector. All systems of peer review will have critics, and this is especially true when success rates for the NHMRC’s research funding schemes are around 20-25%. 

NHMRC recognises four categories of health and medical research - biomedical research, clinical research, public health research and health services research. Each is important to NHMRC. The main criterion for NHMRC funding is excellence, but in all research categories there is also a strong emphasis on outcomes and relevance to improving health. NHMRC review panels have a combination of clinical and biomedical members to ensure that the relevance of improving health is always recognised.

Clinical research is well served by NHMRC. During the period 2000-2009 NHMRC expenditure on ‘clinical medicine and science’ exceeded $1.1 billion. In addition, NHMRC recently announced a further $107m funding for 190 projects in ‘clinical medicine and science’ commencing in 2010.  Numerous examples exist of outstanding clinical research funded via the project grant and program grant funding schemes. 

It is important to remember that NHMRC organises and administers the peer review process, but applications are reviewed by peers, both external and internal to the grant review panels. In 2009, project grant applications were reviewed by one of 45 review panels. Readers of Cancer Forum may be interested to know that specific panels were convened to assess applications in the following categories: cancer biology; cancer biology - signalling; clinical cancer; and haematology and tumor immunology.

In addition, panels were convened to consider applications in related areas such as large scale clinical trials, public health, pharmacology, endocrinology and health services/primary care. The Large Scale Clinical Trials panel was established in 2009 to ensure that these studies are reviewed by a panel specifically constituted for this purpose.  While the panel is not discipline specific NHMRC ensures that its expertise covers the breadth of applications received, in addition to possessing high level clinical trials expertise.

This variety of review panels provides significant scope for applications in the cancer field to be appropriately grouped and suitable membership selected to provide high quality peer review. External assessors, who are independently selected by the NHMRC Academy, are more likely to match the applicant in both discipline and research category and bring valuable additional expertise to panel deliberations.

The membership of NHMRC peer review panels is carefully selected to ensure the required expertise and experience is present to review all applications received. Senior researchers are very well represented on panels. It is also very important to the future of health and medical research in Australia that the assessment process is open and transparent to the best and brightest mid-career researchers, including through their participation on review panels. Our assessment processes benefit from their involvement.

The membership of NHMRC review panels is kept confidential for a period to ensure the integrity of the process. However, the membership of all project grants review panels is published on the NHMRC website at the conclusion of the process. The membership of program grant panels is known to applicants at the time of interview. NHMRC practice is consistent with the international standard for peer review (including publications) that anonymity is preferable.

Review panels use a seven point scoring system to establish an order of merit. Once the order of merit is established the review panel confirms the relative ranking of each grant. Review panels do not attempt to score individual applications to 1% or less, as assumed.

We note Professor Allen’s concerns regarding the use of journal ratings and citations to judge research achievement. NHMRC policy specifically cautions against the uncritical use of these measures.

Each year NHMRC obtains external reviews of applications to its various funding schemes. In 2009, around 4000 external reviews were obtained for project grant applications alone. A very large number of researchers support NHMRC processes by providing external reviews and this contribution deserves acknowledgment.

NHMRC does accept that the quality of external reviews can vary, and this is best addressed by the review panel. The review panel is aware of the identity and expertise of the external reviewer and this knowledge provides a perspective on the review that is unavailable to the applicants.

NHMRC does have excellent information describing how its funded research has contributed to world medical and scientific knowledge. NHMRC also has a significant collection of information about research that has translated into improved health outcomes and can cite numerous examples.  However, NHMRC does face a major challenge to track and communicate the many positive outcomes that are generated by its supported research and researchers.  This is in part due to the time that often elapses between the end of a grant and the translation of findings. 

All members of the research community share a responsibility to continually consider and broadly communicate the improvements to health that flow from our work.  It may be that Cancer Forum can assist in this regard by highlighting the achievements of Australian cancer researchers, including the great number receiving NHMRC support.

Thank you again for the opportunity to respond.