Miscellaneous

Merit Review Procedures for Advanced Scientific Computing Research Projects at the Department of Energy Laboratories

Research projects funded by the Office of Advanced Scientific Computing Research (ASCR) within the Department of Energy (DOE) laboratories are subject to merit review. This document sets forth the procedures for merit review of research projects funded at these institutions. These procedures are patterned after those given in 10 CRF 605, which govern the Office of Science (SC) grant program.

DOE laboratories may submit proposals that include a Field Work Proposal (FWP) according to DOE Order 412.1, “Work Authorization System,” and supporting documentation suitable for peer review as described in the Office of Advanced Scientific Computing Research "Program Manager Manual." The Review Documents are required for funding of all new research or the periodic renewal of ongoing research.

Administrative Review

1. New Field Work Proposals
Upon receipt of a new FWP(s) and supporting Review Document(s), the Office of Advanced Scientific Computing Research determines whether the documentation contains the prescribed information, has been approved by an official authorized to sign for the Laboratory, and falls under the scientific scope of the Office of Advanced Scientific Computing Research activities. After this preliminary review, the FWP and Review Document undergo further review to determine if the documentation contains sufficient technical/scientific information to conduct an evaluation, meets program policies and priorities, and does not duplicate or overlap currently funded research projects.

ASCR program managers may return a new FWP and Review Document that do not include all information required for a complete review. However, if the new FWP and Review Document contain most of the information required, any missing information may be requested from the Laboratory management so that it can be processed. ASCR program managers may request the submission of additional information if it is needed to evaluate the FWP and Review Document. Before any new FWP is funded, its Review Document will be merit reviewed by the responsible ASCR program manager.

2. Renewal of Ongoing Field Work Proposals
The Office of Advanced Scientific Computing Research routinely conducts renewal merit review of ongoing FWPs every three to five years. When an ongoing FWPs or parts thereof are the subject of a renewal merit review, Review Documents will be required as described in the Office of Advanced Scientific Computing Research "Program Mangers Manual.” Certain types of reviews, such as reviews of scientific computing facilities and networks, may require additional supporting material.

In special circumstances, the Laboratory management may request that other information, such as special assignments or work of special significance to the Department, be included as part of the Review Document.

Evaluation Criteria
New and ongoing FWPs and Review Documents meeting the above standards will be subjected to formal merit review and will be evaluated against the following criteria (the first four criteria are listed in order of decreasing importance):

  1. Scientific and/or technical merit of the project;
    - for example, the influence that the results might have on the direction, progress, and thinking in relevant scientific fields of research; the likelihood of achieving valuable results; and the scientific innovation and originality indicated in the proposed research.
  2. Appropriateness of the proposed method or approach;
    - for example, the logic and feasibility of the research approaches and the soundness of the conduct of the research.
  3. Competency of the personnel and adequacy of proposed resources; and
    - for example, the background, past performance, and potential of the investigator(s); and the research environment and facilities for performing the research.
  4. Reasonableness and appropriateness of the proposed budget.
  5. 5. New and renewal proposals may include additional criteria such as synergism among the PIs in a program, programmatic focus of a multi-PI effort, and utilization of unique facilities.

Other criteria may be stated in an announcement for new DOE laboratory proposals. Past performance is a criterion for all renewal proposals.

The Office of Advanced Scientific Computing Research shall consider, as part of the evaluation, other available advice or information and program policy factors, such as ensuring an appropriate balance among the program areas and the special needs of the Department.

Merit Review Process
ASCR project managers will review FWPs and Review Documents for technical/scientific merit and program policy factors. In addition, the project manager will submit the Review Documents to at least three qualified reviewers for expert evaluation. Instructions to reviewers will include a reasonable length of time for responding to request from ASCR for a merit review. In those instances where three or more reviews are not obtained, the project manager must provide a written explanation to be retained in the official file.

Such additional reviewers may be Federal employees (including those from DOE that are neither the selecting official nor those in a direct line of supervision above the project manager) or non-Federal employees. Also, such additional reviewers will not include former employees of the project manager’s immediate office, or anyone having had line authority over that immediate office, within the past one year.

All reviewers serve as advisors to the selecting official and their recommendations are not binding. All significant adverse recommendations will be addressed in writing by the project manager to the selecting official and retained in the official file.

In selecting additional reviewers, such additional reviewers shall not include anyone who, on behalf of the Federal Government, performed or is likely to perform any of the following duties for any of the applications:

  1. Providing substantial technical assistance to Advanced Scientific Computing Research projects at the Laboratory;
  2. Approving/disapproving or having any decision-making role regarding the FWP;
  3. Serving as the project manager or otherwise monitoring or evaluating the recipient’s programmatic performance;
  4. Serving as the Contracting Officer or performing business management functions for the project; or
  5. Auditing the recipient of the project.

Anyone in ASCR who has line authority over a person who is ineligible to serve as an additional reviewer because of the above limitations also is ineligible to serve as an additional reviewer.

It occasionally may be necessary, after the fact, to change project manager designation, thereby resulting in an individual who participated as an additional reviewer in the evaluation of an application being appointed as the project manager. This is not a violation of the policy of objective merit review, provided the assignment was not expected when the review was conducted.

In order to enhance the validity of the evaluation, Review Documents may be evaluated in comparison to each other.

Generally, the Office of Advanced Scientific Computing Research will conduct a renewal merit review of an ongoing FWP and the supporting Review Documents every three to five years. In no situation will an FWP go for more than five years without a merit review. The criteria to be used as a basis for such an extension beyond three years are as follows:

  1. The nature of the project requires additional time for performance, or
  2. Instances where a final period of support is being authorized to provide reasonable time and funding sufficient to bring the project to an orderly close.

ASCR uses various types of review mechanisms to accomplish a merit review of ongoing programs; however, within each mechanism the reviewer is selected based upon his/her expertise and professional qualifications as they relate to the activities contained in the FWP and Review Documents. Each reviewer chosen to participate will be provided with the Review Documents, the ASCR evaluation criteria (stated above), and other programmatic information needed to conduct the review. Based upon his/her review of these documents and site visit, if appropriate, each reviewer is expected to provide the ASCR program manager with a written analysis based on the pertinent evaluation criteria. The types of review mechanisms used by ASCR and the situations in which they are used follow:

  1. Field Readers:
    1. Merit review of Review Documents may be obtained by using field readers to whom such materials are sent for review and comment. Field readers also may be used as an adjunct to review committees when, for example, the type of expertise needed or the volume of applications to be reviewed requires such auxiliary capacity.
    2. Appropriate action should be taken by ASCR program managers to ensure that field readers clearly understand the process, their role, and the criteria upon which the applications are to be evaluated.
    3. The web-based tool Peernet is the preferred method by which field reader input are to be collected for ASCR projects.
  2. On-site of Off-site review
    An on-site or off-site review of the scientific or technical program attended generally by at least three qualified reviewers who evaluate the program and provide their documented findings to the ASCR program official. Typically, ASCR will require the laboratory to provide Review Documents at least two months prior to an on-site review in order to provide sufficient time to select reviewers and forward the Review Documents to them well in advance of the review.
  3. Ad hoc committees
    Ad hoc committees may be used when it is determined that the projects to be reviewed have special review requirements, e.g., facilities; the complexity of subject matter cuts across several areas of expertise; may involve several FWPs on a similar topic; or the subject matter is of a special, nonrecurring nature. ASCR program managers should ensure that each reviewer on the committee clearly understands the process, their role and the criteria upon which the projects are to be evaluated. Each reviewer on an ad hoc committee is expected to provide the ASCR program manager with a written analysis based on the pertinent evaluation criteria and other program information for each application, preferably via the Peernet system.

Reviewers must comply with the requirements of 10 CFR 1010.101(a) and 1010.302(a)(1) concerning conflict of interest. A committee or group of field readers that includes as reviewers any individuals who cannot meet these requirements or the program’s review procedures, with regard to a particular application being reviewed, must operate as follows:

  1. These individuals or officials may not review, discuss, and/or make a recommendation on a FWP in which they have a conflict of interest.
  2. In the case of a review committee, the committee member must absent himself or herself from the committee meeting during the review and discussion of the application(s) in which he/she has a conflict of interest.

Award Selection
Selection of FWP for funding will be done by the authorized ASCR selecting official and will be based upon merit review, the importance and relevance of the proposed project to ASCR’s missions, and funding availability. Cost reasonableness and realism will also be considered to the extent appropriate. The Laboratory management will be advised of the results of the peer review and may be asked to submit additional details or a revised budget. Such actions are not to be considered a commitment that ASCR will provide funding. Until a formal decision is announced, no information can be provided on the probability of support.

DOE Liability
DOE reserves the right to fund, in whole or in part, any, all or none of the individual FWP’s submitted. FWP’s are funded subject to the terms and conditions of the management and operating contracts between DOE and the contractor organization that operates the laboratory.

Withdrawals
An FWP may be withdrawn at any time. A request for withdrawal must be submitted by the authorized organizational representative.

ASCR Review of Funded Projects
ASCR or its authorized representatives may make site visits, at any reasonable time, to review a project.

Termination of FWP
A FWP may be terminated for cause, by mutual agreement, or because of changes in programmatic mission, funding reductions, lack of productivity, or lack of scientific quality.

Last modified: 3/18/2013 10:12:55 AM