MCHB ALL GRANTEES MEETING

Other HRSA Partners: Competitive Review Process

October 4-7,2004

EULAS DORCH: Good afternoon. It’s a pleasure to be here and if you’re like me, I thought Larry was trying to get his eight hours in. Do you think? Yeah, he was going pretty strong there. It is a pleasure to be here and to introduce you to the Division of Independent Review. It is a new activity, a centralized unit where these activities were generally performed by the individual bureaus in the past, that is basically the requirement that the department has for each competition, competitive to undergo a review by outside experts to give advice and support to the decisions that are ultimately made by the agency in its funding decisions. About January of 2003 this centralization took place, we completed our formal first year of that process just this past September. And, while there were lots of ups and downs in that process, I think any time you try to centralize something that has been ongoing in several different agencies or bureaus independently into a centralized and coordinated fashion, there will certainly be some trials and tribulations, and we certainly had some of that this year. But it’s purpose was to identify a centralized unit where these reviews could occur, to make sure that they occur in a consistent dependable or reliable fashion. And, that’s the mission of the centralized unit. We have combined a staff of about 25 to 30 people, we hope that will be sufficient to service the individual bureaus. But its mission again is specifically to carry out those reviews on applications that come from the various grantees, with outside expertise.

This applies to all discretionary grants and it also applies to cooperative agreements. We conduct our reviews utilizing essentially three methods. The first is a face-to-face meeting, which is probably the most traditional one that probably you are most familiar with where outside individuals are provided with applications and they review those applications and come in to a formal meeting here in Washington , usually. Where they discuss those applications in depth and ultimately score those applications. Teleconference is another methodology that we introduced this past year where we have experts organized in an outside telephone conference call. They are sent the same applications. They are given time to review those applications and prepare them. But we engage those dialogues and consensus buildings and discussions around those applications in a telephone format. A third option that we have introduced this year probably no more than five percent of the applications this year. We’re exploring it to see whether or not it has merit. It certainly has some cost-saving features that are of interest to the department, and ultimately the agency. And, that’s the field reader review model. That model basically is one in which applications are sent to outside experts and they independently review and score those applications, send those results in to the centralized unit, where we coordinate with program staff to ultimately develop a rank list and summary statements related to those reviews. As I say, about five percent of those were done this year, we expect that we may expand that as we go forward, but we’re looking at those results and analyzing those results to assure that it is achieving the desired outcome.

The expertise that we certainly are looking for in our outside reviewers have to be program related, and it relates to the specific programs that our agency is responsible for, health professions, HIV AIDS, Maternal and Child Health, organ transplantation, primary care for the underserved and rural health. So, we look for individuals who have those expertise and have an interest in performing reviews for us to serve on committees and panels that will ultimately evaluate the competitive applications that are sent forth. The grant review selection process also includes input and consideration from the bureaus. They all have particular interest and concerns about the qualifications of reviewers. And, we work with them to assure that those requirements and expectations of reviewer expertise are accounted for and represented on the panels that will review those applications. We’re also required under department requirements to assure that the panels that we convene and the committees that we establish have appropriate balance of ethnicity, gender, and geographic considerations. So, we take those things into account in structuring the panels, beyond the mere expertise of the membership of those panels. And, we also value highly the experience that reviewers have in prior reviews, that they bring to the review panel, so that we assure that there’s a wide and diverse, but also an experienced input utilizing our review process.

Now, it’s important that reviewers who we utilize come to the review sessions understanding and knowing that there are particular expectations. One of them is that we only want reviewers who can review those applications and they are void of conflict of interest. So, we require all reviewers to sign a Conflict of Interest Statement. That is part of the formal record. If the organization is within the competition, we do not want those individuals reviewing applications. We also have to explain to reviewers and we make certain that we go over this a number of times with them that they are to maintain confidentiality associated with the review meetings that they attend, that they maintain confidentiality with regard to the content of the application. What goes on in those meetings and in those discussions should remain in those meetings. The specific requirements include no discussion outside the meeting rooms. The results must remain confidential. At the conclusion of reviews we make sure that we capture all of the extraneous materials that were used in the review, and they are ultimately destroyed. Certain pieces are maintained for a period of time for purposes of record maintenance and our grants management requirements, but its in keeping with the notion that the reviews themselves are confidential. The applications that you provide contain information and the review results have to be maintained and managed in a structured environment. Confidentiality exists at all times, meaning when they receive the applications, when they come to the meetings to participate in the meetings, and after the meetings.

We have a HRSA website where we are inviting people to identify themselves if they are interested in being reviewers within HRSA. And, we have gotten thus far somewhere in the neighborhood of seven to 800 people who have expressed an interest. We are implementing a process by which we validate the information that is provided by those reviewers, because we want to assure that they, in fact, meet the qualifications and expectations needed to participate in these reviews. Applications in the review meetings are to be evaluated on their own merit. They are not compared to one another. The content of the application is the single basis on which the scoring and evaluation of the application is considered. No outside information may be brought by the reviewers to add to the review discussion, while they do bring expertise from the field, we don’t want them bringing in extraneous information or particular information that they know about a given entity or organization weighing in on the meeting. Okay. They are also to avoid any personal bias, this is particularly important. Real quickly, let me just kind of go through this because I’m down to two minutes, the outcome of the review is critical. And, that is we produce essentially in the Division of Independent Review two products. One is a rank order list that is representative of the scoring decisions that have been made by those members of the panels that review those applications. And, we produce summary statements that are reflected of the strengths and weaknesses of each application. Those documents are provided to the individual bureau, MCH would receive that information on which it then would make its funding decisions.

So, those are the products that essentially we prepare. One of those products, the summary statements of strengths and weaknesses will be returned back to the individual applicant for their use and consideration for any future applications that they may provide to this agency. Participants in the process include a chair person at every committee, and those chair people are responsible for managing the review sessions to ensure that conflict of interest are adhered to, to make sure that the discussion stays focused, to act as a timekeeper and to assure completeness of the discussion, so that there is a representative consensus at the conclusion of the review. We also have our own staff responsibilities, the Division of Independent Review will participate at those meetings, essentially convene those meetings. We assure the integrity of the process, and we also have program participation in those meetings. And, they are there to provide clarification and guidance to reviewers and questions they may have about the guidance, about the particular aspects of the regulations that those applications are being evaluated against. Those meetings also includes grants management representation so that should there be a need for questions and input from grants management specialists around fiscal matters, they can receive that direct information from our in-house experts such as Larry. We want to capture that information, give it back to applicants at the conclusion of the reviews so that they are better able to file applications in the future, and we want to do that in a way that is meaningful to them, constructive to the program so that they can make use of that as well, and our goal ultimately is to be able to produce products through this process in a consistent way that reflects the department’s and the agency’s mission, that is to give fair and appropriate consideration to all applications that are submitted to HRSA for review. Sorry I don’t have more time, but thank you very much, I appreciate it.