As previously published on the IMHA blog here
By: Dawn Richards, Deb Baranec, Eileen Davidson, Trudy Flynn
What is peer review anyway?
The peer review process can be strange and slightly confusing if you’re not in academia or new to the process. At its simplest, peer review is a process used by an organization that funds research to review grant applications that have been submitted to be considered for funding. Funding organizations include the Canadian Institutes of Health Research, a health or other charity, and research foundations. While the exact process may be different depending on the organization, generally grant applications are ‘assigned’ to people who are called ‘reviewers’. Organizations will select reviewers who have the experience or expertise to assess the applications.
Often, a group of reviewers is assigned to each application and will discuss how they rate it at a virtual or in-person peer review meeting. Before the meeting, reviewers spend time reading and assessing each application based on a set of criteria created from the specific objectives of that research funding competition. They will share their assessments at the peer review meeting and discuss them with other reviewers. Reviewers may decide to change their assessment based on the discussions that take place with other peer reviewers at a meeting.
* Peer Review Overview - https://cihr-irsc.gc.ca/e/39380.html
Patient partners as peer reviewers of grant applications
Some organizations that do peer review are now including patient partners in the process. Patient partners bring lived experience and expertise that complements the expertise of other reviewers who may be researchers, clinicians, and others (e.g., people from community- and not-for-profit organizations, etc.).
Organizations may involve patient partners in different ways. Sometimes patient partners are asked to read and review only part of the grant application and sometimes they’re asked to review the grant application the same as any other reviewer. Patient partners are asked to bring all of their skills and experiences to the review which may include their patient perspective and for some it may also mean critiquing the proposed research. For many patient partners, peer review of grant applications is a new concept and a different way to be involved in research.
As patient partners who have been involved in peer review, we thought it would be helpful to use our own experiences to offer some advice to other patient partners as well as to people and organizations who are running peer review that involves patient partners. Our hope is that we can help make this a smooth and informed process for everyone who is involved.
The table below is structured for different points in the peer review process of grant applications and there is a column for what patient partners might find helpful to ask/know about the process and there is a column to help people and organizations consider what might be most helpful to share about their processes with patient partners.
Tips on involving patient partners in peer review of grant applications - questions to ask and things to consider based on the time point in the process
Note: Downloadable PDFs of the below tables are available here:
Before/preparing for the peer review meeting
-
Patient Partners
- Do I have the ‘right’ lived experience for this (e.g., maybe I live with one condition and I’m being asked to review a grant application about another condition)? Do I feel comfortable with this?
- Do I have the time for this role, and what is the amount of time it might take?
- How many applications am I expected to review and about how long will each application take to review?
- Is there a template to fill out as I review each application? How does scoring work?
- Do I need to read and review the entire application or just part of the application?
- Is there an orientation session - how will the process be explained to me?
- Do I have a contact for one-on-one support if I have questions while I’m reviewing?
- Will I need to use technology to submit my reviews? Who can help me if I have issues with the technology?
- Will I be part of a peer review meeting or will I just submit my reviews? If there is a meeting, is it virtual or is it in-person? If it is in person, where is it (and how will that affect me), and will my expenses be paid or pre-paid to attend?
- How long will the meeting be?
- What does the whole process look like in terms of the overall timeline from when I am assigned my reviews to the peer review meeting itself, meetings, etc.?
- How many other patient partners are reviewing and what is the size of the whole review panel?
- Can patient partners be in contact or talk to each other before the meeting to support each other? If not, can I talk to a patient partner who has done this before and who might be able to mentor me informally?
- Is there a research glossary I can use if some of the research terms are not familiar to me?
- Have I kept my reviews concise and constructive, and do my scores (if there are scores) reflect my comments?
- Is the process confidential, and do I need to sign any paperwork about this?
- Has the competition’s approach to conflict of interest been explained to me (i.e., what do I do if I know the researcher or research team on an application that has been assigned to me?)?
- Will there be food and drinks supplied (whether the meeting is virtual or not), and who can I talk to if I have dietary restrictions?
- If I need childcare, will it be paid for?
- Is there an honorarium provided for my time, effort and expertise? If so, what is the amount and will that affect me (e.g., if I am on disability or another form of support)?
-
People/Organizations
- Do the patient partners you have invited to review represent a diversity of perspectives?
- Will you share expectations of patient partners (e.g. do they review the whole application or just a section?)?
- Will you host an orientation session with all patient partners so you can: review the entire process, including any templates or information about scoring; clearly explain the timeline for the process; provide an overview of any technology used; and, go over any logistics related to the meeting (e.g., how long it is, if it’s virtual or in-person, etc.); and so patient partners can ask questions and meet the chair of the peer review panel, etc.?
- If the review meeting is in person, have you arranged for pre-payment of patient partner reviewer’s expenses related to travel? Will reimbursement be timely and can someone support filling out an expense report?
- Have you taken into account all logistics related to food and drinks supplied (whether the meeting is virtual or not), and dietary restrictions? (e.g., for virtual meetings, having food delivered is appreciated if it’s possible)
- Will childcare costs be covered for patient partner peer reviewers?
- Will patient partners who are peer reviewers be able to talk to each other as they review the applications so they can support each other? Does your organization have a way that patient partner reviewers can mentor each other?
- Will you make the process as simple as possible for patient partner reviewers?
- Will you provide a key contact at your organization for patient partners to ask questions?
- Is there a research glossary for patient partners to use if some of the research terms are not familiar to them?
- Are patient partner reviewers familiar with declaring conflicts of interest if they know researchers who have applied to the competition?
- If there is any paperwork related to confidentiality of the review, is this in easy to read, understandable language that you will review and explain to patient partners?
- Will you communicate clearly about there being an honorarium or not? Will there be tax paperwork provided by your organization based on this?
At the peer review meeting
-
Patient Partners
- Has the approach to the meeting and how it ‘works’ been explained well to me?
- If the review meeting is virtual, do I have to be at the entire meeting with my camera on?
- Will I be asked to comment on applications that weren’t ‘assigned’ to me?
- When will I be asked to share my comments in the order of the reviewers?
- Are there breaks in the meeting, and are those enough for me? What if I need to move during the meeting (no matter the format)?
- If the meeting is virtual, what is the technology that will be used? What happens if I need help?
- How does it work if/when I have a conflict of interest with an application that is being reviewed?
-
People/Organizations
- Have you explained how the meeting ‘works’ to patient partners?
- Has the meeting ‘decorum’ been explained to everyone (e.g., if people need to move/stretch between breaks, cameras being on/off during virtual meetings)
After the peer review meeting
-
Patient Partners
- Is all of my paperwork related to the reviews submitted?
- Will I receive notes or minutes from the peer review meeting, and know which applications were funded?
- Can I provide feedback on my experience, how will this feedback be collected, and am I comfortable with this? (e.g., will it be via anonymous survey or through a meeting, etc.)?
- Will I receive feedback on my contributions so I might learn from the process?
- Will I be able to share my experiences with other patient partners who are asked to peer review for this organization?
- Would I like to peer review again for this organization, and if so, have I let my contact there know?
- If I am out of pocket for any expenses for the meeting, when can I expect to be reimbursed? When can I expect to receive my honorarium (if there is one)?
-
People/Organizations
- Have you been clear about notes or minutes from the peer review meeting being provided back to peer reviewers and the timeline associated with this?
- Do you have a feedback mechanism so you can learn about how the process was for patient partners (and is this appropriate in the event someone has negative feedback)? Have you made it clear how feedback will be used to improve the experience for other patient partner reviewers?
- Do you have a plan to provide feedback to patient partners on their reviews so they may learn from the experience?
- Have you checked to see if patient partners who participated in the review meeting would be interested in reviewing again? What about in mentoring other patient partners (formally or informally)?
- Have you taken care of all expenses and honoraria for patient partners related to the meeting in a timely way?
Given that more organizations are starting to involve patient partners in this process, we hope this blog is helpful to explain what peer review of grant applications is (generally), and the types of questions/items that need to be considered when fully integrating patient partners in peer review of grant applications. We believe that involving patient partners in peer review of grant applications can help lead to more relevant research being funded and done. We’ve been part of review meetings where patient partners have brought unique perspectives that other peer reviewers have not considered.
English (UK) 
Français