Peer Review at UDRC: How does it compare to academic peer review?

Kelsey Martinez, PhD, UDRC Manager
March 24, 2021

As you might already know, the UDRC has a rigorous peer-review process. Each published research project that the UDRC produces undergoes peer review. UDRC’s peer review process is similar in some ways to academic peer review but also varies for a few important reasons. In this blog post, I’ll go over the general steps to UDRC’s review process and discuss how it compares to peer review you might experience were you to submit research to an academic journal.

The first step of UDRC’s review process is our internal review step. After our researchers complete their SQL code for each research project, the code is reviewed by a team member. Then, once a first draft of the research project is complete, two UDRC team members review and provide extensive feedback on the draft from abstract to references.

How does UDRC’s first peer review step compare to peer review for an academic journal?

Our internal review step is similar in some ways to academic journal reviews, with the UDRC manager acting as what is known as the handling editor in academic journals. Handling editors for academic journals are usually subject matter experts in papers that get assigned to them. Academic handling editors make the final recommendations to chief editors to accept or reject submissions to their journals.

Upon initial submission of the first draft, the UDRC manager checks the draft to ensure everything is properly formatted and structured. If the draft is not formatted correctly, it is returned to the researcher for revisions. Academic editors also review research submissions for proper formatting and relevant content. Still, in academic journals, your paper may be “desk-rejected” without even being sent out for peer review if the handling editor does not think your work is up to snuff for the journal.

Once the handling editor decides a submission is worthy of peer review, they look for available subject matter experts, usually professors or graduate students, to review your work. Typically, two to three subject matter experts will review your submission to an academic journal. In our case, the handling editor is the UDRC manager, who assigns reviewers and ensures that each of the reviewers’ requested edits are made before the research moves on to the next step in review.

UDRC’s process for this step varies a bit from an academic journal review because our researchers know who their reviewers are - they are members of the UDRC team. Usually, you do not know who your reviewers are when you submit to an academic journal. This process is known as a “blind peer review.”

After internal review, the second step of UDRC’s review peer review process is data partner review. Our data partners provide invaluable feedback about our work. This step in the review process is unique to UDRC because academic journals do not have data partners. However, this is a critical step from our perspective, as our data partners have subject matter expertise about their data and institutions that UDRC does not always have. This step also ensures that UDRC’s use of our data partners’ data is aligned with their agency’s processes and interests. While UDRC may have its own research goals, we also exist to serve our data partners’ needs. This review step makes sure we are meeting those data partner needs.

The third step is to complete the review at an administrative level by having the finished product presented to the appropriate executive-level individuals within our organization. They provide overarching guidance on the end product of the research topic at hand.

Lastly, just before final publication on our website, UDRC acts as our own typesetters and copy editors. We format all of our final research reports using Adobe InDesign to maintain our products’ professional appearance. This internal process is different from academic journals. Were you to have research accepted to an academic journal, the typesetting and final copy edits, such as reference list checks and typo corrections, would be done for you by the journal or publisher.

In some ways, UDRC’s peer review process is very similar to academic journal review processes. This process is by design, such that UDRC may produce high-quality research products. The UDRC also has unique relationships with our data partners and executive director’s office that play key roles in our review process.