Writing a Systematic Review: Part 2- Reference Managers & Screening

In the first part of my systematic review blog series I reflected on formulating a research question and the search strategy. A database search can often result in thousands of titles and it can become very overwhelming if you do not take the right steps after. This part of the blog series will go through my experience of using reference managers and the process of screening through search results.

Reference managers

I found out the hard way that using a reference manager is a must when undertaking a big project. While doing my undergraduate project I decided to write all my citations and references manually- never do that, particularly if you are using a numbering reference style!! It drove me crazy formatting the citations because each time I would insert a new reference in the middle of the thesis, I had to manually go through and change the numbers of all the other citations that came after. I’m sure you can imagine the nightmare when I had 90+ references.

References managers not only help with inserting your citations and reference list at the end, but it manages all the documents you use throughout your project. For a systematic review this is really helpful because you can export all you search results from the database into a reference manager and screen them there. This allows you to create folders for different subgroups. For example, I had a folder for included titles & abstracts after the first screening and then another folder for final included studies after I read the full texts. I also created a folder for any studies that I found interesting and was related to my project but did not need for the systematic review itself. On reference managers you can also add notes and highlights in the papers. This makes it very easy to go through your studies and pick out important information. Overall, reference managers makes life easier when undertaking a systematic review!

Now, you might be thinking which reference manager should I use? The simple answer is they all pretty much do the same job and have all the same basic features. But, some might be more appealing than other. I used Mendeley during my Masters and I liked it, however, when I started my PhD I was shown two more options: Endnote and Zotero. I ended up using Zotero due to a few reasons. Firstly, it looked nicer. Yes, you read that right, Zotero looked a lot more user friendly and simpler compared to Endnote which seemed very overcrowded. Zotero is free to download, whereas, Endnote is not, unless you have a license for it which Newcastle University does which meant I could use it for free but only while I was a student at Newcastle. This seemed quite limiting to me as I would want to have access to my files even after I leave Newcastle. Now between Mendeley and Zotero, I prefer Zotero as it has certain features such a tags and notes which allow me to categorise and label my studies. It has a Word plug-in which means you can sync your references in your writing to Zotero so its all automatic, and it has a google chrome extension so you can import articles you find online straight into Zotero by just clicking that icon. Easy peasy!!

Screening

Now that you have all your studies from your database in your reference manager, it is time to screen them against your inclusion & exclusion criteria. This can take a while depending on how many studies you have but its important you do it carefully so you don’t miss studies. Here are a few tips!

• As I mentioned above, create folders for subgroups so you can easily visualise and then rewrite your screening process for the review in a PRISMA flowchart. Also, create subgroups for certain outcomes you’re interested in. For example, I created subgroups for antenatal and postnatal studies, this helps with data extraction later on!
• Have a second (or even third) reviewer at hand. This is crucial, particularly for publication, but also really helps with identifying any studies you may have missed. A second reviewer has really helped me in my screening process!
• A third reviewer can help with disagreements between you and the second reviewer
• If you end up with thousands of papers to screen, don’t do it all in one go. Screening can take a long time, do it in chunks so you don’t wear yourself out and end up missing studies.

Supplementary searches

Databases are your main source of studies, however, there are some supplementary searches you should carry out after you have screened your database studies.

• Hand searching referencing lists of your included studies. This often picks out quite a few studies databases may have missed out on.
• Citation searching your included studies. This is done on Google Scholar and it checks other studies which have referenced that included study. This can take a very long time so if you have a lot of included studies then only screen the first few pages of Google Scholar.
• Contacting authors is useful if you want to find the full text of an abstract or if you want data that is not written in the study.

So that was a quick reflection on reference managers and screening. This part really sets you up for data extraction so take time with it. Stay tuned for the next part of this blog series which will look at data extraction and assessing the quality of the studies you have included for your review.