Dissertation and research project success
Systematic and systematised reviews
Information on systematic and systematised reviews, including planning your search strategy, recording your search and screening your results.
Systematic reviews
A systematic review tries to answer a specific research question by collecting and analysing all the evidence from literature that is available on the topic. A systematic review typically:
- Involves a detailed and comprehensive plan and search strategy to eliminate any bias in the finding, appraising and synthesising of the literature.
- Is undertaken by more than one person.
- Can take more than 6 months.
The expectation for undergraduate and master’s level students is more likely to be a systematised review. A systematised review follows the principles of a systematic review but is not as rigorous or as in depth. Make sure you refer to your assignment brief to understand the expectations of your review.
Systematic and systematised reviews sit within a large family of review types. See the following articles for information on different review types:
Grant, M.J. and Booth, A. (2009) ‘A typology of reviews: an analysis of 14 review types and associated methodologies’, Health Information & Libraries Journal, 26(2), pp. 91–108.
Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019) ‘Meeting the review family: exploring review types and associated information retrieval requirements’, Health Information & Libraries Journal, 36(3), pp. 202–222.
Reasons to take a systematic/systematised approach
Reasons for taking a systematic/systematised approach to your review include:
- Methods can be replicated by others.
- Minimises bias and avoids ‘cherry picking’ of articles.
- Increases confidence in the findings of the review.
- Identifies research gaps and shows what the field’s understanding is of a topic.
Where to start
The following texts offer a good starting point for undergraduates and master's level students undertaking a systematic or systematised review:
Aveyard, H. (2023) Doing a literature review in health and social care: a practical guide. 5th edn. Maidenhead: Open University Press. (ebook).
Cherry, G., Boland, A. and Dickson, R. (2023) Doing a systematic review: a student’s guide. 3rd edn. Thousand Oaks: SAGE. (ebook).
Booth, A., Sutton, A., Clowes, M. and Martyn-St James, M. (2022) Systematic approaches to a successful literature review. 3rd edn. Los Angeles: SAGE. (print).
There are several steps involved in undertaking a systematised/systematic review:
- Develop a research question
- Plan your search strategy
- Run and record your search
- Screen your search results
- Appraise the studies and extract the data
- Analyse and synthesise
- Write up
Your Academic Liaison Librarian can assist with steps 2 to 4.
Refer to your supervisor for help with developing your research question, appraisal and data extraction, analysis and synthesis and writing up the review.
If you would like help with developing your critical thinking and writing skills, then you can book a tutorial with the Study Development team.
Plan your search strategy
Your search needs to be as exhaustive as possible so that you don’t miss relevant literature. This usually means using more than one database or search tool. Which databases or search tools you use will depend on the topic of your review and the type of literature you are looking for. In most cases, you will be looking for research to answer a review question.
Key databases
Each subject area will have key databases that search the published research in particular fields. Identify the key databases in the relevant area by selecting a subject on the A-Z of Specialist Subject Resources (Databases).
Your Academic Liaison Librarian can offer advice on which ones to search.
A lot of the key databases have a Global North, English language bias. The University of Leeds have created a list of databases that are good for finding research from underrepresented voices. Consider these when thinking about the comprehensiveness and inclusivity of your search.
Additional search methods
Citation searching
Look at the list of references of key articles to identify possible other studies.
Some resources, such as Web of Science and Google Scholar, allow you to track citations forward and identify papers that have cited a key paper since it was published. Citation searching gives an alternative way of identifying relevant studies which may not be retrieved by your database search.
Hand-searching key journals
If you’ve identified any key journals, then you might want to browse and search these directly. Use the A-Z journals to access individual journals.
In addition to identifying published research, you may also need a strategy for finding grey literature. Grey literature is research material that has not been published in the usual way. This can include:
- Conference papers or conference proceedings
- Clinical trials
- Government documents
- Newsletters, fact sheets, bulletins, pamphlets
- Reports
- Theses
This sort of literature can be found in lots of different places. The best approach is to identify key search tools and relevant organisational websites to search. Examples include:
- Theses: Ethos (currently unavailable), Proquest Dissertations and Theses (Web of Science).
- Conference papers: Web of Science, Google Scholar, IEEE.
- Government documents: Gov.UK, National Archives.
- Clinical trials: TRIP, Clinicaltrials.gov, Cochrane Library.
Useful text:
Bonato, S. (2018) Searching the grey literature: a handbook for searching reports, working papers, and other unpublished research. Lanham: Rowman & Littlefield. (ebook).
Before you run your database searches you need to carefully consider what words you need to include in your search. People can use different terminology and you need to account for this if you would like your search to be as exhaustive as possible.
Think about:
- Synonyms.
- Related terms.
- Alternative spellings, such as UK/US.
- Singular and plural forms of words.
- Abbreviations. Check that these don’t introduce a lot of unwanted results due the abbreviation being used for more than one thing.
It is unlikely that you will come up with the best search terms straight away. There is often plenty of trial and error involved before settling on the search terms to use.
Search operators, also called Boolean operators, allow you to include multiple words and concepts in your searches. This means you can search for all your terms at once rather than carrying out multiple searches for each concept.
There are 3 main operators:
- OR: for combining alternative words for your concepts and widening your results, for example, views OR attitudes.
- AND: for combining your concepts giving more specific results, for example, children AND poverty.
- NOT: to exclude specific terms from your search, for example, jaguar NOT vehicle. Use this with caution as you might exclude relevant results accidentally.
Databases often allow the use of certain symbols for specific searching. Two common ones are used for phrase searching and truncation.
Phrase search
Use quotation marks for example “higher education” to search for 2 or more words next to each other.
Truncation
Use an asterisk after the word stem to search for variant word endings, for example, music* will find literature containing the terms music, musical, musician, musicians, musicality.
It is important to think about which words truncation will introduce to your search, for example, nurs* will find nurse, nurses and nursing but will also find nursery. In some cases, it might be better to search for the alternatives that you want, such as nurse OR nurses OR nursing.
Each database is unique and has its own features and tools. If you are using phrase searching, truncation and other operators, you might need to adapt your search to ensure it works correctly in each database.
Databases allow you to limit where your search terms appear in the literature, for example, title and abstract. This can help focus your search. Think about where you would expect search terms to appear in relevant studies. Be careful about being too specific, such as searching title only.
Often, databases offer limits or filters to help focus your search. Common limits include:
- Date range
- Language
- Source type
Think carefully about using limits as you might exclude relevant literature. You will need to justify their use and be clear about any limitations.
Some databases offer a full text limit option. Avoid using this as it will introduce bias into your review. If you can’t access the full-text through York St John subscriptions, then use the inter-library loans service to request articles.
A comprehensive search may involve the use of subject headings. A subject heading is like a tag, or a label, which describes what the item is about. Subject headings are useful because they provide a consistent way of describing the subject matter of the item and help tackle the issue of authors referring to a particular concept in different ways. They will retrieve articles regardless of the terminology used by authors.
Different databases use their own subject headings and not all databases use them. The subject headings in Medline and PubMed are called 'MeSH terms': MeSH stands for 'Medical Subject Headings' and these are the terms created by the National Library of Medicine.
You should not rely on a subject heading search alone but should combine it with a good keyword search.
Further information on subject headings can be found on the video below.
Screen your search results
Once you have collated all your results, you can remove any duplicates and start screening the results against your inclusion and exclusion criteria. Inclusion and exclusion criteria determine the scope of your review, for example, research type, locality and date.
Example criteria:
Inclusion criteria:
- Research relating to nurses working in The United Kingdom.
- Published literature only.
- English language only.
- 2006 onwards.
Exclusion criteria:
- Research relating to nurses working outside The United Kingdom.
- Unpublished research.
- Non-English language.
- Pre-2006.
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) provides a flow diagram to report the number of records retained and excluded at each step in the screening process. There are different templates to use depending on the approach that you are taking:
- New systematic reviews which included searches of databases and registers only. Use if you are only searching databases to identify your literature. This is often suitable for undergraduate or Masters' projects.
- New systematic reviews which included searches of databases, registers and other sources. Use if you are using other methods, in addition to databases searches, to identify your literature.
- Use one of the other two templates if you are updating a systematic review.
