Main content
Using the BRIDGE guidelines to assess research integrity and research fairness in global health among Netherlands’ alumni
What is the problem?
Research integrity aims to increase the reliability and trustworthiness of research. [1] It has gained momentum over the last decades, as witnessed by the number and wide reach of efforts to promote it. In the Netherlands, as in many other European countries, research integrity has become a cornerstone of higher education. This is witnessed by the publication of the Netherlands Code of Conduct for Research Integrity in 2018 which identifies researchers and institutions and those primarily responsible for good research and which has been adopted by all universities in the Netherlands.* Nevertheless, recent studies in the Netherlands show that over half of researchers from Dutch universities self-report to have engaged in questionable research practices in the past three years. [2]
Applying these principles in practice in global health research can be challenging due to the multidisciplinary and transnational nature of this discipline. [3] Equity concerns in particular should be considered, as global health research often requires collaborations involving both low- and middle-income countries (LMICs) and high-income countries (HICs). [4]
Research fairness is a movement that explicitly addresses equity in research partnerships as a response to the fact that longstanding efforts to improve global health through research partnerships with LMICs have insufficiently translated to improved health systems in these countries. [5,6] Part of the reason is that collaborations between LMICs and HICs have traditionally disproportionally benefitted HIC researchers. [7] Local researchers have not been able to shape research agendas in their countries and have been bypassed in scientific publications. [8,9] This has resulted in skewed opportunities for professional advancement, with consequences for future funding opportunities for research. [10,11] This, in turn, is likely to have hindered the impact of global health research on the main beneficiaries, the local communities, as their interests may not be represented properly in research objectives and the studies performed.
Conducting global health research with a dual focus on integrity and fairness is key as part of a growing awareness for the need to decolonise global health and in order to reach the Sustainable Development Goals (SDG), as emphasised by SDG17. [12,13,14] However, little is known in terms of current practices of global health researchers: to what extent is current research conducted in line with research integrity and research fairness principles? What are the challenges? Here we report on a first pilot study to jointly appraise research integrity and fairness in a sample of Masters in Public and International Health alumni of the KIT Royal Tropical Institute in Amsterdam. The article summarises the study’s approach and main results and implications while the full study report can be found online. (www.bridge-statement.org).
How we investigated it
Recently, the BRIDGE guidelines (Bridging research integrity and global health epidemiology) were published for good epidemiological practice with the aspiration to bridge the gap between research integrity and research fairness in global health research. [15] These guidelines consist of 6 standards and 42 accompanying criteria, and as such, offer a useful tool to assess research practices.
We followed a mixed methods approach combining quantitative and qualitative research components. The quantitative component assessed current practices in global health research as experienced by a sample of KIT alumni who graduated from 2016 to 2020 and who had gained experience in global health research since graduating.
Table 1. Mean level of achievement per statement
| Statements per standard | Mean | SD | n |
|---|---|---|---|
| STUDY PREPARATION PHASE (MEAN 3.8; SD 1.1) | |||
| Research was planned and executed in partnership with local researchers, whilst considering current professional needs and ambitions of those involved locally. | 3.90 | 1.04 | 36 |
| Key stakeholders, including representatives of affected populations and end-users of research, were identified and engaged with consideration of their needs, competences and expectations. | 3.48 | 1.23 | 36 |
| Knowledge gaps were established by searching the literature (peer-reviewed publications and grey literature).* | 3.79 | 1.02 | 36 |
| Knowledge gaps were also established by consulting (local) experts, representatives of affected populations and end-users.* | 3.51 | 1.20 | 36 |
| Research questions and objectives were developed in consultation with research partners and expected end-users. | 3.55 | 1.15 | 36 |
| Study design and research methods were selected to best fulfil the study objectives and give due consideration to multidisciplinary approaches. | 3.91 | 0.98 | 36 |
| Before embarking on primary data collection, it was assessed whether existing data could be used, fully or partly, to fulfil the research objectives. | 3.97 | 1.17 | 36 |
| It was ensured that all research partners had agreed on data ownership and publication agreements. | 3.87 | 1.18 | 36 |
| Work plans and decision-making processes were clarified and agreed on with all study partners. | 3.91 | 1.04 | 36 |
| PROTOCOL DEVELOPMENT PHASE (MEAN 3.5; SD 1.2) | |||
| A detailed research protocol was prepared in consultation with all research partners. | 3.81 | 1.19 | 33 |
| A clear and comprehensive analysis section was written. | 3.74 | 1.06 | 33 |
| Studying the effect of locally relevant equity dimensions was considered. | 3.29 | 1.04 | 33 |
| When conducting multidisciplinary research, the purpose and strategies to integrate different analytical methods was described in the protocol. | 3.45 | 1.18 | 33 |
| It was strived for to make study protocols publicly available, either on a publicly accessible website or in appropriate study registers. | 2.93 | 1.41 | 33 |
| For all data collection and data use concerning human subjects, ethical approval (or a waiver) was obtained from all institutions and countries involved in the protocol. | 3.86 | 1.22 | 33 |
| When working in a setting without ethical review boards or review boards with limited epidemiological capacity, endeavours were made to strengthen local research capacity. | 3.33 | 1.40 | 33 |
| Any data sharing with third parties was explicitly stated in the protocol submitted for ethical review and in the informed consent documents. | 3.83 | 1.28 | 33 |
| DATA COLLECTION PHASE (MEAN 4; SD 1.2) | |||
| Valid and reliable research instruments were chosen, after performing a review of existing instruments and their properties. | 4.11 | 1.13 | 32 |
| It was ensured that research instruments are locally adapted and culturally appropriate. | 4.10 | 1.14 | 32 |
| Concrete guidance for data collection was provided in a document that was available to all data collection staff. | 4.00 | 1.15 | 32 |
| Data collection staff was selected according to technical as well as cultural criteria.* | 3.93 | 1.20 | 32 |
| The roles and responsibilities for each person involved were clarified for which adequate training and support was provided.* | 3.93 | 1.26 | 32 |
| All research instruments were pilot-tested and, if possible, field-tested prior to the start of effective data collection. | 3.81 | 1.27 | 32 |
| Data was collected in a respectful and safe manner and in an environment which safeguards the confidentiality of respondents. | 4.29 | 1.21 | 32 |
| Quality assurance and control mechanisms were put in place to ensure data accuracy, completeness and coherence. | 3.84 | 1.186 | 32 |
Table continues on next page
| Statements per standard | MEAN | SD | N |
|---|---|---|---|
| DATA MANAGEMENT PHASE (MEAN 3.7; SD 1.3) | |||
| Data management procedures were put in place before effective start of data collection and concrete guidance was provided in a document available to all data management staff. | 3.76 | 1.21 | 31 |
| A data entry application was created and pre-tested prior to effective start of data collection. | 3.67 | 1.44 | 31 |
| All variables were described in a codebook. | 3.41 | 1.27 | 31 |
| Quality assurance and control mechanisms were put in place to ensure data accuracy, completeness and coherence. | 3.62 | 1.24 | 31 |
| All data cleaning and processing steps were annotated and reproducibility was strived for by means of stored programming code. | 3.41 | 1.39 | 31 |
| For each data file, levels of anonymisation and privacy protection were defined as well as corresponding access rights in line with national and international frameworks. | 3.89 | 1.25 | 31 |
| At the beginning of the study, an electronic secured study file was prepared to store all study documentation and outputs.* | 3.67 | 1.27 | 31 |
| The electronic secured study file was (or is planned to be) archived at the end of the study.* | 3.86 | 1.21 | 31 |
| Source data was retained safely, in their original form, preserving data confidentiality for as long as has been described in the protocol. | 3.93 | 1.33 | 31 |
| DATA ANALYSES PHASE (MEAN 3.6; SD 1.2) | |||
| Only personal identifiers that are necessary to answer the research questions were worked with. | 3.88 | 1.30 | 30 |
| Statistical analyses were conducted in accordance with the protocol.* | 3.88 | 1.30 | 30 |
| When statistical analyses did deviate from the protocol, this was annotated and a distinction was made between pre-planned and exploratory analyses.* | 3.45 | 1.18 | 30 |
| All analysis steps were fully annotated and reproducibility was strived for by means of programming code. | 3.24 | 1.14 | 30 |
| In multidisciplinary studies, statistical analyses with analyses from other study disciplines were integrated in an iterative process to coherently address the research objectives. | 3.36 | 1.26 | 30 |
| Quality assurance and quality control mechanisms were put in place to ensure that data has been correctly analysed. | 3.71 | 1.20 | 30 |
| DISSEMINATION AND COMMUNICATION PHASE (MEAN 3.2; SD 1.5) | |||
| User-specific dissemination and communication plans were developed in consultation with key stakeholders, which included (amongst others) representatives of the affected populations and end-users. | 3.44 | 1.36 | 30 |
| Data was reported in a non-stigmatising, non-discriminatory, culturally sensitive and non-identifying manner. | 3.64 | 1.60 | 30 |
| Reporting guidelines were conformed to, for the given study design and methods in academic publications. | 3.50 | 1.50 | 30 |
| Quality assurance and quality control mechanisms were put in place to ensure complete, accurate, accessible and interpretable data reporting. | 3.36 | 1.41 | 30 |
| Indexed open access journals were considered for scientific publications. | 3.17 | 1.44 | 30 |
| On study completion, reanalyses of the data by local researchers was encouraged as much as possible.* | 2.48 | 1.59 | 30 |
| On study completion, key stakeholders and research partners were consulted to identify strategies to encourage reanalyses of the data by local researchers.* | 2.39 | 1.53 | 30 |
Table 1. Questionnaire with mean level of achievement reported on a 5-item Likert scale (1 = not achieved, 2 = slightly achieved, 3 = partially achieved, 4 = mostly achieved, 5 = completely achieved) and standard deviation (SD) per statement. N = The number of participants who answered the questions.
* Statements that were originally part of one criterion within the standard in question from the BRDIGE checklist, but split up into 2 statements.
The BRIDGE guidelines were used to develop an online survey to rate the level of achievement of each criterion (rephrased as statement) on a 5-item Likert scale ranging from 1 for “not achieved” to 5 for “completely achieved”. The qualitative component was used to further examine and explain study participants’ views and to identify facilitators and barriers in fulfilling the BRIDGE criteria. This was done by conducting in-depth interviews in a sub-sample of the survey respondents.
What we found
A total of 36 KIT alumni who were eligible responded to the online survey, of which 18 (50%) were men and 17 (47%) women. Twenty-five (69%) were from LMICs (Ghana, India, Nigeria, Mozambique, Indonesia, Myanmar, Gambia, Uganda, Kenya, Yemen, Angola and Bangladesh) and 10 (28%) were from HICs (The Netherlands, Singapore/Australia, The United Kingdom, The United States and Norway), of which Ghana and the Netherlands were the most represented (n= 6), followed by Uganda (n= 4), Nigeria (n= 3), Bangladesh and Indonesia (n= 2). One survey participant gave no home institution information. Of these 36, four alumni participated in a follow-up interview and all four alumni were from Asian or African countries.
Summary survey results can be found in Table 1, where the mean achievement scores (and standard deviations) are presented for each BRIDGE criterion and each standard. On average, respondents reported “mostly achieved” (mean = 4) for the criteria related to data collection, and in particular those related to using valid and reliable instruments, as well as local adaptation and cultural appropriateness of instruments and respectful and confidential data collection procedures. The lowest scores were reported for items related to open science practices: making study protocols available in publicly accessible websites or in appropriate study registers, publishing in open access journals, and encouragement of re-analyses of data by local researchers (around or below a mean score of 3, denoting “partially achieved”).
Communication, resources, local context, incentives and ownership were all mentioned during the interviews as main facilitators and barriers to the achievement of BRIDGE criteria. A lack of resources was often described as a main issue. Whether it concerned time, money or staff, all interviewees felt that if they had unlimited resources, they would have been better able to fulfil criteria. Many other themes mentioned in the interviews can be linked to this central one.
However, interviewees also thought that open communication and familiarity with local context were important success factors in global health research – and lack thereof an important barrier:
| “When I went to the village office, to the leader, there were a lot of arguments to discuss, because I didn’t receive the first briefing, of course. It’s not equal because the village leader, he doesn’t have health education; I’m not sure. Also in the field, when I met the village midwife—there’s a village midwife—she didn’t know about the research for example. We have to make sure that the communication between all the stakeholders involved should be provided.” (Female global health researcher from LMIC in collaboration with researchers from the Global South reporting lower levels of achievement of the BRIDGE guidelines) |
Data ownership and preferences for alternative formats for publication (not peer-reviewed publications) were mentioned as the most important barriers for open science practices. On the other hand, public availability of data and the encouragement of reanalyses by local researchers were directly linked with the willingness of the stakeholders involved. More specifically, the willingness of stakeholders to provide data was often linked to monetary resources and the urge to protect own interests. However, the fear of not receiving the proper credit for one’s work seems to also be of influence with regard to the public availability of data for reanalyses.
| “The people who reanalyse, maybe he do not give enough acknowledgement who actually collect the data, who are the main principal investigators. When you publish papers, or when you write an article in newspapers or for a conference […] then everybody will give credits to you that you have done very good job. But it is actually data from another person. Actually, the audience will not know, […]. Who present, he is the actual owner.” (Male global health researcher from LMIC in collaboration with researchers from the Global North reporting intermediate levels of achievement of the BRIDGE guidelines) |
What are the implications?
Overall this pilot study shows good self-reported achievement of the research integrity and research fairness principles in a sample of KIT alumni who are currently engaged in global health research. In particular, respectful and culturally appropriate data collection are reported as common practice which is very encouraging, though it may also be the result of a positive reporting bias. In general, findings underlined the importance of proper communication and knowledge of the local context as a pre-condition to foster both research integrity and research fairness in global health research.
However, this study does highlight some of the known tensions between research integrity and research fairness in global health, in particular with open science. Making study protocols publicly available ahead of the research study, reproducibility of analyses with stored data management and data analyses programs, and open access publishing are all hallmarks of open science and associated attempts to ensure the reproducibility of science. These were all comparatively less frequently practiced. Furthermore, practices related to data sharing – the cornerstone of open science – were also experienced rather unfairly: efforts to promote re-analyses of data by local researchers were rather uncommon and data sharing was associated with feelings of insufficient acknowledgement and involvement of those who actually collected the data. This raises the question on how promoting open science practices may influence equity dimensions between local researchers and those who re-use their data.
How to move on?
This pilot study was followed by a broader Research Integrity and Research Fairness Survey (https://zenodo.org/record/6632009#.Y2Uu9XbMI2x). Results from both surveys will serve as information to design a BRIDGE coaching program for global health researchers.
* Available at: https://www.universiteitenvannederland.nl/files/documents/Netherlands%20Code%20of%20Conduct%20for%20Research%20Integrity%202018.pdf, https://vsnurestyle.digivotion.com/en_GB/research-integrity
References
- World Conferences on Research Integrity Foundation (WCRIF) [Internet]. Available from: https://wcrif.org/
- Gopalakrishna G, ter Riet G, Vink G, Stoop I, Wicherts JM, Bouter LM. Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in the Netherlands. PLoS One. 2022;17(2 February):1–16.
- Alba S, Mergenthaler C. Lies, damned lies and epidemiology: Why global health needs good epidemiological practice guidelines. BMJ Glob Heal. 2018;3(5):10–2.
- Koplan JP, Bond TC, Merson MH, Reddy KS, Rodriguez MH, Sewankambo NK, et al. Towards a common definition of global health. Vol. 373, The Lancet. Elsevier; 2009. p. 1993–5.
- Lavery J V, Ijsselmuiden C. The Research Fairness Initiative: Filling a critical gap in global Research Fairness Initiative research ethics. 2018;2:58(0).
- Musolino N, Lazdins J, Toohey J, IJsselmuiden C. COHRED Fairness Index for international collaborative partnerships. Lancet. 2015;385:1293–4.
- Walsh A, Brugha R, Byrne E. “The way the country has been carved up by researchers”: ethics and power in north-south public health research. Int J Equity Health. 2016 Dec 12;15(1):1–11.
- Gautier L, Sieleunou I, Kalolo A. Deconstructing the notion of “global health research partnerships” across Northern and African contexts. BMC Med Ethics. 2018;19(1):49.
- Rees CA, Ali M, Kisenge R, Ideh RC, Sirna SJ, Britto CD, et al. Where there is no local author: A network bibliometric analysis of authorship parasitism among research conducted in sub-Saharan Africa. BMJ Glob Heal. 2021 Oct 1;6(10):e006982.
- Hedt-Gauthier B, Airhihenbuwa CO, Bawah AA, Burke KS, Cherian T, Connelly MT, et al. Academic promotion policies and equity in global health collaborations. Vol. 392, The Lancet. Elsevier; 2018. p. 1607–9.
- Cash-Gibson L, Rojas-Gualdrón DF, Pericàs JM, Benach J. Inequalities in global health inequalities research: A 50-year bibliometric analysis (1966-2015). PLoS One. 2018 Jan 1;13(1):e0191901.
- Lawrence DS, Hirsch LA. Decolonising global health: Transnational research partnerships under the spotlight. Int Health. 2020;12(6):518–23.
- Costello A, Zumla A. Moving to research partnerships in developing countries. Br Med J. 2000;321(7264):827–9.
- Addo-Atuah J, Senhaji-Tomza B, Ray D, Basu P, Loh FH (Ellen), Owusu-Daaku F. Global health research partnerships in the context of the Sustainable Development Goals (SDGs). Res Soc Adm Pharm. 2020;16(11):1614–8.
- Alba S, Verdonck K, Lenglet A, Rumisha SF, Wienia M, Teunissen I, et al. Bridging research integrity and global health epidemiology (BRIDGE) statement: Guidelines for good epidemiological practice. BMJ Glob Heal. 2020;5(10):1–10.
