The Quality of Randomized Controlled Trial in Cochrane Kidney and Transplant Group
PDF
Cite
Share
Request
Original Research
VOLUME: 8 ISSUE: 4
P: 266 - 287
December 2021

The Quality of Randomized Controlled Trial in Cochrane Kidney and Transplant Group

J Urol Surg 2021;8(4):266-287
1. Research Center for Evidence-based Medicine, Iranian EBM Centre: A Joanna Briggs Institute (JBI) Center of Excellence, Tabriz University of Medical Sciences, Tabriz, Iran
2. Nikoukari Ophthalmology University Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
3. Hematology and Oncology Research Center, Tabriz University of Medical Sciences, Tabriz, Iran
4. Department of Physiology, Faculty of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
5. Department of Medical‐Surgical Nursing, Faculty of Nursing and Midwifery, Tabriz University of Medical Sciences, Tabriz, Iran
No information available.
No information available
Received Date: 23.02.2021
Accepted Date: 02.07.2021
Publish Date: 16.12.2021
PDF
Cite
Share
Request

ABSTRACT

Objective:

Misconduct is one of the important issues in research integrity. Cochrane systematic reviews are known for their best level of evidence. Since kidney failure is a major public health problem worldwide, the Cochrane Library provides a robust and reliable database to upgrade medical knowledge and make the best medical decisions. Therefore, this study aimed to assess the quality of randomized controlled trials (RCTs) that are included in the Cochrane systematic reviews of kidney and transplant groups.

Materials and Methods:

This analytic cross-sectional study was conducted on systematic reviews of kidney and transplant group of Cochrane reviews. All types of biases in the understudied RCTs or quasi-RCTs of these systematic reviews were evaluated using the Cochrane appraisal checklist. The types of biases in included studies were also separated and stratified. Descriptive statistics were used for data analysis using the Statistical Package for the Social Sciences 16.

Results:

A total of 267 systematic reviews and their understudied 3213 RCTs were evaluated. In the kidney and transplant group, the highest risk of bias was seen in allocation concealment bias, whereas the most common bias was unclear allocation concealment (selection bias). From 2008 to 2009, high random sequence generation bias has dramatically increased, and after decreasing, the gradual growth has been continuing over time. Furthermore, the low detection bias has reduced surprisingly in 2011 then decreased in 2012-2013.

Conclusion:

Regarding high risks of performance and random sequence generation biases in understudied RCTs, critical structure deficiencies were obvious. Therefore, observing integrity principles to prevent research misconduct is recommended.

Keywords:
Risk of bias, randomized controlled trial, Cochrane, systematic review

What’s known on the subject? and What does the study add?

Misconduct is an important issue on research integrity. Cochrane systematic reviews are known for their best level of evidence. Cochrane Kidney and transplant group is one of the chief review groups of this database. A total of 267 systematic reviews and their understudies 3213 RCTs were evaluated. All of the systematic reviews in kidney and transplant group had high quality. In the understudies RCTs of these review, the highest risk of bias had been seen in allocation concealment bias, and the most common bias was unclear allocation concealment (selection bias).  It’s recommended observing integrity principles for preventing research misconduct.

Introduction

Misconduct is an important issue on research integrity (1). In recent decades, a dramatically increased number of published articles in different fields of medical sciences have been reported. As a result, the structure of published articles in medical journals and their adaptation to provide reporting standards and research methodology have been considered more than ever (2). However, the main concern has always been the presence of quantitative growth of research with their qualitative development. In the study’s pyramid, the highest levels of best evidence belong to meta-analysis and systematic reviews (3). Cochrane Library provides a robust and reliable database for upgrading medical knowledge and helps to make the best medical decisions. Cochrane reviews are systematic research reviews in healthcare and health policy published in the Cochrane Database consisting of 52 review groups that focus on specific topics (4). Cochrane kidney and transplant group is one of the chief review groups of this database and is responsible for identifying all renal disease trials, trials quality assessment, collecting and analyzing trial data, and preparing organized reports for inclusion in reviews working on 214 items (5). The activity area of this group includes acute renal failure (ARF), chronic renal failure (pre-dialysis, hemodialysis, and peritoneal dialysis), diabetes mellitus, glomerulonephritis (including nephrotic syndrome, immunoglobulin A nephropathy, lupus nephritis, Henoch–Schönlein purpura, and other glomerular diseases), kidney transplantation, solid organs transplantation, urinary tract infections, and the effects of drugs on renal function (6). Kidney failure is a major public health problem worldwide, with increasing incidence and prevalence, high costs, and poor outcomes (7). A significantly higher prevalence of chronic kidney disease (CKD) in earlier stages and adverse consequences, such as loss of kidney function, premature death, and cardiovascular disease, was reported (8). Moreover, many heterogeneous disease pathways led to CKD that irreversibly altered the function and structure of the kidney in months or years (9). CKD is a frequent phenomenon that affects 1 out of 10 cases (10) in the general population and increases the risk of morbidity and mortality (11). An analysis in 2017 estimated the global prevalence of CKD as 9.1% or 697.5% cases. The age-standardized global prevalence of this disease was higher in females (9.5%) than that in males (7.3%). More than 10 million cases were detected in 10 countries, and more than 1 million cases have been identified in 79 countries. An increase of 29.3% in the all-age global prevalence of CKD was reported between 1990 and 2017, whereas a significant change was not observed in the age-standardized global prevalence (12). These diseases increased globally due to elevation in the prevalence of hypertension, obesity, diabetes mellitus, and most importantly, aging (13). Renal diseases are the ninth most common cause of death in the United States with a higher mortality rate compared to breast and prostate cancers (14,15). In the United States, the unadjusted prevalence of CKD in 2011 through 2014 was estimated at 14.8%. A total of 120,688 new cases of end-stage renal diseases (ESRD) were reported in 2014 (a 1.1% increase compared to 2013). A total of 678,383 individuals were treated for ESRD at the end of 2014 (up 3.5% from 2013), a number that continues to rise due to falling mortality rates among those with ESRD (16). CKD is associated with increased cardiovascular mortality and disability (17). However, the lack of kidney disease registry in many low and middle-income countries has made it difficult to determine the true CKD load. In low and middle-income countries, higher mortality rate is usually due to expensive services of kidney replacement therapy (18). In Iran, according to the result of Safarinejad (19) study (2009), the prevalence of CKD was reported at 12.6%. Other kidney-related disease includes ARF (20) with an incidence of 5%-20% in adolescents admitted to the care unit (21). ARF is associated with high morbidity and mortality, and >70% of people with ARF need supportive care. Despite advances in clinical care, people with ARF have a high risk of mortality and morbidity that needs significant health care resources (22).

The Cochrane systematic reviews are known for their best level of evidence. The Cochrane International Foundation uses a precision instrument to evaluate randomized clinical trials (RCTs) to examine the types of possible bias in each study that distort the credibility and accuracy of the regular Cochrane reviews (23). The Cochrane kidney and transplant group are responsible for identifying all trial-related kidney diseases and transplant, evaluating the relevance and trial quality, collecting and analyzing trial data, and preparing reports including systematic reviews of the Cochrane Database. The Cochrane Library provides a robust and reliable database to upgrade medical knowledge and make the best medical decisions since kidney failure is a major public health problem worldwide with increasing incidence and prevalence, high costs, and poor outcomes. Therefore, this study aimed to assess the quality of understudied RCTs or quasi-RCTs included in the Cochrane systematic reviews of kidney and transplant groups.

Materials and Methods

This analytic cross-sectional study was conducted on published systematic reviews of kidney and transplant groups of the Cochrane reviews to evaluate the quality of their understudied RCTs or quasi-RCTs.

After proposal approval and Ethics Committee confirmation of Research Deputy of Tabriz University of Medical Sciences, Tabriz, Iran (code: IRTBZMED.REC.1396.577), all systematic reviews that were published in kidney and transplant group, were prepared. The quality of Cochrane kidney and transplant group systematic reviews or meta-analysis and their understudied RCTs were evaluated at the presents study, thus informed consent was not applicable.

The Cochrane Library is a collection of databases that contain different types of high-quality and independent evidence to inform healthcare decision-making. Related topics include CKD, hypertension, end-stage kidney disease, kidney transplant, acute kidney injury, and urology.

The current study selected all systematic reviews that focus on the kidney and transplant after an electronic search in the Cochrane Library. Firstly, all included systematic reviews were listed in Table 1. Then, the general information, including title, year of publication, author name, study location, and other necessary information was extracted from each study (supplementary file 1). Next, all of the understudied RCTs of these systematic reviews were evaluated. Therefore, the number of RCTs that were included in the systematic reviews was counted. Then, all kinds of bias, which were evaluated by the authors of these systematic reviews were counted and listed on the column of the related topic of bias.

All included RCTs in the Cochrane reviews were appraised by the authors of systematic reviews using the standard risk of bias tool developed by the Cochrane group. This tool consisted of six dimensions, including the method of random sequencing, random assignment of samples, selective report of consequences, blindness, the existence of any probabilistic suppression of results, and reporting of incomplete data.

Each of the cases examined in the tool was reported in three ways, including low-risk, high risk, and unclear-risk bias. The standard risk of bias tool is a valid and reliable tool for evaluating all RCTs, regardless of the language, time, and location of article publication (24). All types of evaluated biases were counted based on the results of “Risk of bias summary: Review author judgments about each risk of bias item for each included study.” And then, all types of biases were counted for all systematic reviews, listed in appropriate columns, and calculated the sum of all kinds of biases.

Statistical Analysis

All types of biases in RCTs or quasi-RCTs were gathered and finally, all types of biases in included studies were separated according to the publication date. Descriptive statistics were used to analyze data. Data were analyzed using the Statistical Package for the Social Sciences software (SPSS 16, SPSS Inc., Chicago, IL, USA).

Results

A total of 267 systematic reviews, which were published in the Cochrane kidney and transplant group until October 2019, and their understudied 3213 RCTs or quasi-RCT were included.

All published systematic reviews and meta-analyses followed the preferred reporting items for systematic review and meta-analysis for their report (24).

Among the several biases, the highest risk of bias belonged to the allocation concealment. However, the most common bias was the unclear allocation concealment (selection bias). Then unclear random sequence generation (selection bias) and selective reporting bias were in the next ranks. According to the findings, in 2008-2009, high random sequence generation bias dramatically increased and after decreasing, continued to grow gradually over time. Furthermore, low detection bias has decreased in 2011 and 2012-2013, respectively (Figure 1).

From 2014 to 2018, the unclear allocation bias was the most common bias among others. However, the highest risk of bias was seen in 2014 to 2018 in attrition, performance, and reporting, respectively in the included studies.

The number (%) of all kinds of bias in published studies in the Cochrane kidney and transplant group are summarized in Table 1 (Supplementary file 1).

Discussion

Misconduct is one of the most important issues in research integrity of clinical research, which is defined as poor management or administration. The most common causes of misconduct in clinical research are financial interest, professional ambitions to become famous, complex study design, and consequently, the lack of researcher motivation or laziness and expectations of organization or government (25).

The medical literature is an essential and also helpful resource to make the best clinical decision. Hence, improper clinical outcome reporting can influence the health care system at all levels, from patient treatment to modifying and developing national public health policies (26). Therefore, methodological quality assessment of studies is a crucial stage in the best clinical literature selection process. The methodological quality evaluation of the study should be based on evaluating internal and external validity, which characterizes the design conduction, data analysis, or degree of study result generalization, respectively (27). The highest level in the evidence pyramid belongs to meta-analysis and systematic review of RCTs (26). These study types present the best evidence for beneficial treatment in clinical research. Furthermore, the most robust clinical evidence constitutes the systematic reviews of homogeneous RCTs. Therefore, these types of studies had the highest impact on the guidelines, as well as decision-making. However, any misconduct could have a remarkable influence on caregiving quality. In addition, studies with a high risk of bias can lead to false evidence, which affects both the patients and the healthcare system in different aspects.

The use of their results will also be effective in advancing science by promoting the quality of research. Additionally, poor-quality research may lead to inaccurate conclusions. Thus, compliance with research and reporting methodology standards is necessary for the quality improvement of published articles. Incorrect reporting of clinical outcomes can affect health care at all levels, from the design of national public health policies to the treatment of the patient. Therefore, the quality confidence of these articles seems to be critical (28).

A systematic review attempts to identify, appraise, and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers who conduct systematic reviews use explicit methods to minimize bias and produce more reliable findings that can be used to inform decision-making (29). The Cochrane Library provides a robust and reliable database to improve and develop medical knowledge and, most importantly, to make the best medical decisions (30). Therefore, preserving the quality of such studies, which will be utilized in the development of guidelines, is crucial.

Bias can occur in any phase of the conducted research, including planning, data collection, analysis, and publication. Understanding research bias and consequently, its effect on study results allows readers to critically and independently review the scientific literature and avoid suboptimal or potentially harmful treatment (31).

Our study results revealed that among different types of bias in all dates, the highest risk of bias belonged to selection. Unclear allocation concealment was the most common bias in our study in this Cochrane group. Selection bias or systematic differences between baseline characteristics of the groups that are compared may occur during study population identification. It means that the ideal study population was not clearly defined, accessible, reliable, and at increased risk to develop the outcome of interest (32). Prospective studies (particularly RCTs), where the outcome is unknown at the time of enrolment, are less prone to selection bias (33).

However, the evaluation of RCTs in our study showed that the unclear allocation concealment was the most common bias, explaining that the authors did not describe the used method to conceal the allocation sequence in detail to determine the prediction of intervention allocations in advance or during the enrolment. Our study results emphasized that the researchers should focus on preventing various types of misconduct. Therefore, observing integrity principles to prevent research misconduct is recommended. In addition, governments, institutions, and other committees need to take steps for better training and education for the researcher. The strength of this study is the quality assessment of all published systematic reviews and their understudied RCTs or quasi-RCTs, which was conducted in the field of kidney and transplantation in terms of the six-criterion risk of bias for the first time.

Study Limitations

However, our study had limitations, which include the utilization of descriptive statistics, including the frequency of all kinds of biases, to report the outcomes. In addition, the sum of all reported types of bias in understudied RCTs or quasi-RCTs included in the Cochrane kidney and transplant review group was reported. The effect of factors, such as group, year, and type of work is recommended to be examined with the Generalized Linear Models structure in future studies.

Conclusion

The high risks of performance and random sequence generation biases in understudied RCTs have critical structure deficiencies. Therefore, observing the integrity principles to prevent research misconduct is recommended.

Ethics

Ethics Committee Approval: After proposal approval and Ethics Committee confirmation of Research Deputy of Tabriz University of Medical Sciences, Tabriz, Iran (code: IRTBZMED.REC.1396.577).
Informed Consent: The quality of Cochrane kidney and transplant group systematic reviews or meta-analysis and their understudied RCTs were evaluated at the presents study, thus informed consent was not applicable.
Peer-review: Externally peer-reviewed.

Authorship Contributions

Concept: H.S.P., S.H., Design: H.S.P., Data Collection or Processing: A.Mo., Z.S., Analysis or Interpretation: A.Mo., S.H., Literature Search: A.M., L.H., Writing: Z.S., N.A.
Conflict of Interest: No conflict of interest was declared by the authors.
Financial Disclosure: The authors declare that they have no relevant financial.

References

1
Resnik DB. Editor What is ethics in research & why is it important. İdeas; 2015.
2
Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010;7:e1000326.
3
Harris JD, Quatman CE, Manring MM, Siston RA, Flanigan DC. How to write a systematic review. Am J Sports Med 2014;42:2761-2768.
4
Bauer SB. Neurogenic bladder: etiology and assessment. Pediatr Nephrol 2008;23:541-551.
5
Benevento BT, Sipski ML. Neurogenic bladder, neurogenic bowel, and sexual dysfunction in people with spinal cord injury. Phys Ther 2002;82:601-612.
6
Bianchi G, Borgonovo G, Pistoia V, Raffaghello L. Immunosuppressive cells and tumour microenvironment: focus on mesenchymal stem cells and myeloid derived suppressor cells. Histol Histopathol 2011;26:941-951.
7
Eknoyan G, Lameire N, Barsoum R, Eckardt KU, Levin A, Levin N, Locatelli F, MacLeod A, Vanholder R, Walker R, Wang H. The burden of kidney disease: improving global outcomes. Kidney Int 2004;66:1310-1314.
8
Levey AS, Eckardt KU, Tsukamoto Y, Levin A, Coresh J, Rossert J, De Zeeuw D, Hostetter TH, Lameire N, Eknoyan G. Definition and classification of chronic kidney disease: a position statement from Kidney Disease: Improving Global Outcomes (KDIGO). Kidney Int 2005;67:2089-2100.
9
El-Rashid M. The use of drug repurposing to identify new treatment opportunities for kidney injury: University of Sydney; 2020.
10
Noël D, Landais P. Epidémiologie de la maladie rénale chronique [Epidemiology of chronic kidney disease]. Rev Prat 2012;62:38-42.
11
Flores M, Rodríguez JA, Delgado A, García-Trabanino R. Prevalence and association of chronic kidney disease, diabetes, hypertension, and hyperuricemia in an adult urban population of El Salvador. Nefrología Latinoamericana 2017;14:137-143.
12
Carney EF. The impact of chronic kidney disease on global health. Nat Rev Nephrol 2020;16:251.
13
Lv JC, Zhang LX. Prevalence and Disease Burden of Chronic Kidney Disease. Adv Exp Med Biol 2019;1165:3-15.
14
Bianco P, Robey PG, Simmons PJ. Mesenchymal stem cells: revisiting history, concepts, and assays. Cell Stem Cell 2008;2:313-319.
15
Cameron AP. Pharmacologic therapy for the neurogenic bladder. Urol Clin North Am 2010;37:495-506.
16
Saran R, Robinson B, Abbott KC, Agodoa LY, Albertus P, Ayanian J, Balkrishnan R, Bragg-Gresham J, Cao J, Chen JL, Cope E, Dharmarajan S, Dietrich X, Eckard A, Eggers PW, Gaber C, Gillen D, Gipson D, Gu H, Hailpern SM, Hall YN, Han Y, He K, Hebert H, Helmuth M, Herman W, Heung M, Hutton D, Jacobsen SJ, Ji N, Jin Y, Kalantar-Zadeh K, Kapke A, Katz R, Kovesdy CP, Kurtz V, Lavalee D, Li Y, Lu Y, McCullough K, Molnar MZ, Montez-Rath M, Morgenstern H, Mu Q, Mukhopadhyay P, Nallamothu B, Nguyen DV, Norris KC, O’Hare AM, Obi Y, Pearson J, Pisoni R, Plattner B, Port FK, Potukuchi P, Rao P, Ratkowiak K, Ravel V, Ray D, Rhee CM, Schaubel DE, Selewski DT, Shaw S, Shi J, Shieu M, Sim JJ, Song P, Soohoo M, Steffick D, Streja E, Tamura MK, Tentori F, Tilea A, Tong L, Turf M, Wang D, Wang M, Woodside K, Wyncott A, Xin X, Zang W, Zepel L, Zhang S, Zho H, Hirth RA, Shahinian V. US Renal Data System 2016 Annual Data Report: Epidemiology of Kidney Disease in the United States. Am J Kidney Dis 2017;69(3 Suppl 1):A7-A8.
17
Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 2004;351:1296-1305.
18
de Groat WC, Yoshimura N. Changes in afferent activity after spinal cord injury. Neurourol Urodyn 2010;29:63-76.
19
Safarinejad MR. The epidemiology of adult chronic kidney disease in a population-based study in Iran: prevalence and associated risk factors. J Nephrol 2009;22:99-108.
20
Joannidis M, Metnitz PG. Epidemiology and natural history of acute renal failure in the ICU. Crit Care Clin 2005;21:239-249.
21
Sutherland SM, Zappitelli M, Alexander SR, Chua AN, Brophy PD, Bunchman TE, Hackbarth R, Somers MJ, Baum M, Symons JM, Flores FX, Benfield M, Askenazi D, Chand D, Fortenberry JD, Mahan JD, McBryde K, Blowey D, Goldstein SL. Fluid overload and mortality in children receiving continuous renal replacement therapy: the prospective pediatric continuous renal replacement therapy registry. Am J Kidney Dis 2010;55:316-325.
22
Wang H, Deng JL, Yue J, Li J, Hou YB. Prostaglandin E1 for preventing the progression of diabetic kidney disease. Cochrane Database Syst Rev 2010;(5):CD006872.
23
Moher D, Schulz KF, Altman DG, Group C. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Elsevier; 2001.
24
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097.
25
Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One 2009;4:e5738.
26
Rosner AL. Evidence-based medicine: revisiting the pyramid of priorities. J Bodyw Mov Ther 2012;16:42-49.
27
Khorsan R, Crawford C. How to assess the external validity and model validity of therapeutic trials: a conceptual approach to systematic review methodology. Evid Based Complement Alternat Med 2014;2014:694804.
28
Haug A, Zachariassen F, Van Liempd D. The costs of poor data quality. Journal of Industrial Engineering and Management 2011;4:168-193.
29
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6:e1000100.
30
Higgins JP. Cochrane handbook for systematic reviews of interventions version 5.0. 1. The Cochrane Collaboration. http://www cochrane-handbook org. 2008.
31
Smith J, Noble H. Bias in research. Evid Based Nurs 2014;17:100-101.
32
Buring JE. Epidemiology in medicine: Lippincott Williams & Wilkins; 1987.
33
Pannucci CJ, Wilkins EG. Identifying and avoiding bias in research. Plast Reconstr Surg 2010;126:619-625.