Overall study design
We conducted an unblinded cluster-randomized controlled trial (cluster RCT) to determine the effectiveness of the modified TaWai mobile app, combined with a face-to-face educational workshop, compared to the usual practice combined with the same workshop of intervention group, on ADR reporting in hospitals in Lao PDR. A cluster of three tertiary hospitals in Lao PDR were randomly assigned by a researcher, in a 1:1 ratio, to either the intervention group (n = 1 hospital) or the control group (n = 1 hospital). Both groups participated in two sessions of face-to-face educational workshops, each lasting 2 h, during the 4-month follow-up period from July to October 2022. This educational workshop, led by a lecturer in Lao PDR, aimed to ensure that the HCPs’ knowledge of ADRs would not interfere with their reporting. It comprised of an overview of ADRs, ADR monitoring, evaluation, and reporting; common ADRs associated with medicines used in TB and HIV patients; and the management of these ADRs. This study was conducted in accordance with Good Clinical Practice (GCP) guidelines and the Declaration of Helsinki.
Clinical setting
The target setting of this study was all departments in two tertiary hospitals in Lao PDR. However, only the TB and HIV departments of each hospital agreed to participate.
Intervention
The main intervention used in this study were the modified TaWai mobile app in addition to the aforementioned educational workshops. All HCPs in the intervention group received 30 min of training on how to use the modified TaWai mobile app before implementing this tool for ADR reporting in their hospital.
Control
Similar to the intervention group, the control group also participated in the educational workshops. However, all HCPs in this group continued to use the usual tool (paper or computer-based forms) for reporting ADR in their hospital.
Modified TaWai mobile app
Details and required information for both ADR report tools are provided in Supplementary Table 1. The modified TaWai mobile app was developed in Lao language with permission from the TaWai for Health Unit committee in Thailand. This version was designed as a draft for a chatbot to be used on mobile phones. The items in this version were adapted from the Thai version and tailored to the context of Lao PDR, incorporating the national PV guidelines, ADR reporting regulations, and relevant laws. Important ADR reporting information required by the WHO was also included. Modified sections were items related to patient information, as well as the types and sources of suspected medicines. The TaWai mobile app consists of four dimensions of ADR reporting based on WHO criteria: (1) patient information, (2) ADR information, (3) suspected medicine information, and (4) reporter information. An example of the TaWai app features is presented in Fig. 1. Although the Thai and Lao languages are similar in spoken form, their writing systems are completely different. The researchers involved in modifying and drafting the chatbot used both Thai and Lao languages in their work and daily life, demonstrating their proficiency in both languages17. The content validity of this tool was evaluated by three experts including one Ph.D. in clinical pharmacy, one Ph.D. in social and administrative pharmacy, and the head of the PV center, who also holds a Ph.D. in social and administrative pharmacy. The content validity of all items in the modified draft of chatbot was confirmed, with an Index of Content Validity (IOC) of one prior to its implementation in the mobile app.
Example of TaWai mobile app for ADRs reporting.
Outcomes measures
The primary outcomes were the total number or rate, and quality of ADR reports collected during the 4-month study period. Additionally, satisfaction and knowledge of the reporters were evaluated as secondary outcomes. Throughout the study period, the primary outcomes were collected and evaluated every month for four months, resulting in a total of five assessments, including baseline. Knowledge of each HCP was assessed by a questionnaire administered before and after the workshop sessions in both groups, while satisfaction with the TaWai mobile app was evaluated only in the intervention group.
The quality of ADR reports in this study was assessed using WHO criteria1 by a researcher. A report was considered high quality if it included all essential items according to WHO standards. Key elements for good case reports included: (1) a description of the adverse reaction or disease experience, (2) details of suspected and concomitant medicines, (3) patient characteristics, (4) documentation of the diagnosis of the reactions, including methods used, and (5) the clinical course of the reaction and patient outcomes, such as hospitalization or death. The absence of any of these elements, even a single one, automatically classifies ADR reports as low quality1.
The knowledge questionnaire comprised of five domains including the definition of ADRs and PV process, health effects, the relationship between ADRs and mortality rates or costs associated with ADRs, the importance of ADR reporting, and common ADRs related to TB and HIV drugs. The total knowledge score was 46 points, with a higher score indicating greater knowledge. Based on the criteria for evaluating knowledge by Bloom’s Taxonomy18, the scores were categorized into three levels: high knowledge (≥ 80%, or 37–46 scores), moderate knowledge (60–79%, or 28–36 scores), and low knowledge (≤ 60%, or ≤ 28 scores).
The satisfaction questionnaire was specifically developed for this study using a 5-point Likert scale. Each item was rated from 1 (very low satisfaction) to 5 (very high satisfaction). The content validity test of both satisfaction and knowledge questionnaires were approved by three experts including one specialist physician in TB infection (Head of TB Unit), one specialist physician in HIV infection (head HIV Unit), and the head of the PV center, who holds a Ph.D. in social and administrative pharmacy. The assessor’s qualifications are the people who understand Lao language, and working related with PV process or ADR reporting in Lao PDR.
Participants
The inclusion criterion for cluster units was being a tertiary hospital in Lao PDR. After randomizing the hospitals to either the intervention group or the control group, healthcare professional (HCP) teams from all departments in each randomized hospital were recruited via a formal invitation letter sent through the head of each department. Only HCP teams from the tuberculosis (TB) and human immunodeficiency virus (HIV) departments agreed to participate. HCPs aged 18 and older were included if they had worked in the TB and HIV departments for at least six months, were involved in ADR reporting, and were willing to participate in the study. HCPs were excluded if they were not familiar with a smartphone, tablet, or mobile device capable of hosting the app, could not understand or write in Lao, or refused to participate in the study.
Randomization
Since this study was designed to enhance the ADR reporting by HCPs across all ward teams in each hospital, the unit of randomization was the hospital unit in order to minimize data contamination. Based on a researcher survey conducted in 2022, there are ten government hospitals in Lao PDR. However, we selected only tertiary hospitals as the clusters for the targeted setting because this type of hospital plays a key role in ADR reporting for the PV center in Lao PDR. Therefore, three tertiary hospitals were chosen as the population clusters for this study.
Two steps of randomization were performed by a researcher who was not involved in the recruitment and data collection. Initially, two of the three tertiary hospitals were selected as sample clusters using a simple random technique with a computer randomizer. Then, these two hospitals were randomized into the intervention and control groups using the same technique. Random allocation codes were placed in sealed envelopes by a researcher who was not involved in enrollment or data collection. Two hospitals were informed of the randomization on the day of the trial initiation visit. Based on the concept of cluster randomization, all departments in each cluster hospital were assigned into the intervention and the control group following the randomization code for each hospital. Therefore, HCPs in each included tertiary hospital who met inclusion exclusion criteria mentioned above were recruited for this study.
Statistical analysis
Data analyses were performed using STATA 15.0. The intention-to-treat approach was employed for all analyses. Descriptive statistics were used to summarize the characteristics, satisfaction and knowledge of included HCPs. For between-group comparison, categorical variables such as sex, occupation, education level, the number of overall ADR reports, and high-quality reports were analyzed using the Chi-square or Fisher’s exact test. Continuous variables such as age, experience and duration of work, and knowledge scores were analyzed using the Independent t-test or Mann–Whitney U test. Two-sided p-values were calculated, with values less than 0.05 considered statistically significant. Due to the limited number of target population (HCPs involved in ADR reporting), the sample size was not estimated. Instead, we included all HCPs in each cluster who met the inclusion and exclusion criteria. Based on this criterion, approximately 65 HCPs were eligible; however, only 34 HCPs were willing to participate in the study and served as the final sample size. Subgroup analysis was performed based on department type.
Ethics statement and trial registration
The study protocol was approved by Mahasarakham University (ID: 115-074/2022), and the Lao National Ethics Committee for Health Research (ID: 065/NECHR). The study protocol was registered in the Thai Clinical Trials Registry (TCTR20220607002 available at on 07 June 2022. However, some discrepancies between the initial study protocol and the final report are provided in Supplementary Table 2. In summary, we adjusted the frequency of the workshop and the duration of follow-up due to the COVID-19 pandemic. After completing the workshop session, we had further discussions with the team and decided to reprioritize the outcomes by moving the rate and quality of ADR reports from secondary to primary outcomes, as the quality of the reports is just as important as the total number of ADR reports which is the original primary outcome. In addition, the rate of ADR reports is equivalent to the total number of reports, but they reflect different aspects of reporting, making it reasonable to move them to a primary outcome. The knowledge of HCPs was initially not predefined as a study outcome, as the goal of the pre- and post-workshop measurements was to ensure that this factor did not affect the quantity or quality of the reports. However, we decided to include it as a secondary outcome to provide greater clarity about what we measured in the study. This study was reported following CONSORT recommendations extended for cluster randomized controlled trials (Supplementary Table 3–4)19,20. Written informed consent was obtained from all participants before the enrollment procedure.
link