background of the study in practical research 2

• No category

Practical-Research-2 G12

Related documents.

Study collections

Add this document to collection(s).

You can add this document to your study collection(s)

Add this document to saved

You can add this document to your saved list

Suggest us how to improve StudyLib

(For complaints, use another form )

Input it if you want to receive answer

You are using an outdated browser. Upgrade your browser today or install Google Chrome Frame to better experience this site.

Related Resources

• —Identifying the Inquiry and Stating the Problem

PRACTICAL RESEARCH 2 – Choosing Research Topic View Download

Activity Sheets  |  DOC

Curriculum Information

Copyright Information

Technical Information

This paper is in the following e-collection/theme issue:

Published on 18.9.2024 in Vol 13 (2024)

This is a member publication of Imperial College London (Jisc)

Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study

Authors of this article:

• Ahmad Guni 1 * , BSc, MBBS   ; 
• Viknesh Sounderajah 1 * , BSc, MSc, MBBS   ; 
• Penny Whiting 2 , BA, MSc, PhD   ; 
• Patrick Bossuyt 3 , MSc, PhD   ; 
• Ara Darzi 1 , MD, PhD   ; 
• Hutan Ashrafian 1 , BSc, MBA, MBBS, PhD  

1 Institute of Global Health Innovation, Imperial College London, London, United Kingdom

2 Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom

3 Department of Epidemiology & Data Science, Amsterdam University Medical Centres, Amsterdam, Netherlands

*these authors contributed equally

Corresponding Author:

Hutan Ashrafian, BSc, MBA, MBBS, PhD

Institute of Global Health Innovation

Imperial College London

10th Floor QEQM Building, St Mary’s Hospital

London, W2 1NY

United Kingdom

Phone: 44 2075895111

Email: [email protected]

Background: Quality assessment of diagnostic accuracy studies (QUADAS), and more recently QUADAS-2, were developed to aid the evaluation of methodological quality within primary diagnostic accuracy studies. However, its current form, QUADAS-2 does not address the unique considerations raised by artificial intelligence (AI)–centered diagnostic systems. The rapid progression of the AI diagnostics field mandates suitable quality assessment tools to determine the risk of bias and applicability, and subsequently evaluate translational potential for clinical practice.

Objective: We aim to develop an AI-specific QUADAS (QUADAS-AI) tool that addresses the specific challenges associated with the appraisal of AI diagnostic accuracy studies. This paper describes the processes and methods that will be used to develop QUADAS-AI.

Methods: The development of QUADAS-AI can be distilled into 3 broad stages. Stage 1—a project organization phase had been undertaken, during which a project team and a steering committee were established. The steering committee consists of a panel of international experts representing diverse stakeholder groups. Following this, the scope of the project was finalized. Stage 2—an item generation process will be completed following (1) a mapping review, (2) a meta-research study, (3) a scoping survey of international experts, and (4) a patient and public involvement and engagement exercise. Candidate items will then be put forward to the international Delphi panel to achieve consensus for inclusion in the revised tool. A modified Delphi consensus methodology involving multiple online rounds and a final consensus meeting will be carried out to refine the tool, following which the initial QUADAS-AI tool will be drafted. A piloting phase will be carried out to identify components that are considered to be either ambiguous or missing. Stage 3—once the steering committee has finalized the QUADAS-AI tool, specific dissemination strategies will be aimed toward academic, policy, regulatory, industry, and public stakeholders, respectively.

Results: As of July 2024, the project organization phase, as well as the mapping review and meta-research study, have been completed. We aim to complete the item generation, including the Delphi consensus, and finalize the tool by the end of 2024. Therefore, QUADAS-AI will be able to provide a consensus-derived platform upon which stakeholders may systematically appraise the methodological quality associated with AI diagnostic accuracy studies by the beginning of 2025.

Conclusions: AI-driven systems comprise an increasingly significant proportion of research in clinical diagnostics. Through this process, QUADAS-AI will aid the evaluation of studies in this domain in order to identify bias and applicability concerns. As such, QUADAS-AI may form a key part of clinical, governmental, and regulatory evaluation frameworks for AI diagnostic systems globally.

International Registered Report Identifier (IRRID): DERR1-10.2196/58202

Introduction

Despite many promises, the integration of artificial intelligence (AI)–centered systems into clinical workflows has been limited thus far. In the current paradigm, diagnostic investigations require interpretation from expert clinicians in order to generate a diagnosis and subsequently determine management. However, diagnostic services across the world are overburdened with unmanageable workloads, which exceed workforce capacity [ 1 ]. In order to address this, diagnostic AI systems have been characterized by regulators and technologists as medical devices [ 2 ] that may achieve diagnostic accuracy comparable to that of an expert clinician, while concurrently alleviating health-resource use, helping to reduce medical errors. Indeed, the majority of health care–related AI systems that have reached regulatory approval belong to the field of medical diagnostics [ 3 ]. As seminal primary research studies arise on the theme of AI diagnostics [ 4 , 5 ], there has been a concomitant rise in secondary research studies that amalgamate the findings of comparable studies.

Although systematic reviews serve an important role in summarizing evidence, the vast majority related to AI diagnostic accuracy have been conducted in the absence of an AI-specific methodological quality assessment tool [ 6 ]. AI diagnostic accuracy studies are methodologically distinct from traditional diagnostic accuracy studies as they comprise distinct methods, analyses, and outcome measures that mandate specific considerations when assessing quality [ 7 ]. Currently, the most commonly used instrument for the methodological assessment of secondary research studies remains the quality assessment of diagnostic accuracy studies (QUADAS-2) tool [ 8 ]. It is a quality assessment tool designed for use in systematic reviews, initially developed in 2003 [ 9 ] and updated in 2011; its use is strongly encouraged by many biomedical journals. It consists of four key domains: (1) patient selection, (2) index test, (3) reference standard, and (4) flow and timing. These domains allow researchers to undertake a structured appraisal of a research study’s internal validity (biases) and external validity (applicability), respectively. The absence of a robust quality assessment tool in the AI field not only hinders efficient quality appraisal at an evidence synthesis phase but has considerable downstream effects as key stakeholders, such as policy makers, regulatory officials, technologists, and health care professionals, are unable to effectively evaluate the translational potential of these nascent technologies.

We propose an AI-specific extension, termed AI-specific QUADAS (QUADAS-AI), that aims to provide researchers and policy makers with a framework to appraise methodological quality in systematic reviews evaluating the diagnostic accuracy of AI. This work is complementary to the Standards for Reporting Diagnostic accuracy studies (STARD-AI) [ 10 ] and QUADAS-3 initiatives. QUADAS-AI is being coordinated by a project team and a steering committee consisting of clinician scientists, computer scientists, journal editors, Enhancing the Quality and Transparency of Health Research (EQUATOR) Network representatives, regulatory leaders, epidemiologists, statisticians, industry leaders, funders, health policy makers, legal experts, and bioethicists. Given the global reach of this class of technologies and the transformative potential in clinical diagnostics, we view that connecting global stakeholders is of the utmost importance for this initiative. This study aims to produce a novel quality assessment tool (QUADAS-AI) that accounts for the specific considerations required for the appraisal of AI diagnostic accuracy studies.

This protocol has benefitted from the experience and expertise of members of the project team and steering committee who have previously led the development of seminal quality assessment tools over the past 2 decades. These include QUADAS and QUADAS-2 for diagnostic accuracy studies, risk of bias in systematic reviews (ROBIS) for systematic reviews [ 11 ], and prediction model risk of bias assessment tool (PROBAST) [ 12 ] for prediction modeling studies. Moreover, there is shared learning from the development of AI-specific reporting guidelines and risk of bias tools, including STARD-AI [ 10 ] and PROBAST-AI [ 13 ]. The development of QUADAS-AI can be distilled into 3 broad stages, as previously delineated [ 14 ]. Given the pressing need for suitable quality assessment standards of diagnostic studies in this field, development is projected to finish by the end of 2024.

Project Organization

QUADAS-AI is being undertaken by a project team and a steering committee. The project team consists of the founder of QUADAS (PW), the lead for the STARD-AI initiative (HA), and a clinician scientist (AG). The project team is responsible for identifying members of the steering committee, candidate item generation, undertaking the online surveys for the modified Delphi consensus process, organizing the consensus meeting, drafting the QUADAS-AI tool and accompanying documents, coordinating the piloting of the draft QUADAS-AI tool, and leading the dissemination process.

The steering committee was created in order to provide diverse stakeholder guidance in this process, as well as to identify additional experts to invite for the consensus response and draft the final QUADAS-AI tool. The steering committee currently comprises approximately 15 members and consists of health care professionals, computer scientists, epidemiologists, statisticians, regulatory officials, health policy leaders, and industry leaders. These individuals were identified through their notable work in the fields of (1) diagnostic accuracy research, (2) AI in health care, and (3) applied health policy.

Defining Scope

The scope of QUADAS-AI has been defined by the project team and steering committee through a discussion framed around questions previously proposed [ 14 ]. It was predetermined that QUADAS-AI, as per previous iterations of the tool, would focus on the methodological quality of AI diagnostic accuracy studies. This study is complementary to the ongoing QUADAS-3 initiative, which is the next iteration of QUADAS and is currently led by one of the study authors and the project team (PW). If a draft of the QUADAS-3 becomes available during the development of QUADAS-AI or any substantial updates are anticipated in comparison to QUADAS-2, we will base the QUADAS-AI tool on the QUADAS-3 structure; otherwise, we will instead focus on QUADAS-2. Discourse related to the (1) assessments related to the risk of bias (internal validity), (2) assessments related to applicability (external validity), (3) tool structure, and (4) rating system is a dynamic process that will be open to adaptation throughout stage 2 of the study.

Item Generation

In order to generate a candidate list of items to enter the modified Delphi consensus process, the project team will undertake a mapping review, a meta-research study, a scoping survey with a global panel of experts, and a patient and public involvement and engagement (PPIE) exercise.

Mapping Review

A mapping review of both academic and nonacademic literature has been undertaken in order to identify key considerations in the development of QUADAS-AI. An electronic database search of MEDLINE and Embase was conducted through Ovid (Wolters Kluwer). This process was augmented by nonsystematic searches using traditional search engines for gray literature, social networking platforms, as well as personal paper collections highlighted by members of the project team. The extracted material was broadly classified into four categories: (1) general considerations regarding diagnostic accuracy studies and AI, (2) evidence and statements suggesting modifications to current items, (3) evidence and statements suggesting additions of items, and (4) evidence and statements suggesting the removal of specific items.

Meta-Research Study

As previously noted, there have been no studies examining the adherence and suitability of QUADAS-2 for the appraisal of AI diagnostic accuracy study quality. Therefore, a meta-research study was carried out to evaluate the adherence of AI diagnostic accuracy systematic reviews to the existing QUADAS-2 tool. This study demonstrated that there is incomplete uptake of quality assessment tools, as well as inconsistent reporting of bias in AI-diagnostic accuracy systematic reviews, with just over half of the studies using QUADAS-2. This study also identified key biases and features unique to AI diagnostic accuracy studies. These will contribute to the formulation of candidate items for addition or modification.

Online Scoping Survey

The project team and steering committee will undertake a survey of an international panel of experts in order to identify potential further items or modifications that warrant inclusion in QUADAS-AI. A diverse and independent panel of experts will be identified by the Project Team and Steering Committee from the various stakeholder groups outlined above. They will be provided with an information sheet describing the study and asked to participate in an online questionnaire. Participants will be asked to consider whether each item on the existing QUADAS-2 tool should be retained, removed, or modified in the QUADAS-AI tool. Free-text sections will allow participants to express their thoughts on each item as well as suggest modifications or further considerations. Furthermore, participants will be asked to comment on additional candidate items or considerations produced from preceding rounds of the item generation process.

PPIE Exercise

Finally, a focus group will be conducted with patients and members of the public who have expressed an interest in participating in forums related to digital health and AI. The objective of these discussions is two-fold: (1) to further identify issues not uncovered during previous evidence generation steps and (2) to gain further understanding of the perceived importance to the public of specific items that have been raised thus far. These discussions will be conducted remotely using Zoom (Zoom Video Communications).

An expert facilitator will lead a discussion on the current uses of AI in health care, including considerations on the aims of QUADAS-AI and the important items that the participants deem to be important to capture during the study process. As stakeholder discussions will be conducted virtually on Zoom, anonymized post hoc discussion transcripts will be retained.

Collation of Items

The project team and steering committee will group items from the item generation phase into domains and subsequently word items as signaling questions. An online discussion among the project team and members of the steering committee will be held to further refine the domains and signaling questions into a draft tool, which will then enter the Delphi consensus process for approval and refinement.

Modified Delphi Consensus Process

We will adopt a pragmatic modified Delphi consensus methodology. The Delphi consensus methodology is a well-established method of obtaining a collective opinion from a group of experts through a series of questionnaires; each one refined based on feedback from respondents on a previous version [ 15 ]. We will conduct the Delphi consensus process in a similar way as described in the STARD-AI protocol [ 10 ].

Participants from across the world are invited to join the QUADAS-AI Consensus Group on account of their expertise as clinician-scientists, computer scientists, journal editors, EQUATOR Network representatives, epidemiologists, statisticians, health technology industry leaders, funders, health policy makers, legal experts, and bioethicists. The steering committee will identify potential participants from their wider professional network or experts who have made significant contributions to their respective fields. Invited experts will be provided with a written invite detailing the study and given a 6-week timeframe to respond. Those who accept the invitation will be invited to complete each round of the modified Delphi consensus process and will be acknowledged as an author, within a group authorship model, in the publication that arises from this study. Studies of similar scope and breadth, such as STARD-AI, recruited over 150 participants from varied backgrounds across the world. A similar number is anticipated for QUADAS-AI.

During each phase of the modified Delphi consensus process, participants will use a 5-point Likert-like scale to evaluate each item (1—very important, 2—important, 3—moderately important, 4—slightly important, and 5—not at all important). The threshold for consensus will be predefined at ≥75%. Items that achieve ≥75% ratings of 1 or 2 will be put forward for discussion in the final round, which will occur in the form of an online teleconference meeting. Items that achieve ≥75% ratings of 4 or 5 will be excluded. Items that do not meet the 75% consensus threshold will advance to the next phase of the Delphi process. Participants will also have the opportunity to propose additional items that they believe warrant discussion in future rounds through open-ended responses.

In subsequent rounds, the survey will be composed of items for which consensus was not achieved and any new items suggested in prior rounds. Each item will be accompanied by a reminder of the participant’s last rating and the average rating from all participants in the prior round. This allows participants to reconsider their initial evaluations with the benefit of understanding the perspective of the wider group. Items that have not reached a consensus will be put forward for discussion in the following rounds until a consensus is reached. We will conduct descriptive statistical tests on the results for each round (median, range, mean, percentage agreement, and consensus).

Once a consensus is reached, there will be a final meeting between a small group of the project team and the steering committee to finalize the structure and content of the QUADAS-AI tool based on feedback from the Delphi consensus. The primary objective is to develop a draft version of the QUADAS-AI tool. As recommended in the Core Outcome Measures in Effective Trials (COMET) handbook, the nominal group technique, a highly structured group interaction framework, will be used to aid this process [ 16 , 17 ]. Following a brief introduction and explanation of the purpose of the meeting by the facilitators, participants will discuss the inclusion and exclusion of candidate items and share any comments until all contributions are exhausted. This discussion phase will be led by the facilitators to ensure that the discussion will not be dominated by any one individual and be as neutral as possible [ 18 ].

The first 2 rounds of the modified Delphi consensus process will be conducted as online surveys using the Delphi Manager software (version 4.0; Embarcadero Technologies ), which is developed and maintained by the COMET initiative. The final meeting to draft the QUADAS-AI tool will be conducted using Zoom. All data are pseudo-anonymized and no identifiable data will be published.

Development of the Quality Assessment Tool, Statement, and Explanation and Elaboration Document

Upon completion, the project team will construct the initial QUADAS-AI tool. The draft tool, with an accompanying statement, will be shared among the wider steering committee in order to discuss its content and, therefore, allow the steering committee to suggest additions, subtractions, or modifications as they see fit.

Piloting Among Experts and Nonexperts

Upon completion of the first draft of the QUADAS-AI tool, we intend to organize multiple rounds of piloting among expert and nonexpert users (QUADAS-AI Pilot Group). The main aim of these piloting sessions is to test the tool’s usability, as well as identify items that are considered to be vague, ambiguous, or perceived to be missing. We intend to undertake this process among health care professionals, computer scientists, expert statisticians, journal editorial boards, key industry stakeholders, regulatory leaders, as well as policy experts. Interviews among this QUADAS-AI Pilot Group will be undertaken in order to ensure that a granular level of feedback is attained for points of discussion. Members of the pilot group will not be part of the steering committee or have previously participated in the consensus process in order to provide an independent opinion. We anticipate around 20 to 30 members will be recruited. Experts and nonexperts within the Pilot Group will be acknowledged by name as author, within a group authorship model, in the publications that arise from this study.

In conjunction with this piloting process, the project team will prepare the explanation and elaboration document, to provide rationale for the domains, structure, and items associated with the tool.

Stage 3: Dissemination

Following the piloting phase, the final proposed amendments to QUADAS-AI will be discussed among the project team and the steering committee. Once consensus has been reached through email correspondence, the documents will be disseminated.

We strongly anticipate that the dissemination strategy will be principally tailored toward five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry, and (5) patient-representative bodies. Although a significant amount of material will cross over between stakeholders, creating stakeholder-specific material is considered to be the most meaningful way of achieving impact.

Academic Stakeholders

We aim to publish the QUADAS-AI tool, the accompanying statement and the explanation and elaboration document in an open-access format in a high-impact, peer-reviewed journal. In order to further complement this, we aim to create specialty-specific discourse regarding QUADAS-AI through focused editorials in pertinent journals. These journal editors will also be actively encouraged to endorse the use of QUADAS-AI as part of their peer review process. Translations of the tool in various languages are also encouraged in order to further broaden the scope of its impact. We urge interested parties to contact the corresponding author for further information about the translation policies.

Policy Stakeholders

We are in close collaboration with organizations such as Public Health England, National Health Service (NHS) Digital, National Institute for Health and Care Excellence (NICE), and the NHS Accelerated Access Collaborative (AAC) and their wider network to ensure that the tool will form part of their health technology assessment pathways.

Guidelines and Regulatory Stakeholders

QUADAS-AI has been co-designed with senior figures from the United States Food and Drug Administration (FDA) and the United Kingdom Medicines and Healthcare products Regulatory Agency (MHRA). While they do not represent the views of either organization, these steering committee members have a high-level understanding of how QUADAS-AI may be constructed to achieve maximal real-world impact.

Industry Stakeholders

We will present QUADAS-AI to a broad range of health technology companies, ranging from start-ups, small, and medium-sized enterprises to multinational corporations, so that their product pipelines may accommodate this.

Public and Nonspecific Stakeholders

Ensuring that the core material is available in an open-access fashion, through a CC-BY license, is paramount to achieving general impact. In addition, we aim to publish papers in mainstream media and attain distribution through nontraditional means (eg, social networking platforms, webinars, podcast episodes, and blog posts).

Ethical Considerations

Ethics approval for the study has been granted by the Joint Research Compliance Office at Imperial College London (21IC6664). Written consent will be gained for all participants in the online scoping survey, PPIE, Delphi consensus process, and checklist piloting.

As of July 2024, the project team and steering committee have been established, as has the scope of the project. The study is currently in the item generation phase (stage 2), and the mapping and meta-research reviews have been completed. We aim to conduct the scoping survey of experts, PPIE, and Delphi consensus process by the end of 2024 and publish the statement by the first quarter of 2025 for stakeholder use.

QUADAS-AI will be a consensus-derived quality assessment tool that will allow readers to critically appraise the risk of bias and the applicability of study findings in systematic reviews of diagnostic accuracy studies using AI. By providing a framework to evaluate the methodological quality of studies, stakeholders will be in a better position to assess the evidence base and potential for clinical translation of AI-driven diagnostic tools.

AI technology will likely be integrated into several clinical workflows within the next decade in order to enhance patient care and improve clinical outcomes. Specifically, clinical diagnostics has emerged as a key area that has gathered significant interest from global clinical, academic, and industry communities. The importance of evidence synthesis becomes increasingly evident as rapidly advancing AI technology continues to be applied within the diagnostic field; this is typically achieved with systematic reviews to draw clinically relevant conclusions from summarized findings. Therefore, robust methods to evaluate evidence synthesis will be fundamental to the clinical development and implementation of AI technologies as the research community continues to harness the unique ability of AI to generate and process ever-increasing amounts of health data. However, given the notable flaws in using current quality assessment tools, there is a pressing need to develop an AI-specific quality assessment tool that can suitably assess the unique nature of AI diagnostic accuracy studies. We hope that this international, multistakeholder consensus approach will sufficiently address the unique considerations of AI technology, and will ultimately provide a useful tool for clinical, academic, policy, regulatory, and industry stakeholders.

Acknowledgments

Infrastructure support for this research was provided by the National Institute for Health and Care Research (NIHR) Imperial Biomedical Research Centre (BRC).

Data Availability

Datasets generated or analyzed during this study are available on reasonable request from the corresponding author.

Authors' Contributions

AG, VS, PW, PB, AD, and HA developed the concept and methodology of the study. AG and VS drafted the paper. All authors read and approved the paper.

Conflicts of Interest

AD is the Executive Chair for Preemptive Health and Medicine, Flagship Pioneering. HA is the Chief Scientific Officer of Preemptive Health and Medicine, Flagship Pioneering. VC is an employee of Alphabet. All other authors declare no conflicts of interests.

• Williams BJ, Bottoms D, Treanor D. Future-proofing pathology: the case for clinical adoption of digital pathology. J Clin Pathol. 2017;70(12):1010-1018. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) action plan. US Food and Drug Administration. URL: https://www.fda.gov/media/145022/download [accessed 2024-08-22]
• Benjamens S, Dhunnoo P, Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med. 2020;3:118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577(7788):89-94. [ CrossRef ] [ Medline ]
• Milea D, Najjar RP, Zhubo J, Ting D, Vasseneix C, Xu X, et al. Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med. 2020;382(18):1687-1695. [ CrossRef ] [ Medline ]
• Jayakumar S, Sounderajah V, Normahani P, Harling L, Markar SR, Ashrafian H, et al. Quality assessment standards in artificial intelligence diagnostic accuracy systematic reviews: a meta-research study. NPJ Digit Med. 2022;5(1):11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Meskó B, Görög M. A short guide for medical professionals in the era of artificial intelligence. NPJ Digit Med. 2020;3:126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sounderajah V, Ashrafian H, Golub RM, Shetty S, de Fauw J, Hooft L, et al. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol. BMJ Open. 2021;11(6):e047709. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225-234. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51-58. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11(7):e048008. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Whiting P, Wolff R, Mallett S, Simera I, Savović J. A proposed framework for developing quality assessment tools. Syst Rev. 2017;6(1):204. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown BB. Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts. Santa Monica, CA. RAND Corporation; 1968.
• McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016;38(3):655-662. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET handbook: version 1.0. Trials. 2017;18(Suppl 3):280. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188-194. [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung; submitted 08.03.24; peer-reviewed by J Franklin, M Ansell; comments to author 14.07.24; revised version received 31.07.24; accepted 01.08.24; published 18.09.24.

©Ahmad Guni, Viknesh Sounderajah, Penny Whiting, Patrick Bossuyt, Ara Darzi, Hutan Ashrafian. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 18.09.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.

This paper is in the following e-collection/theme issue:

Published on 18.9.2024 in Vol 26 (2024)

Acceptance of Electronic Labeling for Medicinal Product Information Among Malaysian Hospital Patients: Cross-Sectional Study

Authors of this article:

Original Paper

• Xin Yee Loh 1 , PharmB   ; 
• Ai Ling Woo 2 , PharmB   ; 
• Azwa Haris 3 , PharmB   ; 
• Cheryl Shajini Pereira 3 , PharmB   ; 
• Bee Kim Tan 1, 4 , PharmB, MSc, PhD  

1 School of Pharmacy, Faculty of Health and Medical Sciences, Taylor's University, Subang Jaya, Malaysia

2 AstraZeneca Limited, Petaling Jaya, Malaysia

3 Department of Pharmacy, University Malaya Medical Centre, Kuala Lumpur, Malaysia

4 Digital Health and Medical Advancement Impact Lab, Subang Jaya, Malaysia

Corresponding Author:

Bee Kim Tan, PharmB, MSc, PhD

School of Pharmacy

Faculty of Health and Medical Sciences

Taylor's University

1 Jalan Taylors

Subang Jaya, 47500

Phone: 60 3 5629 5000

Fax:60 3 5629 5001

Email: [email protected]

Background: While perceptions of electronic labeling (e-labeling) in developed countries have been generally positive, existing data primarily come from studies involving hospital pharmacists, community pharmacy customers who may not be frequent medication users, and individuals receiving COVID-19 vaccines.

Objective: This study aims to assess e-labeling acceptance, perceptions of its benefits, challenges with its implementation, and preferences among hospital ambulatory care patients in Malaysia. Additionally, the study investigates the factors influencing patients’ acceptance of e-labeling.

Methods: A cross-sectional study using a 28-item questionnaire was conducted at the outpatient pharmacy department of a quaternary hospital in Kuala Lumpur, Malaysia, from May to June 2023. The questionnaire was developed based on a review of published literature related to e-labeling and was guided by the Unified Theory of Acceptance and Use of Technology, second version (UTAUT2). Patients aged 18 years and above were recruited using a stratified sampling method to ensure representative age-related medication usage. A mobile tablet was provided to patients for self-completion of the e-survey in their preferred language (English, Malay, or Mandarin). Categorical data on e-labeling acceptance, perceptions, and preferences were analyzed using descriptive statistics. Qualitative content analysis was performed to characterize participants’ responses to open-ended questions. Univariate and multivariate binomial logistic regression analyses were conducted to identify predictors of e-labeling acceptance.

Results: Out of 462 patients approached, 387 (83.8%) participated in the survey, with 283 (73.1%) accepting e-labeling. Most participants perceived the electronic version of the package insert as beneficial, particularly for understanding their medication better through the choice of language (352/387, 91.0%). However, around half of the participants (197/387, 50.9%) expressed concerns about the potential risks of obtaining illegal medication information via e-labeling. Most participants (302/387, 78.0%) preferred to access electronic leaflets through government websites. However, 221/387 (57.1%) still wanted the option to request printed leaflets. Significant predictors of e-labeling acceptance included perceived benefits such as better understanding of medication (adjusted odds ratio [AOR] 8.02, 95% CI 2.80-22.97, P <.001), environmental protection (AOR 7.24, 95% CI 3.00-17.51, P <.001), and flexibility in information retrieval (AOR 2.66, 95% CI 1.11-6.35, P =.03). Conversely, being of Chinese ethnicity compared with Malay (AOR 0.28, 95% CI 0.13-0.60, P =.005) and perceived lack of self-efficacy in browsing electronic leaflets (AOR 0.25, 95% CI 0.11-0.56, P <.001) were associated with lower acceptance.

Conclusions: The acceptance rate for e-labeling among hospital ambulatory care patients was moderately high and was significantly influenced by ethnicity as well as patients’ perceived benefits and challenges related to its implementation. Future strategies to enhance e-labeling uptake should address patient concerns regarding the challenges of using the digital platform and emphasize the benefits of e-labeling.

Introduction

Pouliot et al [ 1 ] defined medication literacy as an individual’s ability to make safe decisions regarding medications and health based on the processing of patient-centered medication information (eg, written, oral, and visual). Patients with limited medication literacy often struggle with reading medication labels, understanding printed care instructions and health advice, and tend to use medications inappropriately. They are also less adherent to therapy [ 2 , 3 ]. The National Health Morbidity Survey Malaysia 2019 reported that 1 in 3 adult Malaysians has poor health literacy [ 4 ]. The high prevalence of poor health literacy among the public is concerning, as those with low health literacy are more likely to incur higher health care costs, placing a tremendous burden on the health care system. Despite health care professionals (HCPs) conveying medication information verbally, most patients have limited cognitive ability to retain orally transmitted information [ 5 ]. Therefore, medicinal product information leaflets can serve as a useful aid, in addition to verbal counseling, to address postconsultation gaps [ 6 ]. In Malaysia, there are 2 categories of health authority–approved medicinal product information: Package Inserts (PIs) and Consumer Medication Information Leaflets (RiMUP) [ 7 ]. RiMUP is written in layman’s terms and is available in both English and Malay. While PIs are legally required to be printed and enclosed with all products containing scheduled poisons and injectable over-the-counter medicines, distributing RiMUP with the product is optional.

Patients who receive and read medicinal product information leaflets are more likely to discuss their medications with their health care providers. This creates an opportunity for providers to use the leaflets as educational material to improve treatment knowledge and facilitate shared decision-making [ 8 ]. A majority (64.9%) of participants in a Saudi Arabian study reported that their medication adherence improved after reading medicinal product information leaflets [ 9 ]. However, 55% of respondents in a survey conducted by the European Association of Hospital Pharmacists revealed that hospital patients do not receive medicinal product information leaflets [ 10 ]. Furthermore, patients reported several issues with paper PIs that hinder their effective use, such as undersized fonts, medical jargon, and the lack of options in local languages [ 9 , 11 , 12 ].

Electronic labeling (e-labeling) is an emerging trend in disseminating legally approved medicinal product information in dynamic formats, such as XML. By leveraging digital advancements, e-labeling systems enable the use of personalized medication information to meet the specific needs of both patients and HCPs. In today’s digital age, as the public increasingly seeks online health information, e-labeling offers a convenient way to access regulatory-approved medicinal product information through a trusted channel [ 13 ]. E-labeling not only streamlines the updating process, enabling prompt dissemination of medicinal product information to a wide range of HCPs and patients, but also creates opportunities for integration into digital health services. This can support e-prescribing by reducing the risk of medication incompatibilities and enhancing patient safety. From the industry’s perspective, electronic provision of medicinal product information reduces logistical challenges in label updates, lowers printing costs for PIs, and improves the efficiency of the global pharmaceutical supply chain through shared labeling between countries. These advantages of e-labeling collectively contribute to achieving the United Nations Sustainable Development Goals (UNSDG) 3 (Good health and well-being) and 12 (Responsible consumption and production) [ 14 ].

To date, e-labeling is regulated at varying levels across different regions worldwide. The United States Food and Drug Administration (FDA) has mandated the electronic distribution of PIs since 2015 [ 15 ]. By contrast, only selected hospital medicinal products in the Baltic countries have been granted marketing authorization through e-labeling [ 12 ]. In Asia, Japan’s Pharmaceuticals and Medical Devices Agency (PMDA) officially enforced the removal of paper PIs for prescription drugs and medical devices in July 2023, aiming to transition toward a paperless system [ 16 ]. Malaysia’s National Pharmaceutical Regulatory Agency (NPRA) released the Guideline on Electronic Labelling (E-labeling) for Pharmaceutical Products, which came into effect on May 1, 2023. According to this guideline, approved PIs, RiMUPs, or both must be provided electronically via a machine-readable QR code on the product’s outer carton or inner label, linking to the NPRA QUEST system. The implementation of e-labeling is voluntary and applies to newly registered pharmaceutical products, biologics, and generic products containing scheduled poisons. An extension to other categories of products is still under review [ 17 ].

Understanding patients’ acceptance, perception, and preferences regarding e-labeling can help implement a more patient-centric approach to foster engagement. Such data are lacking in Malaysia, a developing country in Southeast Asia with a multiethnic population and a unique socioeconomic context that could influence the public’s readiness to adopt new digital health services. While perceptions of e-labeling in developed countries are generally positive, concerns have been reported, particularly among older adults and those with low digital literacy. Moreover, data representing patients were primarily obtained from HCPs [ 10 ], customers visiting community pharmacies who may not be frequent medication users [ 11 ], and individuals receiving COVID-19 vaccines [ 18 ]. These data cannot be generalized to hospital ambulatory care patients, who are primarily managing chronic diseases and require ongoing medication information for self-administration. Therefore, this study aimed to assess e-labeling acceptance, perceptions of its benefits, challenges with local implementation in Malaysia, and preferences among hospital ambulatory care patients. Additionally, the factors influencing patients’ acceptance of e-labeling were investigated.

Study Design and Population

A cross-sectional study was conducted at the University Malaya Medical Centre (UMMC), a quaternary teaching hospital with 1617 beds and multidisciplinary clinics spanning 40 clinical specialties. Established in 1962 and located in Kuala Lumpur, the capital of Malaysia, UMMC served as the study site. The study population consisted of a convenience sample of patients who visited the outpatient pharmacy department of UMMC. The inclusion criteria were patients aged 18 years and older; collecting prescription medications; and capable of reading English, Malay, or Mandarin. Exclusion criteria were patients with limited cognitive abilities, those collecting medication on behalf of others, those who were unwell, or those who refused to participate.

The calculated minimum sample size was 386, based on government hospital outpatient statistics from 2020, which totaled 16,635,350 [ 19 ]. This calculation used a 5% margin of error, a confidence level of 95%, and a response distribution of 50% [ 20 ]. A stratified sampling method was used based on the estimated proportions of prescription drug users: 18.0% under 40 years of age, 46.0% between 40 and 64 years of age, and 85.0% over 65 years old, to ensure age representativeness [ 21 ].

Questionnaire

The study instrument was a 28-item questionnaire developed based on a review of the published literature related to e-labeling and informed by the Unified Theory of Acceptance and Use of Technology, second version (UTAUT2) [ 22 ]. The questionnaire consisted of 3 sections/pages: (A) demographics (4 items) and utility of medicinal product PIs (6 items), (B) perceptions of the benefits (6 items) and challenges with e-labeling implementation (5 items), and (C) acceptance (1 item) and preferences regarding e-labeling (6 items). Acceptance, as defined by Adell et al [ 23 ], is the willingness to use a system based on theoretical knowledge or experience.

The demographic information of the participants included age, gender, ethnicity, and education level. Age was categorized into 3 groups: 18-39, 40-64, and 65 and older. Ethnicity was categorized into Malay, Chinese, Indian, and other, while education level was classified as university/college, secondary school, primary school, and no formal schooling. The utility characteristics of medicinal product PIs captured included the sources of written medicine information, reasons for choosing these sources, and the frequency and reasons for reading PIs. Participants’ practices regarding the frequency of using PIs were rated as follows: always, sometimes, only when receiving new medication, or never. Acceptance, perceived benefits (such as ease of retrieval, medication understanding, personalization, up-to-date information, and environmental protection), perceived challenges (including issues with electronic gadgets, digital literacy, internet access, and label security), and preferences regarding e-labeling (such as format, access, and options) were assessed using a 5-point Likert scale, ranging from 1=strongly disagree to 5=strongly agree. The questions were presented in a choice format, except for 2 open-ended questions designed to elicit reasons for not reading PIs and to gather additional views from participants regarding e-labeling.

The content validity of the questionnaire was assessed by 6 subject matter experts using a 4-point Likert scale to evaluate the relevance of each survey item. The experts included 2 regulatory pharmacists, 2 hospital pharmacists, and 2 academic pharmacists. The degree of relevance was categorized into 2 groups: “not relevant” and “somewhat relevant” were considered as “0=irrelevant,” while “quite relevant” and “very relevant” were considered as “1=relevant.” The scale-level content validity index based on the average method and the universal agreement method were 0.97 and 0.85, respectively, meeting the satisfactory level (≥0.83). The wording of some questions and choices was modified following a discussion within the research team based on the feedback received.

The English version of the questionnaire was translated into Malay and Mandarin using forward and backward translation methods. The translation was performed by 2 native Malay speakers proficient in English. The translations were reviewed by the research team and reconciled into an optimal version based on the appropriateness of the wording. This reconciled version was then back-translated into English by 2 additional native Malay speakers with a strong command of English. The original English version of the questionnaire and its translations were compared by the research team, and any discrepancies were discussed ( Multimedia Appendix 1 ). Revisions were made to the Malay version of the questionnaire as needed ( Multimedia Appendix 2 ). A similar translation method was used for the Chinese version of the questionnaire ( Multimedia Appendix 3 ).

To ensure the feasibility of the recruitment procedure and the face validity of the questionnaire, a pilot study was conducted with 30 participants (10 for each language: English, Malay, and Mandarin) through cognitive debriefing to assess clarity and understanding. Cronbach α coefficients were calculated, resulting in .73 for section B on perceived benefits and .79 for challenges with e-labeling implementation. The usability and technical functionality of the electronic questionnaire were tested before the pilot study.

Data Collection

Data collection took place from May to June 2023 at the outpatient pharmacy department of UMMC. Potential participants were approached by XYL and invited to participate while waiting for their prescriptions to be filled. Those who agreed to participate were briefed on the study objectives and the estimated time required to complete the survey. A tablet was provided to each participant to indicate their informed consent, followed by the self-completion of the e-survey form in their preferred language (English, Malay, or Mandarin). The survey was designed to be open, allowing participants to review and change their answers using a back button. Completeness was ensured before survey submission through mandatory items in the e-survey form. Participation was voluntary, and participants could opt out without facing any negative consequences. No incentives were provided. Participants’ data were anonymized and stored in a password-protected file.

Data Analysis

Statistical analysis was performed using the SPSS software (version 27; IBM Corp.). Descriptive statistics, including frequencies and percentages, were generated for all categorical variables. To facilitate analysis, responses for acceptance, perceived benefits, perceived challenges, and preferences were dichotomized into 2 categories: “strongly disagree,” “disagree,” and “neutral” were classified as “No,” while “agree” and “strongly agree” were classified as “Yes.” Univariate logistic regression was used to test the effect of each independent variable (demographic characteristics, utility of medicine PI, perceived benefits, and perceived challenges) on the probability of acceptance of e-labeling. Covariates with P <.25 were selected [ 24 ] and subsequently tested in a multivariate logistic regression model to identify significant predictors of e-labeling acceptance. P values <.05 in the multivariate logistic regression model were considered statistically significant.

A qualitative content analysis was performed following the 8 steps outlined by Zhang and Wildemuth [ 25 ] to characterize participants’ responses to the open-ended questions. The procedures included the following: (1) Importing participants’ response text data into qualitative data analysis software (NVivo version 10; Lumivero). (2) Coding data related to participants’ reasons for not reading the PI and their opinions on e-labeling implementation. (3) A coding scheme and a list of initial categories were developed using the constant comparison method. (4) To validate the coding scheme and ensure consistency, 2 researchers (XYL and BKT) independently coded the data from the first 5 participants. The coding by both researchers was found to be in agreement. (5) XYL then coded the remaining data and added new categories as needed. (6) BKT assessed the coding consistency against the raw data. (7) The categories/themes were refined based on the patterns observed in the coded data. Homogeneity of codes within each category and heterogeneity of codes across categories were reviewed to ensure there was no overlap; and finally, (8) the inductive content analysis process and results were reported descriptively.

Ethics Approval

The study was granted ethics approval by the Medical Ethics Committee of UMMC (MREC ID Number: 2023214-12138, dated April 3, 2023), conducted in accordance with the Declaration of Helsinki, and reported according to the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) checklist [ 26 ].

Participant Demographics and Characteristics of Package Insert Use

Out of the 462 patients approached, 387 agreed to participate and completed the e-survey, resulting in a response rate of 83.8%. Participant demographics and characteristics of PI use are summarized in Tables 1 and 2 . Participants were predominantly male (n=202, 52.2%), Chinese (n=185, 47.8%), aged 40-64 years (n=173, 44.7%), and had a university/college education (n=289, 74.7%). Among the 387 participants, more than three-quarters (n=312, 80.6%) reported seeking written information about their medication. PI was the second most popular source of written medicine information, with a utility rate of 34.6% (n=108), following the internet (n=188, 60.3%). By contrast, only 1.6% (n=5) of participants read the RiMUP published on the NPRA website. The internet was perceived as more readily accessible (162/299, 54.2%, vs 76/176, 43.2%) and easier to understand (98/299, 32.8%, vs 48/176, 27.3%) compared with PI, but was considered less trustworthy (16/299, 5.4%, vs 47/176, 26.7%). Most participants read the PI only when they received a new medication (n=125, 40.1%).

Among the 48 responses to an open-ended question, reasons for not reading the PI included the leaflets being voluminous and containing too much information (n=15, 31.2%), small font size that is hard to read (n=10, 20.8%), difficulty understanding medical terms (n=9, 18.7%), preference for Google due to convenience (n=6, 12.5%), the paper being too small (n=5, 10.4%), and already being well-informed by doctors (n=3, 6.2%). Conversely, side effects (228/824, 27.7%) and information on the medication’s purpose and how it works (224/824, 27.2%) were the main reasons for reading the PI ( Tables 1 and 2 ).

a Participants can choose more than 1 reason for the chosen source of written information about medicine.

b RiMUP: Consumer Medication Information Leaflets.

c NPRA: National Pharmaceutical Regulatory Agency.

Perceived Benefits and Challenges With e-Labeling Implementation

Most participants strongly agreed or agreed that the electronic version of the PI is beneficial, with 352/387 (91.0%) appreciating the ability to understand their medication better through their preferred language, 348/387 (89.9%) valuing the inclusion of images and videos, and 344/387 (88.9%) benefiting from advanced features such as adjustable font size and keyword search. Participants also agreed that e-labeling could help protect the environment by reducing paper use (340/387, 87.9%); provide the most up-to-date medication information (325/387, 84.0%); and allow access to information anywhere, anytime, without the fear of losing it (325/387, 84.0%; Figure 1 ).

At the same time, around half of the participants (197/387, 50.9%) were concerned about obtaining potentially illegal medication information via e-labeling. A minority of participants expressed concerns about limited skills in browsing electronic medicinal product information (70/387, 18.1%), limited skills in using electronic gadgets (39/387, 10.1%), limited internet access (27/387, 7.0%), and not owning electronic gadgets (23/387, 5.9%; Figure 2 ).

Acceptance of e-Labeling and Influencing Factors

Overall, the participants’ acceptance rate of e-labeling was moderately high at 283/387 (73.1%; Figure 3 ). Univariate regression analysis revealed that all independent variables—including demographic characteristics, utility of PI, perceived benefits, and perceived challenges with e-labeling implementation—were potential factors associated with e-labeling acceptance ( P <.25; Table 3 ).

Using a forward stepwise elimination method, multivariate regression analysis ( Table 3 ) showed that participants who perceived a benefit in understanding medication better through images and videos were 8 times more likely to accept e-labeling (adjusted odds ratio [AOR] 8.02, 95% CI 2.80-22.97, P <.001). Those who perceived a benefit in using a paperless system to protect the environment had a 7 times higher probability of acceptance (AOR 7.24, 95% CI 3.00-17.51, P <.001) and those who perceived a benefit in being able to retrieve information anywhere, anytime, and without fear of losing it had 2 times the likelihood of accepting e-labeling (AOR 2.66, 95% CI 1.11-6.35, P =.03). By contrast, Chinese ethnicity was associated with a 72% lower probability of accepting e-labeling compared with Malay ethnicity (AOR 0.28, 95% CI 0.13-0.60, P =.005). Participants who perceived limited skills in browsing electronic medicinal product information were 75% less likely to accept e-labeling (AOR 0.25, 95% CI 0.11-0.56, P <.001). The binary logistic regression model was statistically significant ( χ 2 6 =49.285, P <.001). The model explained 39.3% of the variance in e-labeling acceptance (Nagelkerke R 2 ). The Hosmer and Lemeshow test indicated that the model was a good fit for the data ( P =.21, >0.05). Overall, the model had a good accuracy rate of 84% and exhibited excellent sensitivity (96.6%) in predicting e-labeling acceptance.

a N/A: not applicable.

b P <.25.

c P <.05.

Preference Toward e-Labeling

Most participants preferred accessing electronic medicinal product information through official or government websites (302/387, 78.0%). Participants also showed interest in scanning a digital code, such as a QR code printed on the outer medication package (282/387, 72.9%), or accessing information through digital patient services, such as medication apps (282/387, 72.9%), compared with receiving a link via SMS text message or email (194/387, 50.1%). However, 221/387 (57.1%) of participants still preferred the option to request a printed copy of the medicinal product information ( Figure 4 ).

In response to the open-ended question about views on e-labeling implementation for medicinal product information, most participants (33/83, 40%) emphasized that the e-labeling platform should consistently provide updated medication information that is neutral and free from product advertisements. They also highlighted the importance of the platform being easily accessible, user-friendly, easy to understand, and compatible with various electronic devices. Some participants also expressed that the content of the electronic label (e-label) must be reliable and protected from third-party modifications or cybersecurity attacks to ensure it is safe for patient use (7/83, 8%). Suggestions included accessing e-labeling through hospital websites verified by competent authorities. Additionally, participants recommended features for the e-labeling platform, such as the ability to compare information across medications for the same indication, separate sections for medication information on different diseases to facilitate easy location, links to journal or research articles, a section for user feedback, and a notification function to alert patients about new updates (6/83, 7%). As e-labeling for medicinal product information is a new initiative, participants also felt that a helpline should be available for patients needing assistance (3/83, 4%).

Despite the generally positive perception of e-labeling for medicinal product information, some participants expressed concerns about certain populations, including older adults, individuals with low digital literacy, those without internet access, and those without electronic devices (23/83, 28%). They suggested that it might be necessary to provide both paper and electronic inserts and recommended that authorities implement the e-labeling initiative in phases to allow the public time to adapt to the new platform (10/83, 12%).

Principal Findings and Comparison With Prior Work

We found a moderately high acceptance rate (283/387, 73.1%) for e-labeling among hospital patients, with more than half (221/387, 57.1%) preferring to retain the option to request a printed copy. Most participants viewed the electronic version of the PI as beneficial, especially for understanding their medication better through language choices (352/387, 91.0%). However, around half of the participants (197/387, 50.9%) were concerned about the potential risk of accessing illegal medication information via e-labeling. Most participants preferred accessing e-labels from trusted sources such as government websites (302/387, 78.0%). Acceptance of e-labeling was significantly influenced by patients’ perceptions of benefits, including a better understanding of medication, environmental protection, and flexibility in information retrieval. By contrast, patients of Chinese ethnicity and those who perceived limited skills in using electronic inserts were less likely to accept e-labeling.

Compared with older studies, the acceptance rate for e-labeling in our study was higher. For example, a study conducted in Sweden before the pandemic reported that only 41% of 406 customers surveyed in community and hospital pharmacies were interested in using electronic medicinal product information. Additionally, 54% of respondents indicated they would request a printed version from the pharmacy if the paper leaflet was not included in the package [ 11 ]. During the pandemic, a survey of 2518 vaccine recipients or their parents across 4 European countries (Belgium, Italy, Bulgaria, and France) reported an acceptance rate for electronic leaflets ranging from 55% to 82%, with an overall acceptability of 64% when a printed leaflet option was available [ 18 ].

Our patients’ perception of e-labeling as enhancing their understanding of medication aligns with findings from a Saudi Arabian study, where patients reported that reading medicinal product information leaflets positively impacted their knowledge about medicines and medication adherence [ 9 ]. However, only 1.6% (5/312) of participants in our study who obtained written medicine information used RiMUP, in contrast to the 91.1% utility rate of patient information leaflets observed in the Saudi Arabian study. This discrepancy may be attributed to the fact that RiMUPs are not distributed with products but are instead available as PDFs on the Malaysian NPRA website. HCPs are responsible for retrieving, printing, and disseminating them to patients if needed. Similar to experiences in Australia, this practice has not led to widespread dissemination of RiMUP as intended [ 27 ]. In our study, patients’ perceptions of the convenience of accessing e-labeling anytime and anywhere, as well as the ease of information retrieval, align with their primary source of medical information—the internet. Studies have shown that the availability of the internet has increased the use of online sources for medication information [ 11 , 18 , 28 ].

Malaysian patients have shown support for transitioning from paper medicinal product PIs to e-labeling for several reasons. First, there is widespread awareness of the negative impact of paper consumption associated with printing paper inserts [ 29 ]. This awareness is likely influenced by frequent media reports on extreme weather events and the broader effects of deforestation on climate change, which have heightened public concern about environmental issues. Second, the public adopted new health behaviors during the pandemic, which required transitioning many occupational and social activities to online platforms as a preventive measure against COVID-19 transmission [ 30 ]. Malaysians adapted to paperless systems such as QR codes and mobile apps, which explains the high preference for digital code scanning and medication apps among patients [ 31 ]. As socioeconomic activities resumed in the postpandemic period, this practice has become the new norm. By contrast, receiving a link to electronic medication information was the least favored option among patients. This reluctance may be attributed to the rising incidence of scams in Malaysia in recent years, which has made patients wary of clicking on links [ 32 ]. Additionally, several nationwide digitalization programs, such as the paperless road tax and online passport renewal policies recently introduced by the Malaysian government, have increased public acceptance of digital services [ 33 ]. This aligns with the mission of Malaysia’s National Fourth Industrial Revolution (4IR) policy, which aims to leverage digital technology to transform the economy in line with the Shared Prosperity Vision of creating a fair, equitable, and inclusive society by 2030 [ 34 ]. Additionally, the low perceived challenges related to digital gadget ownership, usage, and internet access may be due to ongoing income tax exemptions on laptops and the incentives promoting smartphone and laptop ownership. These measures have contributed to the public’s high readiness to adopt the e-labeling platform [ 35 ].

Paper leaflets for medicinal product information have an unavoidable environmental footprint, and shifting to electronic versions can significantly reduce production costs [ 12 , 16 , 28 , 36 ]. Additionally, features such as zooming and search functions on electronic devices make it easier and faster for patients to locate information [ 11 , 28 , 36 , 37 ]. These functionalities address the limitations of paper inserts and enhance the overall patient experience in managing their medication. This shift to e-labeling could potentially encourage patients who were previously hesitant to use medicinal product information leaflets to view electronic formats as a reliable source of information. Additionally, participants in our study suggested that the e-labeling system should present information in a comparative format across different drugs with the same indications and link to credible sources such as journals or research articles. This indicates that Malaysian patients are eager to learn about their medications and take an active role in managing their treatment. Providing patient-centric medicinal information in local languages can enhance medication literacy. Ultimately, e-labeling has the potential to improve medication use and lead to better health outcomes.

In our study, patients considered the legitimacy of the e-labels as a crucial aspect of the e-labeling system. Most patients preferred accessing e-labels through trusted platforms, such as government or official websites. This preference is likely due to the prevalent cybersecurity issues in Malaysia [ 38 ]. Consequently, patients emphasized the importance of maintaining system security to mitigate the risk of biased information that could impact patient safety. Our findings suggest that patients’ perceived limited skills in browsing e-labels correlate with lower acceptance of e-labeling, a phenomenon explained by Bandura’s Theory of Self-Efficacy [ 39 ]. According to this theory, individuals who feel confident in their ability to use the e-labeling platform are more likely to engage with and accept the technology. Consequently, providing a helpline for patients could facilitate their adaptation to the e-labeling system. Despite the generally high acceptance of e-labeling, 221/387 (57.1%) participants preferred not to completely eliminate paper inserts, a preference consistent with previous studies [ 11 , 18 ]. Therefore, it is important to implement procedures that support patients with limited digital skills until the e-labeling platform is fully established and effective.

Our study found that acceptance of e-labeling was lower among Chinese patients. Currently, the RiMUP is not available in Chinese or Tamil, the 2 major languages in our region of Malaysia, which may have impacted Chinese patients’ perceptions of e-labeling. Further research is needed to explore the underlying reasons for ethnic discrepancies in e-labeling acceptance among Malaysians. In our study, age was a significant factor in the univariate analysis but not in the multivariate analysis. This finding contrasts with Hammar et al [ 11 ], which suggested that older age might hinder the adoption of electronic patient information leaflets. The perceived lack of digital literacy skills among patients, which could be a more relevant factor, may have been reflected in our study, thereby minimizing age as a potential confounder. Similar results were reported in a recent European vaccine study that focused on individuals aged over 60 years [ 18 ].

Implications for Policies and Strategies

Future strategies to enhance patient uptake of e-labeling should address concerns about the challenges associated with using digital platforms for medical information. First, flexibility should be provided to allow patients to request a printed copy of medicinal product information leaflets when necessary. Second, public awareness campaigns could encourage individuals who have not previously utilized medicinal product information leaflets to start using e-labeling, thereby increasing overall engagement. Patients should be informed about the benefits of e-labeling, including personalized information, enhanced medication safety, improved supply chain efficiency, and environmental protection. Educational materials should be provided in common local languages to ensure accessibility for patients from diverse backgrounds. Third, to help patients adapt to the e-labeling platform, practical demonstration videos with simple, clear instructions and visual aids can be displayed in pharmacy waiting areas to create a positive learning experience. Fourth, periodic reviews of the e-labeling system should be conducted to ensure it remains user-friendly and compatible with various electronic devices. Features such as linking e-labels to research studies and journals, enabling medication comparisons for the same indications, and including a section for public feedback can enhance the platform’s patient-centric approach. Fifth, a robust process should be established to update information in a centralized database. The responsible authority must verify and ensure the accuracy of the data before they are made available to the public. Lastly, a helpline should be provided to offer patients assistance whenever needed.

Strength and Limitations

To the best of our knowledge, this is the first study assessing patient acceptance of e-labeling for medicinal product information during the postpandemic transition. The study also identified factors influencing patients’ acceptance of e-labeling. A stratified sampling method was used to ensure the sample accurately represents the patient population’s distribution in terms of age-related medication usage.

This study has several limitations. Conducted in a single hospital pharmacy located in Kuala Lumpur, a highly urbanized and densely populated city in Malaysia, the findings may not be generalizable to suburban or rural populations elsewhere in the country. Patients in rural areas may have limited internet access, which could influence their acceptance of e-labeling. Currently, the Malaysian government is working with the industry to enhance internet connectivity as part of the Madani Economy framework, aiming to provide stable and affordable internet access to Malaysians across all regions [ 40 ]. The proportion of participants with higher education levels in this study was greater than that observed in the national population census, which may introduce bias, as higher education is often linked with better economic status, ownership of digital devices, and proactive health information–seeking behavior [ 41 ]. However, the results from the multivariate analysis indicated that education level was not a significant predictor of e-labeling acceptance.

Conclusions

Malaysian hospital patients demonstrated a moderately high level of acceptance of e-labeling of medicinal product information. Key factors predicting high acceptance included perceived benefits such as improved understanding of medication, environmental protection, and flexibility in information retrieval. By contrast, lower acceptance was associated with being of Chinese ethnicity and having perceived limitations in digital self-efficacy. Future strategies to enhance e-labeling uptake should focus on addressing patients’ concerns about digital platform challenges and emphasizing the advantages of e-labeling.

Availability of Data and Materials

The data sets used and analyzed in this study are available from the corresponding author on reasonable request.

Acknowledgments

The authors express their gratitude to all UMMC patients who have participated in the study and appreciate Ms Wan Lee Chow, Dr Jason Loo Siau Ee, and Mr Selvarajah Seeragam who were involved in the validation of the questionnaire. We also thank AstraZeneca Sdn Bhd for funding this study in accordance with the Good Publication Practice Guideline 2022. We acknowledge AstraZeneca Sdn Bhd and Taylor’s University for open-access funding of this publication.

Authors' Contributions

BKT, XYL, ALW, AH, and CSP conceived and designed the study. XYL collected the data. XYL and BKT analyzed the data. XYL drafted the manuscript. BKT, ALW, AH, and CSP critically revised the manuscript for important intellectual content and approved the final version for publication. AstraZeneca did not have any role in decision-making or in the preparation of consensus guidelines, if any.

Conflicts of Interest

None declared.

Questionnaire: Acceptance of electronic labeling among hospital ambulatory patients (English).

Questionnaire: Acceptance of electronic labeling among hospital ambulatory patients (Malay).

Questionnaire: Acceptance of electronic labeling among hospital ambulatory patients (Mandarin).

• Pouliot A, Vaillancourt R, Stacey D, Suter P. Defining and identifying concepts of medication literacy: an international perspective. Res Social Adm Pharm. Sep 2018;14(9):797-804. [ CrossRef ] [ Medline ]
• Pourhabibi N, Mohebbi B, Sadeghi R, Shakibazadeh E, Sanjari M, Tol A, et al. Determinants of poor treatment adherence among patients with type 2 diabetes and limited health literacy: a scoping review. J Diabetes Res. 2022;2022:2980250. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Caruso R, Magon A, Baroni I, Dellafiore F, Arrigoni C, Pittella F, et al. Health literacy in type 2 diabetes patients: a systematic review of systematic reviews. Acta Diabetol. Jan 2018;55(1):1-12. [ CrossRef ] [ Medline ]
• Ganapathy S, Aris H, Ahmad N. National Health and Morbidity Survey. Shah Alam, Selangor, Malaysia. National Institute of Health; 2019.
• Bekker CL, Mohsenian Naghani S, Natsch S, Wartenberg NS, van den Bemt BJF. Information needs and patient perceptions of the quality of medication information available in hospitals: a mixed method study. Int J Clin Pharm. Dec 2020;42(6):1396-1404. [ CrossRef ] [ Medline ]
• Young A, Tordoff J, Leitch S, Smith A. Patient-focused medicines information: general practitioners’ and pharmacists’ views on websites and leaflets. Health Education Journal. Nov 14, 2018;78(3):340-351. [ CrossRef ]
• National Pharmaceutical Regulatory Agency (NPRA). Drug Registration Guidance Document (DRGD). NPRA Official Portal. 2023. URL: https:/​/www.​npra.gov.my/​easyarticles/​images/​users/​1047/​DRGD%20Jan%202023/​Complete-Drug-Registration-Guidance-Document-DRGD-3rd-Edition-4th-Revision-January-2023.​pdf [accessed 2024-09-04]
• Wongtaweepkij K, Corlett S, Krska J, Pongwecharak J, Jarernsiripornkul N. Patients' experiences and perspectives of receiving written medicine information about medicines: a qualitative study. Patient Prefer Adherence. 2021;15:569-580. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Al Jeraisy M, Alshammari H, Albassam M, Al Aamer K, Abolfotouh MA. Utility of patient information leaflet and perceived impact of its use on medication adherence. BMC Public Health. Mar 14, 2023;23(1):488. [ FREE Full text ] [ CrossRef ] [ Medline ]
• European Association of Hospital Pharmacists (EAHP). EAHP survey on the future potential of electronic product information (ePI). EAHP. Feb 22, 2022. URL: https://www.eahp.eu/practice-and-policy/ehealth-and-mhealth/ePIsurvey [accessed 2024-09-04]
• Hammar T, Nilsson A, Hovstadius B. Patients' views on electronic patient information leaflets. Pharm Pract (Granada). 2016;14(2):702. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sirkas K, Juppo A, Miettinen M, Siven M. Could paper package leaflet be left out from hospital products? Explor Res Clin Soc Pharm. Sep 2022;7:100176. [ CrossRef ] [ Medline ]
• Use of electronic labeling. IFPMA. Feb 7, 2022. URL: https://www.ifpma.org/publications/position-paper-use-of-electronic-labeling/ [accessed 2024-09-04]
• United Nations. Sustainable development goals. UNDP. URL: https://www.undp.org/sustainable-development-goals [accessed 2024-09-04]
• Chaudhary P, Shetty V. E-labeling: change is underway. PharmExec. URL: https://www.pharmexec.com/view/e-labeling-change-underway [accessed 2024-04-09]
• Matsui R, Yamaguchi K, Lee JJV, Ting I, Khairilisani D, Chang J, et al. Survey result for E-labeling initiatives in Asia. Ther Innov Regul Sci. Mar 2023;57(2):251-260. [ FREE Full text ] [ CrossRef ] [ Medline ]
• National Pharmaceutical Regulatory Agency (NPRA). Guideline on Electronic Labelling (E-labeling) for Pharmaceutical Products in Malaysia. NPRA. Apr 2023. URL: https:/​/npra.​gov.my/​easyarticles/​images/​users/​1047/​Lampiran-A-Guideline-on-E-labeling-for-Pharmaceutical-Products-in-Malaysia.​pdf [accessed 2024-09-04]
• Bamberger M, De Loof H, Marstboom C, Oury S, Bonanni P, Launay O, et al. Replacing vaccine paper package inserts: a multi-country questionnaire study on the acceptability of an electronic replacement in different target groups. BMC Public Health. Jan 24, 2022;22(1):156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ministry of Health Malaysia. Health facts 2021. Ministry of Health Malaysia. Nov 2021. URL: https://www.moh.gov.my/index.php/pages/view/58?mid=19 [accessed 2024-09-04]
• Sample size calculator by Raosoft, Inc. Raosoft. URL: http://www.raosoft.com/samplesize.html [accessed 2024-09-04]
• Martin CB, Hales CM, Gu Q, Ogden CL. Prescription drug use in the United States, 2015-2016 (NCHS data brief, no 334). CDC. Hyattsville, MD. National Center for Health Statistics; 2019. URL: https://www.cdc.gov/nchs/products/databriefs/db334.htm [accessed 2024-09-04]
• Venkatesh, Morris, Davis, Davis. User acceptance of information technology: toward a unified view. MIS Quarterly. 2003;27(3):425. [ CrossRef ]
• Adell E, Várhelyi A. Driver Acceptance of New Technology. Theory, Measurement and Optimisation. London, UK. Taylor and Francis Group; Jun 12, 2014.
• Bursac Z, Gauss CH, Williams DK, Hosmer DW. Purposeful selection of variables in logistic regression. Source Code Biol Med. Dec 16, 2008;3:17. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Y, Wildemuth BM. Qualitative Analysis of Content. In: Wildemuth BM, editor. Applications of Social Research Methods to Questions in Information and Library Science. London, UK. Libraries Unlimited; 2017:318-329.
• Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. Sep 29, 2004;6(3):e34. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hamrosi KK, Raynor DK, Aslani P. Pharmacist, general practitioner and consumer use of written medicine information in Australia: are they on the same page? Res Social Adm Pharm. 2014;10(4):656-668. [ CrossRef ] [ Medline ]
• Bechini A, Chiesi F, Giammarco B, Gori E, Di Tommaso M, Strambi N, et al. Electronic package leaflets for vaccines: what are people's perceptions in Italy? Vaccines (Basel). Jul 04, 2022;10(7):1075. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mei NS, Wai CW, Ahamad R. Environmental Awareness and Behaviour Index for Malaysia. Procedia - Social and Behavioral Sciences. Jun 2016;222:668-675. [ CrossRef ]
• Yadegaridehkordi E, Nilashi M, Nizam Bin Md Nasir MH, Safie Bin Mohd Satar N, Momtazi S. Evaluating COVID-19 infection prevention measures in Malaysia: a fuzzy DEMATEL approach. Digit Health. 2023;9:20552076231211670. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ang ZY, Cheah KY, Shakirah MS, Fun WH, Anis-Syakira J, Kong Y, et al. Malaysia's health systems response to COVID-19. Int J Environ Res Public Health. Oct 22, 2021;18(21):11109. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wilson S, Hassan N, Khor K, Sinnappan S, Abu Bakar A, Tan S. A holistic qualitative exploration on the perception of scams, scam techniques and effectiveness of anti-scam campaigns in Malaysia. JFC. Sep 27, 2023:0151. [ FREE Full text ] [ CrossRef ]
• Khalid NA, Yang Y. Advancing democracy: the digital Government of Malaysia. ScienceOpen Posters. Apr 8, 2024. URL: https://www.scienceopen.com/hosted-document?doi=10.14293/P2199-8442.1.SOP-.PWXQD9.v1 [accessed 2024-09-04]
• Economic Planning Unit, Prime Minister's Department. Putrajaya, Malaysia. National Fourth Industrial Revolution (4IR) Policy. Economic Planning Unit, Prime Minister's Department. URL: https://www.ekonomi.gov.my/sites/default/files/2021-07/National-4IR-Policy.pdf [accessed 2024-09-04]
• Edrak B, Nor ZM, Shaik AR. The readiness of Malaysia digital economy: a study of three government policies from 1991 to 2020. IJEF. Nov 30, 2022;14(12):84. [ CrossRef ]
• Bolislis WRR, Mortazavi C, Riccioni R, Schaeffer P, Kühler TC. From print to screen: regulatory considerations to adopting innovative approaches for patient information and safety. Ther Innov Regul Sci. Jul 2020;54(4):831-838. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fung EWT, Au-Yeung GTF, Tsoi LM, Qu L, Cheng TKW, Chong DW, et al. Pharmacists' perceptions of the benefits and challenges of electronic product information system implementation in Hong Kong: mixed-method study. J Med Internet Res. Nov 10, 2020;22(11):e20765. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ministry of Communications and Digital Malaysia. Cybersecurity Malaysia. Ministry of Communications and Digital Malaysia. URL: https://www.cybersecurity.my/en/media_centre/media_releases/2023/main/detail/2782/index.html [accessed 2024-09-04]
• Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. Mar 1977;84(2):191-215. [ CrossRef ] [ Medline ]
• Madani economy to boost Malaysian economy, improve quality of life – PM Anwar. Kementerian Kewangan Malaysia. Jul 27, 2023. URL: https:/​/www.​mof.gov.my/​portal/​en/​news/​press-citations/​madani-economy-to-boost-malaysian-economy-improve-quality-of-life-pm-anwar [accessed 2024-09-04]
• Jia X, Pang Y, Liu LS. Online health information seeking behavior: a systematic review. Healthcare (Basel). Dec 16, 2021;9(12):1740. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T de Azevedo Cardoso; submitted 21.01.24; peer-reviewed by N Li, BF Ababneh; comments to author 04.06.24; revised version received 29.06.24; accepted 30.07.24; published 18.09.24.

©Xin Yee Loh, Ai Ling Woo, Azwa Haris, Cheryl Shajini Pereira, Bee Kim Tan. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.09.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Journal Proposal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Advancing phantom fabrication: exploring 3d-printed solutions for abdominal imaging research.

1. Introduction

2. materials and methods, 2.1. characterization of appendix simulation in an anthropomorphic phantom, 2.2. material selection and characterization, 2.3. printing of the abdominal mold, 2.4. phantom fabrication, 4. discussion, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Mørup, S.D.; Stowe, J.; Precht, H.; Gervig, M.H.; Foley, S. Design of a 3D printed coronary artery model for CT optimization. Radiography 2022 , 28 , 426–432. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tino, R.; Yeo, A.; Leary, M.; Brandt, M.; Kron, T. A systematic review on 3D-printed imaging and dosimetry phantoms in radiation therapy. Technol. Cancer Res. Treat. 2019 , 18 , 1533033819870208. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hazelaar, C.; van Eijnatten, M.; Dahele, M.; Wolff, J.; Forouzanfar, T.; Slotman, B.; Verbakel, W.F. Using 3D printing techniques to create an anthropomorphic thorax phantom for medical imaging purposes. Med. Phys. 2018 , 45 , 92–100. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Amini, I.; Akhlaghi, P.; Sarbakhsh, P. Construction and verification of a physical chest phantom from suitable tissue equivalent materials for computed tomography examinations. Radiat. Phys. Chem. 2018 , 150 , 51–57. [ Google Scholar ] [ CrossRef ]
• Tejo-Otero, A.; Fenollosa-Artés, F.; Achaerandio, I.; Rey-Vinolas, S.; Buj-Corral, I.; Mateos-Timoneda, M.Á.; Engel, E. Soft-tissue-mimicking using hydrogels for the development of phantoms. Gels 2022 , 8 , 40. [ Google Scholar ] [ CrossRef ]
• Buchwald, R.; Breed, M.D.; Greenberg, A.R. The thermal properties of beeswaxes: Unexpected findings. J. Exp. Biol. 2008 , 211 , 121–127. [ Google Scholar ] [ CrossRef ]
• Yan, Q.; Dong, H.; Su, J.; Han, J.; Song, B.; Wei, Q.; Shi, Y. A review of 3D printing technology for medical applications. Engineering 2018 , 4 , 729–742. [ Google Scholar ] [ CrossRef ]
• Rajan, K.; Samykano, M.; Kadirgama, K.; Harun, W.S.; Rahman, M.M. Fused deposition modeling: Process, materials, parameters, properties, and applications. Int. J. Adv. Manuf. Technol. 2022 , 120 , 1531–1570. [ Google Scholar ] [ CrossRef ]
• Vyavahare, S.; Teraiya, S.; Panghal, D.; Kumar, S. Fused deposition modelling: A review. Rapid Prototyp. J. 2020 , 26 , 176–201. [ Google Scholar ] [ CrossRef ]
• Gear, J.I.; Cummings, C.; Craig, A.J.; Divoli, A.; Long, C.D.; Tapner, M.; Flux, G.D. Abdo-Man: A 3D-printed anthropomorphic phantom for validating quantitative SIRT. EJNMMI Phys. 2016 , 3 , 17. [ Google Scholar ] [ CrossRef ]
• Qiu, J.; Hou, K.; Dyer, B.A.; Chen, J.C.; Shi, L.; Sun, Y.; Xu, L.; Zhao, H.; Li, Z.; Chen, T.; et al. Constructing customized multimodal phantoms through 3D printing: A preliminary evaluation. Front. Phys. 2021 , 9 , 605630. [ Google Scholar ] [ CrossRef ]
• Filippou, V.; Tsoumpas, C. Recent advances on the development of phantoms using 3D printing for imaging with CT, MRI, PET, SPECT, and ultrasound. Med. Phys. 2018 , 45 , e740–e760. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, K.; Ho, C.C.; Zhang, C.; Wang, B. A review on the 3D printing of functional structures for medical phantoms and regenerated tissue and organ applications. Engineering 2017 , 3 , 653–662. [ Google Scholar ] [ CrossRef ]
• Coles-Black, J.; Bolton, D.; Robinson, D.; Chuen, J. Utility of 3D printed abdominal aortic aneurysm phantoms: A systematic review. ANZ J. Surg. 2021 , 91 , 1673–1681. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Higgins, M.; Leung, S.; Radacsi, N. 3D printing surgical phantoms and their role in the visualization of medical procedures. Ann. 3D Print. Med. 2022 , 6 , 100057. [ Google Scholar ] [ CrossRef ]
• Anwari, V.; Lai, A.; Ursani, A.; Rego, K.; Karasfi, B.; Sajja, S.; Paul, N. 3D printed CT-based abdominal structure mannequin for enabling research. 3D Print. Med. 2020 , 6 , 3. [ Google Scholar ] [ CrossRef ]
• Schenkl, S.; Muggenthaler, H.; Hubig, M.; Erdmann, B.; Weiser, M.; Zachow, S.; Heinrich, A.; Güttler, F.V.; Teichgräber, U.; Mall, G. Automatic CT-based finite element model generation for temperature-based death time estimation: Feasibility study and sensitivity analysis. Int. J. Leg. Med. 2017 , 131 , 699–712. [ Google Scholar ] [ CrossRef ]
• Rengier, F.; Mehndiratta, A.; Von Tengg-Kobligk, H.; Zechmann, C.M.; Unterhinninghofen, R.; Kauczor, H.U.; Giesel, F.L. 3D printing based on imaging data: Review of medical applications. Int. J. Comput. Assist. Radiol. Surg. 2010 , 5 , 335–341. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Okkalidis, N. 3D printing methods for radiological anthropomorphic phantoms. Phys. Med. Biol. 2022 , 67 , 15TR04. [ Google Scholar ] [ CrossRef ]
• Kairn, T.; Crowe, S.B.; Markwell, T. Use of 3D printed materials as tissue-equivalent phantoms. In Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Toronto, ON, Canada, 7–12 June 2015; Springer International Publishing: Cham, Switzerland, 2015; pp. 728–731. [ Google Scholar ]
• Lamba, R.; McGahan, J.P.; Corwin, M.T.; Li, C.S.; Tran, T.; Seibert, J.A.; Boone, J.M. CT Hounsfield numbers of soft tissues on unenhanced abdominal CT scans: Variability between two different manufacturers’ MDCT scanners. Am. J. Roentgenol. 2014 , 203 , 1013–1020. [ Google Scholar ] [ CrossRef ]
• D’Souza, W.D.; Madsen, E.L.; Unal, O.; Vigen, K.K.; Frank, G.R.; Thomadsen, B.R. Tissue mimicking materials for a multi-imaging modality prostate phantom. Med. Phys. 2001 , 28 , 688–700. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Holmes, R.B.; Negus, I.S.; Wiltshire, S.J.; Thorne, G.C.; Young, P.; Alzheimer’s Disease Neuroimaging Initiative. Creation of an anthropomorphic CT head phantom for verification of image segmentation. Med. Phys. 2020 , 47 , 2380–2391. [ Google Scholar ] [ CrossRef ]
• Kim, S.Y.; Park, J.W.; Park, J.; Yea, J.W.; Oh, S.A. Fabrication of 3D printed head phantom using plaster mixed with polylactic acid powder for patient-specific QA in intensity-modulated radiotherapy. Sci. Rep. 2022 , 12 , 17500. [ Google Scholar ] [ CrossRef ]
• Marro, A.; Bandukwala, T.; Mak, W. Three-dimensional printing and medical imaging: A review of the methods and applications. Curr. Probl. Diagn. Radiol. 2016 , 45 , 2–9. [ Google Scholar ] [ CrossRef ]
• Haleem, A.; Javaid, M.; Suman, R.; Singh, R.P. 3D printing applications for radiology: An overview. Indian J. Radiol. Imaging 2021 , 31 , 10–17. [ Google Scholar ] [ PubMed ]
• Giannopoulos, A.A.; Steigner, M.L.; George, E.; Barile, M.; Hunsaker, A.R.; Rybicki, F.J.; Mitsouras, D. Cardiothoracic applications of 3-dimensional printing. J. Thorac. Imaging 2016 , 31 , 253–272. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rai, R.; Manton, D.; Jameson, M.G.; Josan, S.; Barton, M.B.; Holloway, L.C.; Liney, G.P. 3D printed phantoms mimicking cortical bone for the assessment of ultrashort echo time magnetic resonance imaging. Med. Phys. 2018 , 45 , 758–766. [ Google Scholar ] [ CrossRef ]
• Cook, T.S.; Steingall, S.J.; Steingall, S.R.; Boonn, W.W. Establishing and running a three-dimensional and advanced imaging laboratory. Radiographics 2018 , 38 , 1799–1809. [ Google Scholar ] [ CrossRef ]
• Linguraru, M.G.; Yao, J.; Gautam, R.; Peterson, J.; Li, Z.; Linehan, W.M.; Summers, R.M. Renal tumor quantification and classification in contrast-enhanced abdominal CT. Pattern Recognit. 2009 , 42 , 1149–1161. [ Google Scholar ] [ CrossRef ]
• Cardenas, C.E.; Yang, J.; Anderson, B.M.; Court, L.E.; Brock, K.B. Advances in auto-segmentation. In Seminars in Radiation Oncology ; WB Saunders: Philadelphia, PA, USA, 2019; Volume 29, pp. 185–197. [ Google Scholar ]
• Demehri, S.; Muhit, A.; Zbijewski, W.; Stayman, J.W.; Yorkston, J.; Packard, N.; Senn, R.; Yang, D.; Foos, D.; Thawait, G.K.; et al. Assessment of image quality in soft tissue and bone visualization tasks for a dedicated extremity cone-beam CT system. Eur. Radiol. 2015 , 25 , 1742–1751. [ Google Scholar ] [ CrossRef ]
• Goldman, L.W. Principles of CT: Radiation dose and image quality. J. Nucl. Med. Technol. 2007 , 35 , 213–225. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mitsouras, D.; Liacouras, P.; Imanzadeh, A.; Giannopoulos, A.A.; Cai, T.; Kumamaru, K.K.; George, E.; Wake, N.; Caterson, E.J.; Pomahac, B.; et al. Medical 3D printing for the radiologist. Radiographics 2015 , 35 , 1965–1988. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jusufbegović, M.; Pandžić, A.; Šehić, A.; Jašić, R.; Julardžija, F.; Vegar-Zubović, S.; Beganović, A. Computed tomography tissue equivalence of 3D printing materials. Radiography 2022 , 28 , 788–792. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hepburn, H.R.; Pirk, C.W.; Duangphakdee, O.; Hepburn, H.R.; Pirk, C.W.; Duangphakdee, O. The chemistry of beeswax. In Honeybee Nests: Composition, Structure, Function ; Springer: Berlin/Heidelberg, Germany, 2014; pp. 319–339. [ Google Scholar ]
• European Food Safety Authority (EFSA). Risk Assessment of Beeswax Adulterated with Paraffin and/or Stearin/Stearic Acid When Used in Apiculture and as Food (Honeycomb). EFSA Support. Publ. 2020 , 17 , 1859E. [ Google Scholar ]

Click here to enlarge figure

Share and Cite

Becircic, M.; Delibegovic, S.; Sehic, A.; Julardzija, F.; Beganovic, A.; Ljuca, K.; Pandzic, A.; Jusufbegovic, M. Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research. Appl. Sci. 2024 , 14 , 8384. https://doi.org/10.3390/app14188384

Becircic M, Delibegovic S, Sehic A, Julardzija F, Beganovic A, Ljuca K, Pandzic A, Jusufbegovic M. Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research. Applied Sciences . 2024; 14(18):8384. https://doi.org/10.3390/app14188384

Becircic, Muris, Samir Delibegovic, Adnan Sehic, Fuad Julardzija, Adnan Beganovic, Kenana Ljuca, Adi Pandzic, and Merim Jusufbegovic. 2024. "Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research" Applied Sciences 14, no. 18: 8384. https://doi.org/10.3390/app14188384

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 17 September 2024

Linear and non-linear associations of depressive symptoms with oral health knowledge, attitudes, and practices among rural older adults in China: a cross-sectional study

• Wenwen Cao 1   na1 ,
• Chenglin Cao 1   na1 ,
• Ying Guo 1 ,
• Zixuan Hong 1 ,
• Xin Zheng 1 ,
• Bohua Ren 2 , 3 ,
• Ren Chen 1 &
• Zhongliang Bai 1 , 3 , 4  

BMC Public Health volume  24 , Article number:  2528 ( 2024 ) Cite this article

Metrics details

Depression affects the oral health of older adults; however, little is known about its impact on oral health among rural older adults in developing countries, which warrants further research. Taking China as an example, there is a large population base of rural older adults suffering from depression, and many rural older people also have long-term oral health problems, which have seriously affected their quality of life in later life. Therefore, this study aimed to explore linear and non-linear associations of depressive symptoms with oral health knowledge, attitudes, and practices among rural older adults in China.

From November 2020 to December 2020, 1,902 rural community-dwelling older people aged 60 years and older were investigated, via a cross-sectional survey. The general information, depressive status, oral health knowledge, attitudes, and practices of the participants were obtained through face-to-face structured questionnaires. Among them, the Zung Depression Self-Rating Scale was used to investigate the depressive symptoms of the participants in this survey. The generalized linear model and classification and regression tree model were used, separately.

Based on linear analysis results, we found that minimal to mild depressive symptoms [regression coefficient ( β ) = -0.345; 95% confidence interval (CI): -0.582 to -0.109, P  = 0.004] and depressive symptoms ( β  = -1.064; 95% CI: -1.982 to -0.146, P  = 0.023) were significantly correlated with oral health knowledge. A negative correlation was observed between minimal to mild depressive symptoms ( β  = -0.385; 95% CI: -0.600 to -0.170, P  < 0.001) and oral health attitudes. In addition, while both minimal to mild depressive symptoms ( β  = 0.018; 95% CI: -0.312 to 0.347, P  = 0.916) and depressive symptoms ( β  = 0.604; 95% CI: -0.675 to 1.883, P  = 0.355) were associated with oral health practices. Furthermore, the non-linear analysis showed a combined effect of depressive symptoms on oral health attitudes, indicating that older people of a younger age, not living alone, and not suffering from depressive symptoms are more likely to report better oral health attitudes.

Both the linear and non-linear analyses in our study showed that depressive symptoms are significantly correlated with the poor oral health attitudes of older adults in rural communities. Furthermore, depressive symptoms were associated with oral health knowledge in the linear analysis. However, no statistically significant difference was found between depressive symptoms and oral health practices in either analysis. This research deepens our knowledge and understanding of relevant evidence in the mental and oral health of people in later life. In addition, analyzing the factors that affect the oral health of older people from the perspective of their depressive status provides new thinking directions and scientific references for improving the oral health of older adults in practical life.

Peer Review reports

Nowadays, with an aging population worldwide, the health of older people has become a great concern [ 1 ]. Oral health is of great significance to the general health and quality of life of older adults [ 2 ]. For example, a survey from China showed a positive correlation between oral health and cognitive function among rural older adults in China, with lower levels of oral health leading to poorer cognitive function [ 3 ]. Another study suggests that maintaining good oral health in older people can effectively reduce the probability of frailty [ 4 ]. As a result, much attention has been paid to oral health, especially to oral health knowledge, attitudes, and practices [ 5 ]. The purpose of maintaining oral health and the conception of how to maintain oral health practice are all part of oral health knowledge, which is the basic premise of health-related behaviors [ 6 ]. Oral health attitudes refer to people’s attitudes towards oral health [ 7 ], while oral health practices represent the basic behaviors or actions people take to maintain oral health [ 8 ]. Previous studies have shown that oral health knowledge, attitudes, and practices negatively correlate with oral conditions including dental caries, periodontitis, and tooth loss [ 9 , 10 ]. Currently, World Health Organization defines oral health literacy as the motivation and ability to access, understand, and use information in ways that promote and maintain good health depending on their cognitive and social skills [ 11 ]. Previous studies have suggested that a lower oral health literacy could lead to poorer oral health outcomes [ 12 , 13 ]. Therefore, maintaining good oral health literacy and improving it in terms of related knowledge, attitudes, and practices remains an urgent social issue.

Depression is a common health issue in later life [ 14 ]. A previous study has shown that approximately 28.4% of older adults worldwide suffer from depressive symptoms [ 15 ]. Similarly, the prevalence of depression among older people in China is exceedingly high, especially in rural areas where the prevalence of older people is about 24.0% [ 16 ]. Depression is recognized as a significant risk factor for many adverse outcomes, particularly in older adults [ 17 ]. For example, depressive symptoms can compromise cognitive performance [ 18 ], increase the probability of suffering from anxiety [ 19 ], and may even lead to suicide [ 20 ]. At present, the relationship between depressive symptoms and oral health knowledge, attitudes, and practices is understudied. In previous studies, the relationship between depressive status and poor oral health outcomes has been demonstrated [ 21 , 22 ], indicating that older people with depressive symptoms tend to ignore oral hygiene procedures and avoid the need for dental care, eventually leading to an increased risk of oral disease [ 23 ]. However, according to current findings, the association between depressive symptoms and oral health in older adults is bi-directional [ 24 , 25 ]. For example, some studies proved that people with poor oral health are more likely to suffer from depressive status [ 24 , 26 ], while others concluded that depressive symptoms can harm the oral health of the respondents [ 27 , 28 ]. Furthermore, in studies investigating the relationship between depressive symptoms and oral health in older adults, the methods used have mainly focused on linear models, showing that depressive status is harmful to oral health [ 23 , 28 ].

In practice, the oral health outcomes of older adults are influenced by many factors, such as advanced age, being male, lower educational attainment, and living alone [ 28 , 29 , 30 ]. Although these factors have been fully explored in many studies, whether they interact with depressive symptoms to affect oral health in older adults remains unclear. Exploring multiple interactions is conducive to obtaining effective interventions to reduce the occurrence of oral health diseases, and these findings may be valuable in finding effective ways to improve oral health. Therefore, in order to explore the interactions between variables and their combined relationships, we adopted the classification and regression tree (CART) model, which is an original and complex non-parametric approach [ 31 ].

In addition, in the past decades, studies that examined the impact of depressive symptoms on oral health were mainly conducted in developed countries, such as Australia and the United States [ 28 ]. Findings on this topic focusing on developing countries are limited. Furthermore, the participants recruited for analyses were from urban communities, whereas rural residents were less frequently discussed. Given the lack of medical and health resources in rural areas as well as the fact that older people are relatively poorly educated, more attention should be paid to these challenged communities [ 7 , 32 ].

Therefore, linear and non-linear models were used in this study to explore the impact of depressive symptoms on oral health knowledge, attitudes, and practices of older people in rural China. The relationship between depressive status and oral health among rural older adults in developing countries can be better illustrated by the results of this study, which provides a foundation for promoting the oral health of older adults, thus improving their oral health care.

Data and study sample

In accordance with the study design requirements, a cross-sectional study was conducted from November to December 2020. A multi-stage stratified random sampling method was applied to recruit subjects. The sampling process of this study can be divided into the following three steps: Firstly, according to the research design and geographical location, we selected four regions from the east of China: Jinshan of Shanghai, Huzhou of Zhejiang Province, Changzhou of Jiangsu Province, and Huainan of Anhui Province. Secondly, two county-level regions were randomly selected from the abovementioned four regions, yielding a total of eight county-level regions. Finally, 2 to 4 communities, including urban communities (streets) and rural communities (townships), were randomly selected from the abovementioned eight county-level regions, resulting in 24 communities as the sampling areas for this study.

Criteria for inclusion of subjects included those with clear verbal expression and consciousness, aged ≥ 60 years, resided ≥ 3 years in the local community, and agreed to be investigated. Meanwhile, we excluded those who were not able to carry out proper verbal communication (e.g., being deaf or mute and dementia or cognitive impairments, and bedridden patients). At the same time, the participants received a verbal description of the purposes and procedures of the study, and informed consent was needed before the interview. The study involved 4,257 participants who met the study design requirements. After excluding invalid questionnaires, 4,218 participants were included in the final analysis, of which 1,902 lived in rural areas. To reduce overlaps, the data collection and study design details had been described previously [ 33 , 34 ].

Independent variable

In this study, depressive status was taken as the independent variable, which was assessed using the Zung Self-Rating Depression Scale, with a score from 1 to 4 for each item on the scale. There are 20 items on the scale, with an overall depressive symptoms score ranging from 20 to 80, with higher scores indicating more severe depressive symptoms. For data analysis, we categorized this score into three groups: no depressive symptoms (< 50), minimal to mild depressive symptoms (50 – 59), and depressive symptoms (> 60), which was similar to previous research [ 35 ]. The samples had good internal consistency, as Cronbach’s α of the scale was 0.666.

Outcome variables

Based on the Oral Health Surveys: Basic Methods from the World Health Organization and the Fourth National Oral Health Epidemiology Survey in China [ 36 , 37 ], we measured oral health knowledge, attitudes, and practices as follows.

Oral health knowledge

There were eight questions in the oral health knowledge questionnaire, with 2 points for correct answers, 1 point for unknown answers, and 0 points for wrong answers, totalling 16 points. The sum of the scores for each question is the total score for each participant; the higher the respondents’ scores, the better their knowledge of oral health. Cronbach’s α was 0.653, which showed a good consistency.

Oral health attitudes

The questionnaire on oral health attitudes included six questions, in which negative attitudes were scored with 0 points and gradually positive attitudes with 1 to 2 points. The score for each question was added to obtain the total score for each respondent’s oral health attitudes, a total score of 12 points. The higher the respondent's score, the more positive their oral health attitude. Cronbach’s α was 0.681 in this population.

Oral health practices

The oral health practices questionnaire consisted of 12 questions, each with a maximum and a minimum score of 3 and 0 points, respectively. If the frequency of eating sweets, smoking, drinking, and other harmful practices is lower, the score of oral health practices is higher; similarly, the higher the frequency of oral health practices, such as brushing teeth, using fluoride toothpaste, flossing, and dental cleaning, the higher the score of oral health practices. The score range was 0 to 36 points; the higher the score of the interviewees, the more correct their oral health practices. Cronbach’s α for this measurement was 0.676.

Assessment of demographic data

In addition, we also collected the characteristic information of interviewees during the survey. These data consisted of age (years), body mass index (BMI, kg/m 2 ), gender (male or female), marital status (married or single), living status (living alone and not living alone), and educational attainment (primary school or below, junior school, high school, and college or above). Moreover, health practice information regarding smoking and drinking status and socioeconomic status regarding the source of income was also collected.

Data analysis

Initially, we used mean ± standard deviations to express continuous variables. Next, we aimed better to display the characteristics and current situation of the participants. A general linear model (GLM) was used to explore the relationship between depressive symptoms and oral health knowledge, attitudes, and practices. The GLM can be specified as follows:

In this model, oral health knowledge score is the dependent variable, represented by \(Y\) ; \(\alpha\) is the intercept; \(\beta 1\) is the corresponding coefficient; \(\beta 2\) Confounders1 + … + \(\beta n\) Confoundersn indicate potential confounders, with \(\beta 2\dots \beta n\) as their corresponding coefficients. Based on previous studies [ 28 , 38 ], we included age, gender, marital status, living status, and education as potential variables in the analysis. Due to the low number of cases, we combined the high school and college numbers above. In the data analysis process, according to the variance inflation factor (VIF), there was no collinearity between any of the independent variables (VIF > 10 indicated collinearity).

Subsequently, to identify the interactions among the risk factors for oral health, a Classification And Regression Tree (CART) was employed. This non-parametric model has been used to explore the interactions between different variables in public health studies [ 39 ].

Lastly, completed data were processed using SPSS version 23.0 for statistical analysis, and the significance level was set at 5%.

Descriptive analysis results

As shown in Table  1 , the women’s scores were 9.96 ± 2.48, 7.53 ± 2.30, and 18.67 ± 3.33, respectively, while those of the elder adults living alone were 9.53 ± 2.33, 7.06 ± 2.29, and 17.44 ± 3.78, respectively. The scores in oral health knowledge, attitudes, and practices of single respondents were 9.70 ± 2.40, 7.35 ± 2.18, and 17.49 ± 3.62, respectively.

In addition, respondents with educational attainment of college or above scored 11.67 ± 1.53 and 9.67 ± 2.08 for their oral health knowledge and attitudes, respectively, and had mean scores of 15.00 ± 5.29 for their oral health practices.

GLM results

The generalized linear model results are presented in Table  2 . It shows that after controlling the variables, minimal to mild depressive symptoms ( β  = -0.345; 95% CI: -0.582 to -0.109) and depressive symptoms ( β  = -1.064; 95% CI: -1.982 to -0.146) had a statistically significant negative correlation with the oral health knowledge of the respondents. Specifically, among all the participants, for each unit increase in minimal to mild depressive symptoms or depressive symptoms, oral health knowledge decreased by 1.064 and 0.345, respectively. Table 2 also displays that among participants, minimal to mild depressive symptoms ( β  = -0.385; 95% CI: -0.600 to -0.170) had a negative effect on oral health attitudes. In addition, no statistically significant correlation between depressive symptoms and oral health practices was observed in this study ( P  > 0.05).

Results of the CART model

Figures  1 , 2 , and 3 are used to show the analysis results of the CART model. Figure  1 shows that oral health knowledge was associated with the living status, gender, and age of the subjects, among which the living status was the first classifying factor.

The main factors affecting oral health knowledge were obtained by using CART model

The main factors affecting oral health attitudes were obtained by using CART model

The main factors affecting oral health practices were obtained by using CART model

In Fig.  2 , age, living status, and depressive symptoms were related to the oral health attitudes of the respondents, with age as the primary factor. Based on these factors, the samples were divided into multiple subsets. Participants aged 60 – 69 years (Node 1), living with others (Node 6), and with no depressive symptoms (Node 8) were most likely to have a high oral health attitude score. Conversely, respondents aged 70 years or older (Node 1) and those living alone (Node 3) were most likely to have low scores in oral health attitudes.

In addition, the results of the analysis in Fig.  3 indicate that the factors related to oral health practices were mainly gender, age, and education. Among these, gender was the most important, while depressive status was not significant.

Before this study, there had been no investigation of the impact of depressive symptoms on oral health knowledge, attitudes, and practices of rural older adults in developing countries. Overall, in this paper, the linear analysis results showed that depressive status leads to poorer oral health knowledge and attitudes. Furthermore, depressive symptoms were positively correlated with oral health practice scores. In addition, in the non-linear analysis, depressive status was one of the joint factors affecting the oral health attitudes of older adults.

Most recent studies on the oral health effects of depressive status have focused on older adults in urban areas and have shown that depressive symptoms can determine poor oral health outcomes [ 28 , 40 ]. Our research also indicates that depressive symptoms could lead to poor oral health knowledge in older adults and that the degree of depressive status in rural older adults is negatively related to their oral health knowledge. Previous studies have demonstrated that depressive symptoms can result in cognitive ability and subjective memory decline in older adults and that depressive symptom deterioration is significantly associated with faster cognitive and subjective memory decline [ 41 , 42 ]. This may explain why older adults with depressive status had lower levels of oral health knowledge.

The linear analysis in our research uncovered that both minimal to mild depressive symptoms and depressive symptoms were harmful to the oral health attitudes of older adults, with a statistically significant relationship between minimal to mild depressive status and oral health attitudes. Older people with depressive symptoms tend to have a negative attitude towards life, which may have contributed to the results, and thus pay less attention to their physical health, eventually affecting their health choices [ 43 ].

In contrast to oral health knowledge and attitudes, depressive status was found to improve the oral health practices of participants in our linear analysis, although it was not statistically significant. This result is different from that in previous research on the relationship between depressive status and oral health [ 44 ]. This may be because of the other dependent variables, with various oral diseases or outcomes used as the dependent variables in previous studies. Furthermore, prior studies have shown that poor oral health knowledge, attitudes, and practices can increase the incidence of oral diseases, such as cavities, caries, and periodontitis [ 9 , 45 ]. Therefore, further studies are needed to explore the effects of depressive symptoms on oral health practices in older adults.

In addition, the non-linear results of our study show that the main factors influencing the oral health knowledge of older people are living status, gender, and age. Previous research has confirmed this finding [ 38 ]. Through linear analysis, we found that older women living alone were more likely to have poor oral health knowledge. However, depressive status was not a significant factor affecting oral health in our non-linear analysis, unlike previous results [ 24 ]. This difference may be due to the different analytical methods used in previous studies. Therefore, more studies involving non-linear analytical methods are needed to confirm our findings in order to better understand the role of depressive symptoms in respondents’ oral health knowledge.

Furthermore, gender, age, and educational achievement were found to be factors affecting the oral health practices of older adults in the CART model, whereas depressive status was not a main factor. The results of the non-linear analysis were consistent not only with the linear analysis in this study but also with previous studies showing that advanced older men have the worst oral health practices [ 28 , 29 ]. This outcome can be explained by the fact that men are less likely to visit a doctor and engage in oral preventive care than women [ 46 ]. Smoking is also a poor oral health practice, and the probability of men smoking is greater than that of women [ 47 ].

Interestingly, the non-linear analysis of the effect of depressive symptoms on oral health attitudes differed from the above. Depressive symptoms, age, and oral health attitudes were negatively correlated. Additionally, in the present study, an interactive relationship between depressive symptoms, age, and living status was observed. In other words, older people aged 60 – 69 years who lived with others and did not suffer from depressive symptoms had better oral health attitudes. Depressive status was one of the factors that influenced the oral health attitudes of the respondents, and minimal to mild depressive symptoms and depressive symptoms could have different degrees of influence on the oral health attitudes of older adults. Furthermore, this study suggests that older adults' age and living status are related to oral health improvement. This further confirms the results of previous studies and the linear analysis performed in ours [ 38 , 48 ].

In current studies on depressive symptoms and oral health, the latter is mainly represented by various oral diseases or symptoms of oral discomfort. However, there has been little research into the links between depressive symptoms and oral health knowledge, attitudes, and practices. Although these factors differ from those associated with various oral diseases and discomfort, they also play an essential role in the oral health of older adults. For example, previous studies have displayed that oral health knowledge, attitudes, and practices affect older adults' quality of life and well-being [ 49 , 50 ]. Therefore, the findings of our study may help enhance the oral health knowledge, attitudes, and practices of older adults and alleviate the harmful effects of depressive symptoms on oral health. For instance, older adults with depressive status and poor oral health attitudes may choose to live with others because it is easier for them to protect their oral health. Meanwhile, families and governments should teach older adults how to improve their oral health by flossing, toothpicking, and brushing on time.

However, the study still has limitations. For example, this study cannot provide enough evidence to establish causality because it is cross-sectional. Future longitudinal studies are needed to verify these results. Second, the participants in this study were only from four provinces in China, thus, the results may not be generalizable to all older adults living in rural areas. In addition, this study only investigated the subjective attitudes or cognitions of older adults towards oral health, which does not accurately reflect their actual oral health level in life.

Conclusions

The linear analysis showed that depressive status was significantly associated with poor oral health knowledge and attitudes among older adults living in rural areas. Interestingly, in the non-linear analysis, depressive symptoms were related to oral health attitudes, but it was not a co-acting factor of oral health knowledge or practices. According to the results of our study, measures and programs should be designed to optimize oral health care and management among rural older adults.

Availability of data and materials

The datasets analysed during the current study are not publicly available due to data management regulations but are available from the corresponding author on reasonable request.

Abbreviations

General linear model

Body mass index

Variance inflation factor

Classification and regression tree

Bai Z, Xu Z, Xu X, Qin X, Hu W, Hu Z. Association between social capital and depression among older people: evidence from Anhui Province. China BMC Public Health. 2020;20(1):020–09657.

Google Scholar  

Zhou J, Liu Y, Wang P, Yu H, Yang W, Xu J, Zhao L. Analysis of oral health care status and influencing factors of elderly people in elderly care institutions in Chengdu. Chinese Rural Health Serv Admin. 2018;38(06):765–8 [In Chinese].

Gu W, Li J, Li F, Ho TE, Feng X, Wang Y, Fan M, Cui M, Xu K, Chen X, et al. Association between oral health and cognitive function among Chinese older adults: the Taizhou imaging study. BMC Oral Health. 2023;23(1):640.

Article   PubMed   PubMed Central   Google Scholar  

Zhang J, Xu G, Xu L. Number of teeth and denture use are associated with frailty among Chinese older adults: a cohort study based on the CLHLS from 2008 to 2018. J Nutr Health Aging. 2023;27(11):972–9.

Article   CAS   PubMed   Google Scholar  

Lin HC, Wong MC, Wang ZJ, Lo EC. Oral health knowledge, attitudes, and practices of Chinese adults. J Dent Res. 2001;80(5):1466–70.

Chowdhury S, Roy S, Hasan M, Sadique AA, Islam T, Hasan M, Arafat MY, Bhuiyan MAR, Islam AMK, Khalid O, et al. Oral health knowledge, practice, and oral health status among rohingya refugees in Cox’s Bazar, Bangladesh: a cross-sectional study. PLoS ONE. 2022;17(6):e0269359.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mariño R, Hopcraft M, Ghanim A, Tham R, Khew CW, Stevenson C. Oral health-related knowledge, attitudes and self-efficacy of Australian rural older adults. Gerodontology. 2016;33(4):530–8.

Article   PubMed   Google Scholar  

Bastani P, Bahrami MA, Kapellas K, Yusefi A, Rossi-Fedele G. Online oral health information seeking experience and knowledge, attitudes and practices of oral health among iranian medical students: an online survey. BMC Oral Health. 2022;22(1):29.

Sharda AJ, Shetty S. Relationship of periodontal status and dental caries status with oral health knowledge, attitude and behavior among professional students in India. Int J Oral Sci. 2009;1(4):196–206.

Akinyamoju CA, Taiwo JO, Uwadia E, Agbogidi JM, Ambeke A. Oral health knowledge and practice among traders in Ibadan. Ann Ibadan Postgraduate Med. 2018;16(2):150–6.

CAS   Google Scholar  

Valdez R, Spinler K, Kofahl C, Seedorf U, Heydecke G, Reissmann DR, Lieske B, Dingoyan D, Aarabi G. Oral health literacy in migrant and ethnic minority populations: a systematic review. J Immigr Minor Health. 2022;24(4):1061–80.

Baskaradoss JK. Relationship between oral health literacy and oral health status. BMC Oral Health. 2018;18(1):172.

Batista MJ, Lawrence HP, Sousa M. Oral health literacy and oral health outcomes in an adult population in Brazil. BMC Public Health. 2017;18(1):60.

Marvanova M, McGrane IR. Treatment approach and modalities for management of depression in older people. The Senior care pharmacist. 2021;36(1):11–21.

Hu T, Zhao X, Wu M, Li Z, Luo L, Yang C, Yang F. Prevalence of depression in older adults: a systematic review and meta-analysis. Psychiatry Res. 2022;311:114511.

Tang T, Jiang J, Tang X. Prevalence of depressive symptoms among older adults in mainland China: a systematic review and meta-analysis. J Affect Disord. 2021;293:379–90.

Capurso A, Capurso C, Solfrizzi V, Colacicco AM, D’Introno A, Panza F. Depression in old age: a diagnostic and therapeutic challenge. Recenti Prog Med. 2007;98(1):43–52.

PubMed   Google Scholar  

Leng Y, Diem SJ, Stone KL, Yaffe K. Antidepressant use and cognitive outcomes in very old women. J Gerontol A Biol Sci Med Sci. 2018;73(10):1390–5.

Tiller JW. Depression and anxiety. Med J Aust. 2013;199(S6):S28-31.

Aziz R, Steffens DC. What are the causes of late-life depression? Psychiatr Clin North Am. 2013;36(4):497–516.

Cademartori MG, Gastal MT, Nascimento GG, Demarco FF, Corrêa MB. Is depression associated with oral health outcomes in adults and elders? A systematic review and meta-analysis. Clin Oral Invest. 2018;22(8):2685–702.

Article   Google Scholar  

Hugo FN, Hilgert JB, de Sousa MD, Cury JA. Depressive symptoms and untreated dental caries in older independently living South Brazilians. Caries Res. 2012;46(4):376–84.

Skośkiewicz-Malinowska K, Malicka B, Ziętek M, Kaczmarek U. Oral health condition and occurrence of depression in the elderly. Medicine. 2018;97(41):e12490.

Takiguchi T, Yoshihara A, Takano N, Miyazaki H. Oral health and depression in older Japanese people. Gerodontology. 2016;33(4):439–46.

Hybels CF, Bennett JM, Landerman LR, Liang J, Plassman BL, Wu B. Trajectories of depressive symptoms and oral health outcomes in a community sample of older adults. Int J Geriatr Psychiatry. 2016;31(1):83–91.

Quine S, Morrell S. Hopelessness, depression and oral health concerns reported by community dwelling older Australians. Community Dent Health. 2009;26(3):177–82.

CAS   PubMed   Google Scholar  

Friedlander AH, Norman DC. Late-life depression: psychopathology, medical interventions, and dental implications. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2002;94(4):404–12.

Wright FAC, Takehara S, Stanaway FF, Naganathan V, Blyth FM, Hirani V, Le Couteur DG, Handelsman DJ, Waite LM, Seibel MJ, et al. Associations between oral health and depressive symptoms: findings from the concord health and ageing in men project. Australas J Ageing. 2020;39(3):e306–14.

Lipsky MS, Su S, Crespo CJ, Hung M. Men and oral health: a review of sex and gender differences. Am J Mens Health. 2021;15(3):15579883211016360.

Han MA. Oral Health Status and Behavior among Cancer Survivors in Korea Using Nationwide Survey. Int J Environ Res Public Health. 2017;15(1):14.

Nakabe T, Sasaki N, Uematsu H, Kunisawa S, Wimo A, Imanaka Y. Classification tree model of the personal economic burden of dementia care by related factors of both people with dementia and caregivers in Japan: a cross-sectional online survey. BMJ Open. 2019;9(7):e026733.

Cericato GO, Agostini BA, Costa FDS, Thomson WM, Demarco FF. Rural-urban differences in oral health among older people in Southern Brazil. Braz Oral Res. 2021;35:e135.

Cao W, Cao C, Zheng X, Ji K, Liang Q, Wu Y, Hu Z, Bai Z. Factors Associated with Medication Adherence among Community-Dwelling Older People with Frailty and Pre-Frailty in China. Int J Environ Res Public Health. 2022;19(23):16001.

Cao C, Cao W, Zheng X, Ji K, Wu Y, Hu Z, Chen R, Bai Z. Association of social capital with self-perceived eHealth literacy among community-dwelling older people: age and gender differences. Front Public Health. 2023;11: 1088863.

Cheng C, Liu X, Fan W, Bai X, Liu Z. Comprehensive rehabilitation training decreases cognitive impairment, anxiety, and depression in poststroke patients: a randomized, controlled study. J Stroke Cerebrovasc Dis. 2018;27(10):2613–22.

Zhang K, Li J, Lu Z. Oral health status and knowledge of only and non-only children in China. British Dental J. 2021;1–7.

Lu HX, Tao DY, Lo ECM, Li R, Wang X, Tai BJ, Hu Y, Lin HC, Wang B, Si Y, et al. The 4th national oral health survey in the mainland of china: background and methodology. Chinese J Dental Res. 2018;21(3):161–5.

Kim EJ, Jin BH. Comparison of oral health status and daily nutrient intake between elders who live alone and elders who live with family: Based on the Korean National Health and Nutrition Examination Survey (KNHANES VI) (2013–2015). Gerodontology. 2018;35(2):129–38.

Bai Z, Wang Z, Shao T, Qin X, Hu Z. Association between social capital and loneliness among older adults: a cross-sectional study in Anhui Province, China. BMC Geriatr. 2021;21(1):26.

Harikiran AG, Pallavi SK, Hariprakash S, Nagesh KS. Oral health-related KAP among 11- to 12-year-old school children in a government-aided missionary school of Bangalore city. Indian J Dental Res. 2008;19(3):236–42.

Article   CAS   Google Scholar  

Hou Z, Wang X, Wang Y, Wang J, Zhong J. Association of depressive symptoms with decline of cognitive function-Rugao longevity and ageing study. Neurolog Sci. 2020;41(7):1873–9.

Sabatini S, Woods RT, Ukoumunne OC, Ballard C, Collins R, Clare L. Associations of subjective cognitive and memory decline with depression, anxiety, and two-year change in objectively-assessed global cognition and memory. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2022;29(5):840–66.

Ploughman M, Wallack EM, Chatterjee T, Kirkland MC, Curtis ME. Under-treated depression negatively impacts lifestyle behaviors, participation and health-related quality of life among older people with multiple sclerosis. Mult Scler Relat Disord. 2020;40:101919.

Friedlander AH, Friedlander IK, Gallas M, Velasco E. Late-life depression: its oral health significance. Int Dent J. 2003;53(1):41–50.

Levin L, Shenkman A. The relationship between dental caries status and oral health attitudes and behavior in young Israeli adults. J Dent Educ. 2004;68(11):1185–91.

Thompson AE, Anisimowicz Y, Miedema B, Hogg W, Wodchis WP, Aubrey-Bassler K. The influence of gender and other patient characteristics on health care-seeking behaviour: a QUALICOPC study. BMC Fam Pract. 2016;17:38.

Zhang Y, He J, He B, Huang R, Li M. Effect of tobacco on periodontal disease and oral cancer. Tob Induc Dis. 2019;17:40.

An JY, Darveau R, Kaeberlein M. Oral health in geroscience: animal models and the aging oral cavity. GeroScience. 2018;40(1):1–10.

Choi E, Jung D. Factors Influencing Oral Health-Related Quality of Life in Older Adults in Rural Areas: Oral Dryness and Oral Health Knowledge and Behavior. Int J Environ Res Public Health. 2021;18(8):4295.

Kwon SR, Lee S, Oyoyo U, Wiafe S, De Guia S, Pedersen C, Martinez K, Rivas J, Chavez D, Rogers T. Oral health knowledge and oral health related quality of life of older adults. Clinical and experimental dental research. 2021;7(2):211–8.

Download references

Acknowledgements

The authors wish to express sincere appreciation to all who assisted and engaged in our study.

This work was supported by the National Natural Science Foundation of China (No. 72304003); Outstanding Research and Innovation Team Program of the Education Department of Anhui Province (No. 2023AH010036); Key Laboratory of Public Health Social Governance, Philosophy and Social Sciences of Anhui Province (No.PHG202309); the China Scholarship Council (No. 202209095002).

Author information

Wenwen Cao and Chenglin Cao contributed equally to this work.

Authors and Affiliations

Department of Health Services Management, School of Health Services Management, Anhui Medical University, Hefei, 230032, China

Wenwen Cao, Chenglin Cao, Ying Guo, Zixuan Hong, Xin Zheng, Zhi Hu, Ren Chen & Zhongliang Bai

Department of Epidemiology and Biostatistics, School of Public Health, Guangdong Medical University, Dongguan, 523808, China

Faculty of Education, Health and Wellbeing, University of Wolverhampton, Wolverhampton, WV1 1QU, UK

Bohua Ren & Zhongliang Bai

Key Laboratory of Public Health Social Governance, Philosophy and Social Sciences of Anhui Province, Hefei, 230032, China

Zhongliang Bai

You can also search for this author in PubMed   Google Scholar

Contributions

ZB and RC conceived and designed the study. WC, CC, YG, ZH and XZ performed statistical analyses, and drafted the manuscript. ZB undertook data collection. ZB, BR, and ZH revised the manuscript. All authors checked, interpreted results, and approved the final version.

Corresponding authors

Correspondence to Ren Chen or Zhongliang Bai .

Ethics declarations

Ethics approval and consent to participate.

Informed consent was obtained from all subjects involved in the study. The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Anhui Medical University (No. 20150927).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Cao, W., Cao, C., Guo, Y. et al. Linear and non-linear associations of depressive symptoms with oral health knowledge, attitudes, and practices among rural older adults in China: a cross-sectional study. BMC Public Health 24 , 2528 (2024). https://doi.org/10.1186/s12889-024-19892-x

Download citation

Received : 21 April 2023

Accepted : 26 August 2024

Published : 17 September 2024

DOI : https://doi.org/10.1186/s12889-024-19892-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Oral health
• Depressive symptoms
• Older people

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]