The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely. As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer After it was first defined by Jameton, much work has been done to refine MD as a concept for improved understanding. A core tenet of MD is that clinical practitioners cannot take action in accordance with their professional ethical standards.

A clinical nurse who is experiencing MD may suffer from a series of serious consequences, 3 - 5 which have negatively impact on psychological and physical wellbeing. And then a series of distressing symptoms will be occurred, such as frustration, powerlessness, anger, sleep disorders, fatigue, insecurity, and guilt. 6 , 7 Studies also suggested that MD is also associated with decreased job dissatisfaction, increased the risk of turnover, early retirement, and long-term absences from work. 2 , 8 - 12 As a result, the safety and quality of patient care provided by these nurses who were staying in serious MD may be significantly impaired because they will feel having not ability to provide quality care for patients because of job dissatisfaction 13 - 15 or abandonment of the profession.

So when a clinical nurse is forced to endure serious MD for a long time and also experience burnout, he/she will avoid aspects of patient care with reduced patient advocacy. It is not surprising that MD will also have a negative impact on organizations and system because this serious status occurred in clinical nurses has been demonstrated to be associated with increased nursing turnover, poor quality of care, and decreased patient satisfaction.

Developing effective intervention strategies to mitigate the negative impact of MD on on clinical nurses, organizations, and system is the presently critical issue. 18 - 20 However, development of valid and reliable instruments is the basis of accurately and early detecting MD, and then providing references for development of intervention strategies. To date, numerous instruments in healthcare have been reported, however it is a problematic issue how to select an effective and specified one because no systematic review to comprehensively analyze and compare available instruments for MD in clinical nurses. Precise validated psychometric instruments play an important role in accurately measuring the level and degree of MD among clinical nursing practitioners. More importantly, however, the psychometric characteristics of all available instruments for measuring MD have not yet been systematically and objectively evaluated, which also deeply obstacle the precise selection of instruments.

Objective

Although several instruments have been reported in literature and can be used to measure the frequency and intensity of MD in health care professionals, no systematic literature review has been performed to comprehensively summarize the psychometric characteristics of instruments of clinical nurses’ MD. Therefore, we designed the current systematic review to document and analyze all potential instruments for detecting clinical nurses’ MD in order to contribute to appropriately select tool to measure the MD in clinical nurses.

Research Design

We designed the framework of the present systematic review in accordance with the recommendations proposed by the Cochrane Collaboration (CC). And then, we reported our findings in line with the structure of the preferred items for systematic review and meta-analysis (PRISMA) statement. We did not need the ethic approval and informed consent because all results in our systematic review were obtained from published studies.

Identification of Citations

A comprehensive search, which was performed by 2 independent investigators, was performed in PubMed, Embase, and China National Knowledge Infrastructure (CNKI) database from their inception to June 2020. We updated the search results per week in order to identify any newly potential records. We developed the search strategy with a combination of medical subject heading (MeSH) and text words. All search strategies were amended according to the unique criteria of each database. The search strategies were summarized in Table 1 . We also hand checked bibliographies of topic-related reviews and eligible studies for the purpose of capturing any potential studies. Finally, we used EndNote software to manage all records. The principle of consulting a third senior investigator was introduced to solve any divergency about identification of studies.

Table 1.

Search Strategies of All Targeted Databases.

a. Pubmed
No. Query
3 ((((moral distress[Title/Abstract]) OR (moral dilemma[Title/Abstract])) OR (Ethic Dilemma[Title/Abstract])) OR (ethical dilemma[Title/Abstract])) AND (((((scale[Title/Abstract]) OR (questionaire[Title/Abstract])) OR (instrument[Title/Abstract])) OR (thermometer[Title/Abstract])) OR (survey[Title/Abstract]))
2 ((((scale[Title/Abstract]) OR (questionaire[Title/Abstract])) OR (instrument[Title/Abstract])) OR (thermometer[Title/Abstract])) OR (survey[Title/Abstract])
1 (((moral distress[Title/Abstract]) OR (moral dilemma[Title/Abstract])) OR (Ethic Dilemma[Title/Abstract])) OR (ethical dilemma[Title/Abstract])
b. Embase
No. Query
#9 #8 AND [embase]/lim NOT ([embase]/lim AND [medline]/lim)
#8 #3 AND #7
#7 #4 OR #5 OR #6
#6 ‘questionnaire’/exp
#5 ‘tool’/exp
#4 scale:ti,ab,kw OR questionaire:ti,ab,kw OR instrument:ti,ab,kw OR thermometer:ti,ab,kw OR survey:ti,ab,kw
#3 #1 OR #2
#2 ‘moral distress’/exp
#1 ‘moral distress’:ti,ab,kw OR ‘moral dilemma*’:ti,ab,kw OR ‘ethic dilemma*’:ti,ab,kw OR ‘ethical dilemma*’:ti,ab,kw
c. CNKI (in Chinese)
1. 检索主题 (Topic) 道德困扰 (moral distress)
2. 检索范围 (Sources) 中国学术期刊网络出版总库,中国博士学位论文全文数据库,中国优秀硕士学位论文全文数据库,中国重要会议论文全文数据库,国际会议论文全文数据库,中国重要报纸全文数据库,中国学术辑刊全文数据库
3. 检索年限 (Search time) 建库至2020年8月 (from inception to August 2020)
4. 检索式 (search algorithm) ( (主题=道德困境 或者 题名=道德困境 或者 v_subject=中英文扩展(道德困境) 或者 title=中英文扩展(道德困境)) 或者 (主题=道德困扰 或者 题名=道德困扰 或者 v_subject=中英文扩展(道德困扰) 或者 title=中英文扩展(道德困扰)) ) 并且 ( ( ( (主题=量表 或者 题名=量表 或者 v_subject=中英文扩展(量表) 或者 title=中英文扩展(量表)) 或者 (主题=问卷 或者 题名=问卷 或者 v_subject=中英文扩展(问卷) 或者 title=中英文扩展(问卷)) ) 或者 (主题=调查 或者 题名=调查 或者 v_subject=中英文扩展(调查) 或者 title=中英文扩展(调查)) ) 或者 (主题=工具 或者 题名=工具 或者 v_subject=中英文扩展(工具) 或者 title=中英文扩展(工具)) ) (模糊匹配)

CNKI = China National Knowledge Infrastructure.

Selection Criteria

In this systematic review, we only considered methodology article of developing and validating original and expanded instruments of detecting MD among clinical nurses or healthcare professionals involving clinical nursing practitioners. For instruments with several versions, we only included the latest version for the final analysis. Moreover, we also considered studies which enrolled clinical nurses as the part of the sample for psychometric evaluation to be eligible. We excluded studies for validation of the modified version of original instruments, and also excluded studies reporting the development and validation of instruments for other health care professionals rather than clinical nurses. In our systematic review, only studies published in English- or Chinese language were taken into account for inclusion.

Data Extraction

In this systematic review, all essential data including leading author, publication year, country, number of items, sample for psychometric properties test, and factor structure were obtained as a result of 2 independent investigators with a standard data extraction sheet. The corresponding author was contacted when we cannot obtain sufficient information from an included version. Any divergency about data extraction was solved through consulting a third senior investigator.

Assessment of Quality

As the goal of the current systematic review was to qualitatively summarize the evidence of all available instruments for detecting MD in clinical nurses, we did not aim at assessing the overall quality of all eligible studies. Instead, we critically assessed the reliability and validity of included instruments with some items from critical appraisal tool (CAT) which was developed by Brink and colleagues in 2012 as part of the scope of this systematic review, and not as a criterion to exclude studies or to assess the validity of the results of single studies. According to the aims, we utilized item 1 (if human subjects were used, did the authors give a detailed description of the sample of subjects used to perform the test?), 8 (was the stability of the variable being measured taken into account when determining the suitability of the time interval between repeated measures?), 10 (was the execution of the test described in sufficient detail to permit replication of the test?), 12 (were withdrawals from the study explained?), and 13 (were the statistical methods appropriate for the purpose of the study?) to assess the psychometric properties of all included instruments.

Method of Analysis

In our current study, no quantitative information will be accumulated. Considering our specific objective, we qualitatively summarized the essential information of eligible instruments, and then evaluated the advantages and disadvantages as following 3 aspects: psychometric evaluation, factor structure, and application objects.

Findings

Result of Identification

We delineated the process of retrieval and selection of studies in Figure 1 . We captured 863 records at initial search stage in PubMed (n = 438), Embase (n = 328), and CNKI (n = 97) and added additional 3 records from application studies, and 846 records remained after removing duplicates using the literature management software EndNote X7. We further excluded 786 records after carefully checking the title and abstract, and thus, 60 items were included to be checked in full-text. Finally, 16 eligible studies 12 , 24 - 38 including 10 original and 6 revised instruments were included for the final analysis after excluding 44 studies due to several reasons including unrelated to topic (n = 29), validation of instruments in different populations (n = 3), cross-cultural adaptation and validation of instruments (n = 7), validation of modified instrument (n = 1), and development and validation of other instruments such as moral distress map (n = 4).

Flow chart of identification and selection of studies.

Characteristics of Included Studies

We documented the basic characteristics of all eligible studies in Table 2 . Of these 16 eligible studies, 12 , 24 - 38 5 were performed in USA, 27 , 29 , 30 , 34 , 38 3 were in Italy, 26 , 31 , 32 2 were in Sweden, 24 , 37 and remaining studies were in other countries such as Brazil, Japan, and Iran. All studies were published between 2000 and 2019. Ten original instruments were reported, and other 6 instruments were all developed after revised the MDS or MDS-R. Out of these 16 instruments, 8 were designed as generic purpose and remaining 8 were as exclusive to certain population such as pediatric nurses and intensive care nurses.

Table 2.

Characteristics of All Instruments Included in the Final Qualitative Analysis.

Study Country Instrument Factor structure No. of items Sample for psychometric test
Raines (abstract) USA Ethic stress scale (ESS) (a) Somatic response, (b) self reliance, and (c) uncertainness 56 229 oncology nurses in USA
Corley USA Moral distress scale (MDS) (a) Individual responsibility, (b) not in the patient’s best interest, and (c) deception 32 214 nurses from several Unites States hospitals
Corley et al USA Moral distress scale (MDS) (a) Individual responsibility, (b) not in patient’s best interest, and (c) deception 38 106 nurses from 2 medical centres in USA
Sporrong et al Sweden Moral distress questionnaire (a) Level of moral distress and (b) tolerance/openness toward moral dilemmas 9 259 staff members (including 117 nurses) from 4 medical departments and 3 pharmacies in Sweden
Glasberg (2007) Sweden Stress Conscience Questionnaire (SCQ) (a) Internal demands and (b) external demands and restrictions 9 444 healthcare personnel in Sweden from 8 primary healthcare centers
Eizenberg et al Israel Moral Distress Questionnaire (MDQ) (a) Problems caused by work relationships among staff, (b) problems due to lack of resources, and (c) problems caused by time pressure 15 179 nurses from a variety of work settings
Ohnishi et al Japan Moral distress scale for psychiatric nurses (a) The unethical conduct by caregivers; (b) low staffing; (c) acquiescence to patient’s right violation 15 369 Japanese psychiatric nurses from 6 hospitals
Hamric et al USA Moral distress scale–revised (MDS-R) Unidimensional structure 21 169 nurses and 37 physicians in 8 intensive care units in the southeastern United States
Lazzarin et al Italy Moral distress scale pediatric version (MDS-PV) Unidimensional structure 33 235 pediatric oncology and hematology nurses from 6 Italian hospitals
Wocial and Weaver USA Moral distress thermometer (MDT) n.a. n.a. 529 participants from various clinical areas
Shoorideh et al Iran ICU nurses’ moral distress scale (IMDS) (a) Inappropriate competencies and responsibilities, (b) errors, and (c) not respecting the ethics principles 30 159 intensive care unit nurses in Iran
Badolamenti et al Italy Modified moral distress scale (MDS-11) (a) Futility dimension and (b) potential damage dimension 11 347 Italian registered nurses employed in a university hospital in Rome
Ramos et al Brazil Brazilian moral distress scale in nurse (a) Acknowledgment, power, and professional identity, (b) safe and qualified care, (c) defense of values and rights, (d) work conditions, (e) ethical infractions, and (f) work teams 49 1227 Brazilian nurses
Schaefer (2019) Portugal Moral distress risk scale (MDRS) (a) Organizational and management issues: challenges in the workplace, (b) end of life care: difficult to provide adequate end of life care, (c) autonomy: low professional autonomy, (d) workload: excessive workload, (e) security: lack of professional security, (f) resources: lack of resources, and (g) conflicts: conflicts in care 30 268 nurses from hospitals and primary healthcare settings
Awosoga et al Canada Moral distress in dementia care survey (MDDCS) (a) Triggers of moral distress: the challenges of managing dementia–related behaviours; concern for safety; a culture that allows poor treatment of residents by staff; issues and conflicts with family members, and an under–esourced model of care that did not fit the needs of residents with dementia, (b) consequences: emotional effects; physical effects; relational consequences; participation in unhealthy coping behaviours; and job dissatisfaction, and (c) mitigating strategies: leadership and policy support; peer support; education and training; staff resources; and self-care 55 389 registered nurses, licensed practical nurses, and healthcare aides
Lazzari et al Italy Moral distress scale for correctional nurses (MDS-CN) Unidimensional structure 20 238 correctional nurses from the Italian correctional facilities and affiliated with the “Simspeonlus” association
Epstein et al USA Measure of moral distress for healthcare professionals (MMD-HP) (a) System-level root causes, (b) clinical root causes at the patient level, (c) compromises to integrity occurring within a team, and (d) breakdowns in the team’s interactions with patients and families 27 653 HCPs (including 440 nurses) from all HCPs in adult and ediatric critical, acute, or long-term acute care settings
a We extratced the essential information of MDS-PV because original version of this instrument was not accessable.
b MDS-CN was initially created by Muccio et al in 2017 and then validated by Lazzari and colleagues in 2019.

Assessment of Psychometric Properties

The overall result of assessment of psychometric properties of each seligible instrument is shown in Figure 2 . According to the assessment result, the development of most of the instruments adequately considered reliability and validity, however test-retest reliability was not adequately evaluated.

An external file that holds a picture, illustration, etc. Object name is 10.1177_0046958021996499-fig2.jpg

Assessment of reliability and validity of all included instruments in the present study. These assessments were performed as following: (1) if human subjects were used, did the authors give a detailed description of the sample of subjects used to conduct the test?, (2) was the stability of the variable being measured taken into consideration in assessing the suitability of the time interval between repeated measures?, (3) was the execution of the test described in sufficient detail to allow replication of the test?, (4) were withdrawals from the study explained?, and (5) were the statistical methods appropriate for the purpose of the study? The overall quality was moderate because most of instrument have inadequate test-retest reliability evaluation.

Qualitative Summaries of All Instruments

Moral distress scale and revisions

Moral distress scale (MDS) is the first instrument which was initially developed to measure the frequency and intensity of MD in intensive care nurses in 2001 by Corley and colleagues. Original MDS was designed to have 32 items with 3 factors structure on a 7 points Likert scale. The psychometric properties of MDS were examined among 214 intensive care nurses from American hospitals, and test result indicated a Cronbach’s alpha of .98 for individual responsibility, .82 for not in patient’s best interest, and .84 for deception, respectively. After validated, MDS has been extensively used to detect the frequency and intensity of MD in intensive care nurses. 3 , 39 For example, Rice and colleagues performed a study to determine the prevalence and contributing factors of MD in 260 medical and surgical nurses, and found that MD is common and can be elicited from different types of situations encountered in the work environment.

Considering the limitations of the original version, Corley and colleagues reported a modified version in 2005. Additional 6 items were added in the modified version, and a content validity index of 1.0 was obtained for 38 items. After employed this modified scale in 106 registered nurses, a Cronbach’s alpha of .98 was produced. To date, this modified scale has also been extensively implemented in empirical studies. For example, Elpern et al performed a study to assess the level of nurses’ MD working in the medical intensive care unit, and found a high level of MD.

Fruet et al adapted MDS and then assess the applicability of this version in the context of nursing in hemato-oncology services in Brazil in 2017. Finally, 26 questions were included in this modified version with 3 factors structure including lack of competence in the team, denial of the nursing role as the patient’s advocate, and disrespect for the patient’s autonomy. Psychometric test suggested a Cronbach’s alpha of .98. To date, however, this revised version of MDS has not been validated and employed in further studies.

In 2017, Badolamenti et al modified MDS and developed a brief instrument (MDS-11) to determine MD. Factor analysis of 347 nurses generated a 2-factor structure with 11 items. Psychometric test suggested a high reliability with Cronbach’s alpha of .823 for the futility dimension and .756 for the potential damage dimension, respectively. However, this version was not presently validated and utilized in further studies.

In 2010, for the purpose of proposed a reliable and valid MD assessment tool which can be used in multiple health care settings and with multiple disciplines, Hamric and colleagues adapted MDS to cover more distressing situations and shortened to include 21 items, which were evaluated using a 5-point Likert scale. Authors utilized this scale in 169 registered nurses and 37 physicians to demonstrate its reliability and construct validity. As the first generic instrument of MD for healthcare professionals, moral distress scale - revised (MDS-R) has 3 parallel versions for nurses, physicians, and other health care providers, and has also been extensively utilized by other subsequent survey or comparative studies, 42 - 44 as well as been extensively validated in other countries including Turkey, Italy, 32 , 46 Australia, China, Korea, Brazil, Sweden, and Iran. 52 , 53 To date, the MDS-R has 6 versions, designed for different healthcare provider populations: adult-nurse, adult-physician, adult-other, pediatric-nurse, pediatric-physician, and pediatric-other.

In 2015, Shoorideh expanded the MDS-R to specifically use in intensive care units. This scale was developed with the content analysis approach based on the published theoretical frameworks including Jameton’s conceptualization of MD, House and Rizzo’s role conflict theory, and Rokeach’s value theory. The initial scale was structured with 3-factor including 30 items, which were answered at 0 to 4 Likert scale format, with 0 indicating none and 4 indicating great extent. The content validity index was .98, .95, and .96 for relevance, clarity, and simplicity, respectively. Cronbach alpha of .96 also indicated a highl reliability. Unfortunately, this instrument has not previously been extensively employed in subsequent empirical studies.

Although the aim of the MDS-R and corresponding expanded version are to detect MD in healthcare professionals or specific populations, some root causes were not covered by them. 54 - 56 As a result, Epstein et al constructed and validated an updated instrument, measure of moral distress for healthcare professionals (MMD-PH), to detect the MD in healthcare professionals. Application in 653 professionals demonstrate its reliability and validity with a Cronbach’s alpha of .93. These authors therefore recommend to replace the MDS-R with the MMD-HP as a generic instrument of MD in healthcare professionals. However, further studies should be designed to extensively validate MMD-PH as a preferred tool of measuring MD in healthcare professionals because several limitations existed in the methodology study. Certainly, the validation in different cultural settings should also be considered.

Moral distress measures for specific populations

The parallel version MDS primarily focused on pediatric setting has been used in clinical practice, and the psychometric properties of revised version of the original MDS-PV has also been validated in Iran and Sweden. Moreover, Professor Corley also developed moral distress scale neonatal-pediatric version (MDS-NPV), which which has also been employed in neonatal intensive care unit nurses although the Corley’s original version was not published. In 2014, Lazzarin et al revised and employed MDS for nurses in pediatric context (MDS-PV), which was initially developed by Corley et al based on MDS. In their empirical study, Lazzarin et al shorten items from initial 38 to 33 which was answered with 0 to 6 Likert scale, and produced a highly reliability with a Cronbach’s alpha of .959. Meanwhile, Professor Corley also confirmed the equivalence between adapted version and the original version. The psychometric properties of MDS-NPV in nurses working other pediatric settings rather than intensive care should be further investigated.

In 2017, Muccio et al initially created a MDS for correctional nurses (MDS-CN) based on MDS-R, which was subsequently employed and validated by Lazzari and colleagues in 2019. This scale was structured with unidimensiona with 20 items, which should be indicated using the numerical number from 0 to 5 format. Content validity index, Cronbach’s alpha, and test-retest reliability was 99.0, .91, and .99 in original version respectively, and the subsequent study of 238 correctional nurses confirmed its reliability with a Cronbach’s alpha of .89.

Eizenberg et al found that systematic examination for measurement equivalence of all applicable instruments has not been reported, and thus they developed a culture-sensitive MD questionnaire for nurses working in across working settings in 2008. Authors firstly elicit the culture specific themes based on focus group interview, and then tested the psychometric properties of questionnaire in 179 nurses from a variety of work settings. The psychometric analysis suggested a Cronbach’s alpha of more than .79, with a Cronbach’s alpha of .851, .791, and .804 for the first, second, and the third factor, respectively. Although as the first generic instrument of measuring MD in clinical nurses, no published study utilized it to measure MD. Therefore, further studies are needed to evaluate the measure in different cultural settings.

Because MDS was not administered in psychiatric setting, Ohnishi et al developed the MDS for psychiatric nurses (MDS-P) through combining the 24 items derived from the MDS with and then employed it in Japanese psychiatric nurses in 2010. MDS-P consists of 3 factors structure with 15 items in a 7-Likert scale. In this scale, the frequency and intensity were all marked from 0 to 6, and the score was positively associated with the severe levels of MD. A total of 369 psychiatric nurses from 6 Japanese hospitals were enrolled to respond the scale, and subsequent psychometric analysis suggested a Cronbach’s alpha of .90 for the whole scale, with .85, .82, and .79 for factor 1, 2, and 3 structure, respectively. As an instrument specific to psychiatric nurses, further studies should be designed to test its psychometric properties across cultural settings.

Moral Distress in Dementia Care Survey (MDDCS) tool was created by Awosoga and colleagues in 2018, with the purpose of specifically examine MD among nursing staff caring for the unique population with dementia. The team conducted an exploratory sequential mixed method to generate item pool, and then enrolled 389 sample of consisting of registered nurses, licensed practical nurses, and healthcare aides to test the psychometric properties. After completing factor analysis, authors got a 3-factor structure covering 55 items on a 5-Likert scale. The construct validity was demonstrated by the CFA method. Psychometric evaluation indicated a Cronbach’s alpha of .94, .92, .93, and .83 for the frequency of MD, severity, effects, and mitigating factors, respectively.

Other moral distress measures for nurses

Sporrong et al found that, in 2006, several instruments can be used to closely measure MD, however the scope of application of these instruments is limited. Thus, they developed a 9-item MD questionnaire in 2006 to cover this gap, which can be called as moral distress questionnaire for clinical nurse (MDQ-CN). And then, authors tested the validity and reliability of this questionnaire in 259 staff members working in 200 departments and 59 pharmacies, and obtained a Cronbach’s alpha of .78 for factor 1 and a Cronbach’s alpha of .62 for factor 2, respectively. Although this study enrolled numerous professionals with diverse working settings to validate the reliability and validity of the questionnaire, inadequate root causes limited the application value. Moreover, no further empirical study to determine the psychometric properties.

Brazilian scale of moral distress in nurses was initially developed by Ramos and colleagues in 2017 in order to specifically detect the frequency and intensity of MD. The initial version included 57 questions but did not performed factor analysis. Since then, original authors performed a methodology study to test the psychometric properties of this scale. Factor analysis shorten items from 57 to 49 and thus generated a 6-factor structure including (a) acknowledgment, power, and professional identity; (b) safe and qualified care; (c) defense of values and rights; (d) work conditions; (e) ethical infractions; and (f) work teams. Psychometric test indicated a high reliability with a Cronbach’s alpha of .980 for the whole scale. Moreover, these authors also investigated the application of this scale in the primary health care setting and also obtained a high reliability. Moral distress thermometer (MDT) is a tool of rapidly measuring MD and tracking changes in MD over time, as well as quantitatively the time-specific level of MD, which was developed by Wocial and colleagues in 2012. Authors analyzed the data from 529 nurses to demonstrate the convergent validity and concurrent validity of MDT, and then indicated that MDT has great potential as a screening tool for use in research, evaluating the effectiveness of interventions designed to decrease a nurse’s level of MD. To date, some empirical studies have employed this instrument to examine the level of MD in school nurses 62 , 63 and intensive care unit nurses. 64 , 65 However, the application in other clinical nurses or cultural settings has not yet been reported.

Other measures of moral distress and related topics

We also identified another 4 instruments including Moral distress risk scale (MDRS), MD assessment questionnaire (MDAQ), ethics stress scale (ESS), and stress of conscience questionnaire (SCQ), which were all also designed to measure the level of MD in nurses.

Moral distress risk scale (MDRS) was created by Schaefer et al in 2019 in order to offer a tool that could be useful beyond hospital care settings. After identified 53 items, the team enrolled 268 nurses from hospitals and primary healthcare setting to perform factor analysis and psychometric evaluation. Finally, this scale was developed with 7 factors covering 30 items on a 4-Likert scale (1 indicates never and 4 represents always). Psychometric test indicated a good internal consistency of .913. Compared to previous instruments which were all built based on causes, MDRS was developed at the basis of risk factors which may reduce the negative impact of subjectivity of MD on the measurement result. Thus, considering this advantage, further studies should be performed to validate the psychometric properties of MDRS in other working and cultural settings.

The MDAQ, which was established in 2002 by Hanna, can measure MD from type, intensity, frequency, and duration of the experience. According to published literature, this questionaire can be utilized across disciplines; however psychometric evaluation for it has not yet been performed.

In 2000, Raines and Tymchuk developed a novel and self-administered instrument, which was named as ESS, to measures the individual’s perception for MD. In this scale, 56 questions were designed and the first 52 questions rated on a Likert-type scale from 1 (agree strongly) to 5 (disagree strongly). Questions 53 through 56 are designed to provide additional information to the researcher and are intended to be answered in various ways. After developed this scale, Raines employed it in emprical study and obtained a content validity of .89 ( P < .05), a test-retest reliability of .82 ( P < .005), and a Cronbach α of .87. Unfortunately, this scale can not be extensively validated or utilized in subsequent studies. 67 , 68

SCQ was developed by Glasberg in 2007 with the purpose of measuring stress arising from a disturbed conscience. The questionnaire was structured with 2 parts covering 9 items. In part 1, the respondent was asked to indicate the stressful situation with a 6-Likert scale from 0 (indicating never) to 5 (indicating every day). In part 2, the respondent was required to indicate the level of bad conscience based on the degree of guilt for every situation in the part 1 on a 10 cm visual analogue scale (VAS) from 0 (indicating no, not at all) to 10 (indicating yes, gives me very bad conscience). In this questionnaire, higher total scores represents higher the perceived stress level. Psychometric evaluation indicated a Cronbach’s alpha of .83, which also demonstrated in empirical study. Moreover, SCQ was also revalidated by Ahlin and colleagues in 2012 to be valid for Swedish settings.

Discussion

Moral distress has become the critically negative phenomenon faced by clinical nurses in various workplaces. Moral distress has demonstrated to be associated with several negative consequences including increased risk of turnover, dissatisfaction with work, intention to leave from work. 26 , 29 Appropriate detection and subsequent interventions play a critical role in mitigating the negative impact of moral distress on physical and psychological aspects of nurses. Validated and reliable instrument is the effective approach to early detect moral distress and then provide references for developing intervention strategy.

After development of the original MDS in 2001, several other instruments which were designed to measure the MD in healthcare professionals or clinical nurses working in different settings have been created. In the current systematic review, we captured 10 original and 6 revised instruments which were all created to measure the frequency and intensity of MD in healthcare professionals or clinical nurses. 12 , 24 - 38 Of these instruments, 5 were all reported to be specific to psychiatric nurses, pediatric nurses, dementia care nurses, and correctional nurses, and intensive care nurses. It is noted that 6 instruments were developed based on previous 2 instruments including MDS and MDS-R. 25 , 26 , 29 , 31 - 33 Out of 16 instruments, 2 including original MDS and MDS-R have been extensively translated to other language and have also been validated. 5 , 50 - 52 As stated above, generic instrument for measurement of MD in nurses and healthcare professionals has been extensively investigated, however the investigation on the specific instrument is still inadequate. Further studies should be performed to develop and validate instruments specific to certain populations, and also to demonstrate the application of existing specific instruments in other working or cultural settings.

Theoretical framework play a critical role in designing and developing instrument in order to evaluate and correlate indicators and constructs, strengths, and deficiencies of theories. For the purpose of guarantying the scientific and logic characteristics of an instrument, it is imperative to report the theoretical framework as followed in method section. Meanwhile, considering this issue, theoretical framework must be followed when revised or adapted from existing instruments for application in other cultural or working settings. Of these 16 instruments captured in our study, 7 instruments clearly stated the followed theoretical frameworks. 25 , 27 - 30 , 32 , 35 , 38 Therefore, the remaining 9 instruments which did not reported the followed theoretical framework should be selected cautiously in empirical study, and should also be further validated in different settings.

MD may be changed as the time change, and thus longitudinal study of exploring the dynamic chang model of MD in clinical nurses is decisive for further developing effective intervention strategy. In order to address this issue, analysis of test-retest reliability must be performed. Of these 16 analyzed instruments in the present study, only 4 clearly reported to have satisfactory test-retest reliability, and thus can be utilized to investigate the application value in other settings. 27 , 28 , 30 , 31 This issue was also the leading cause of reducing the overall quality of all instruments in the present study. Therefore, we strongly suggested that remaining 12 instruments without test-retest reliability test must be still to be tested for assessment of psychometric properties.

Regardless of the fact that several instruments have been reported to date, and some of these instruments have also been translated or adapted into other cultural or working settings, no study has been reported to test the measurement equivalence of the original and revised or modified versions. Meanwhile, measurement equivalence of 1 instrument in the application of different populations such as adult and pediatric patients was not reported. In order to detect reliable frequency and intensity of MD in diverse cultural or working settings, further studies should be designed to test the measurement equivalence of these instruments.

Our study systematically summarized the evidence of instruments of measuring MD in clinical nurses after searching and checking potential studies, however some limitations must be further interpreted. Firstly, despite the fact that we searched relevant studies from 3 electronical databases including PubMed, EMBASE, and CNKI, others such as SCOPUS and Web of Science did not search. So some potentially relevant instruments may be missed. However, 3 targeted databases searched in our study were recommended mainly by Cochrane handbook for performing systematic review because it covers most of the published literature. So, we convinced this issue may not significantly impaired our findings. Secondly, we did not search gray literature, and thus those developed instruments but not published were also missed. However, some gray instruments have been employed and validated in some published studies, we therefore extracted indispensable information from these instruments for further analyzed. 31 , 32 So, we convinced this issue may also not negatively the reliability of our findings. Thirdly, our systematic review only considered studies published in English and Chinese language, and thus some instruments developed and evaluated in other languages or cultures may be also missed.

Conclusion

As one of the most common negative phenomenons experienced by clinical nurses daily, MD has greatly attracted the attention from health care professionals. Regardless of the fact that several instruments have been developed to measure the frequency and intensity of MD, most of them have not been extensively validated in other working or cultural settings, and some were not assessed for test-retest reliability. Meanwhile, some instruments were not developed based on established theoretical frameworks, and most instruments specific to certain populations were created through modifying or revising existing instruments. Therefore, further studies were suggested in order to validate the existing instruments and develop new instrument under consideration for variability of diverse populations and cultures.

Acknowledgments

The authors would like to thank the authors who performed all studies included in the systematic review.

Footnotes

Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) disclosed the following financial support for the research, authorship, and/or publication of this article: This study was supported by Grant from the Technological Innovation and Demonstrational Application Project of Chongqing Science and Technology Bureau (project no. cstc2018jscx-msybX0030).

ORCID iD: Xu Tian An external file that holds a picture, illustration, etc. Object name is 10.1177_0046958021996499-img1.jpg https://orcid.org/0000-0002-3275-8751

References

1. Connie M, Ulrich ABH, Grady C. Moral distress: A growing problem in the health professions? Hastings Center Rep . 2010; 40 ( 1 ):20-23. [ PubMed ] [ Google Scholar ]
2. Tschudin V. Ethics in Nursing: The Caring Relationship . Elsevier; 2003. [ Google Scholar ]
3. Zuzelo PR. Exploring the moral distress of registered nurses . Nurs Ethics . 2007; 14 ( 3 ):344-359. [ PubMed ] [ Google Scholar ]
4. Woods M, Rodgers V, Towers A, La Grow S. Researching moral distress among New Zealand nurses: a national survey . Nurs Ethics . 2015; 22 ( 1 ):117-130. [ PubMed ] [ Google Scholar ]
5. Zhang WW, Wu XY, Zhang YF, Ci LF, Sun CC. Moral distress and its influencing factors: a cross-sectional study in China . Nurs Ethics . 2018; 25 ( 4 ):470-480. [ PubMed ] [ Google Scholar ]
6. Harrowing JN, Mill J. Moral distress among Ugandan nurses providing HIV care: a critical ethnography . Int J Nurs Stud . 2010; 47 ( 6 ):723-731. [ PubMed ] [ Google Scholar ]
7. Weber E. Moral distress, workplace health, and intrinsic harm . Bioethics . 2016; 30 ( 4 ):244-250. [ PubMed ] [ Google Scholar ]
8. De Veer AJE, Francke AL, Struijs A, Willems DL. Determinants of moral distress in daily nursing practice: a cross sectional correlational questionnaire survey . Int J Nurs Stud . 2013; 50 ( 1 ):100-108. [ PubMed ] [ Google Scholar ]
9. Wiegand DL, Funk M. Consequences of clinical situations that cause critical care nurses to experience moral distress . Nurs Ethics . 2012; 19 ( 4 ):479-487. [ PubMed ] [ Google Scholar ]
10. Laurs L, Blaževičienė A, Capezuti E, Milonas D. Moral distress and intention to leave the profession: Lithuanian nurses in municipal hospitals . J Nurs Scholarship . 2020; 52 ( 2 ):201-209. [ PubMed ] [ Google Scholar ]
11. Wachholz A, Dalmolin GL, Silva AMD, Andolhe R, Barlem ELD, Cogo SB. Moral distress and work satisfaction: what is their relation in nursing work? Rev Escola Enfermagem U S P . 2019; 53 :e03510. [ PubMed ] [ Google Scholar ]
12. Shoorideh FA, Ashktorab T, Yaghmaei F, Alavi Majd H. Relationship between ICU nurses’ moral distress with burnout and anticipated turnover . Nurs Ethics . 2015; 22 ( 1 ):64-76. [ PubMed ] [ Google Scholar ]
13. Cutcliffe JR, Links PS. Whose life is it anyway? An exploration of five contemporary ethical issues that pertain to the psychiatric nursing care of the person who is suicidal: part two . Int J Ment Health Nurs . 2008; 17 ( 4 ):246-254. [ PubMed ] [ Google Scholar ]
14. Rutenberg C, Oberle K. Ethics in telehealth nursing practice . Home Health Care Manage Pract . 2008; 20 ( 4 ):342-348. [ Google Scholar ]
15. Kain VJ. Moral distress and providing care to dying babies in neonatal nursing . Int J Palliat Nurs . 2007; 13 ( 5 ):243-248. [ PubMed ] [ Google Scholar ]
16. Jones A. Moral distress among nurses . J Emerg Nurs . 2014; 40 ( 1 ):4. [ PubMed ] [ Google Scholar ]
17. Corley MC, Elswick RK, Gorman M, Clor T. Development and evaluation of a moral distress scale . J Adv Nurs . 2001; 33 ( 2 ):250-256. [ PubMed ] [ Google Scholar ]
18. Aultman J, Wurzel R. Recognizing and alleviating moral distress among obstetrics and gynecology residents . J Grad Med Educ . 2014; 6 ( 3 ):457. [ PMC free article ] [ PubMed ] [ Google Scholar ]
19. Bruce CR, Miller SM, Zimmerman JL. A qualitative study exploring moral distress in the ICU team: the importance of unit functionality and intrateam dynamics . Crit Care Med . 2015; 43 ( 4 ):823-831. [ PubMed ] [ Google Scholar ]
20. Musto LC, Rodney PA, Vanderheide R. Toward interventions to address moral distress: navigating structure and agency . Nurs Ethics . 2015; 22 ( 1 ):91-102. [ PubMed ] [ Google Scholar ]
21. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 . The Cochrane Collaboration; 2011. [ Google Scholar ]
22. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . J Clin Epidemiol . 2009; 62 ( 10 ):1006-1012. [ PubMed ] [ Google Scholar ]
23. Brink Y, Louw QA. Clinical instruments: reliability and validity critical appraisal . J Eval Clin Pract . 2012; 18 ( 6 ):1126-1132. [ PubMed ] [ Google Scholar ]
24. Glasberg AL. Stress of Conscience and Burnout in Healthcare: The Danger of Deadening One’s Conscience . Umeå University Medical Dissertations, 0346-6612, Umeå University; 2007. [ Google Scholar ]
25. Awosoga O, Pijl EM, Hagen B, Hall B, Sajobi T, Spenceley S. Development and validation of the moral distress in dementia care survey instrument . J Adv Nurs . 2018; 74 ( 11 ):2685-2700. [ PubMed ] [ Google Scholar ]
26. Badolamenti S, Fida R, Biagioli V, et al. Modified moral distress scale (MDS-11): validation study among Italian nurses . Professioni Infermieristiche . 2017; 70 ( 4 ):238-248. [ PubMed ] [ Google Scholar ]
27. Corley MC, Minick P, Elswick RK, Jacobs M. Nurse moral distress and ethical work environment . Nurs Ethics . 2005; 12 ( 4 ): 381-390. [ PubMed ] [ Google Scholar ]
28. Eizenberg MM, Desivilya HS, Hirschfeld MJ. Moral distress questionnaire for clinical nurses: instrument development . J Adv Nurs . 2009; 65 ( 4 ):885-892. [ PubMed ] [ Google Scholar ]
29. Epstein EG, Whitehead PB, Prompahakul C, Thacker LR, Hamric AB. Enhancing understanding of moral distress: the measure of moral distress for health care professionals . AJOB Empirical Bioethics . 2019; 10 ( 2 ):113-124. [ PubMed ] [ Google Scholar ]
30. Hamric AB, Borchers CT, Epstein EG. Development and testing of an instrument to measure moral distress in healthcare professionals . AJOB Primary Res . 2012; 3 ( 2 ):1-9. [ Google Scholar ]
31. Lazzari T, Terzoni S, Destrebecq A, Meani L, Bonetti L, Ferrara P. Moral distress in correctional nurses: a national survey . Nurs Ethics . 2020; 27 ( 1 ):40-52. [ PubMed ] [ Google Scholar ]
32. Lazzarin M, Biondi A, Di Mauro S. Moral distress in nurses in oncology and haematology units . Nurs Ethics . 2012; 19 ( 2 ):183-195. [ PubMed ] [ Google Scholar ]
33. Ohnishi K, Ohgushi Y, Nakano M, et al. Moral distress experienced by psychiatric nurses in Japan . Nurs Ethics . 2010; 17 ( 6 ):726-740. [ PubMed ] [ Google Scholar ]
34. Raines ML. Ethical decision making in nurses. Relationships among moral reasoning, coping style, and ethics stress . JONA’s Healthcare Law Ethics Regul . 2000; 2 ( 1 ):29-41. [ PubMed ] [ Google Scholar ]
35. Ramos FRS, Barlem ELD, Brito MJM, Vargas MAO, Schneider DG, de Farias Brehmer LC. Construction of the Brazilian scale of moral distress in nurses - a methodological study . Texto Contexto Enferm . 2017; 26 ( 4 ):e0990017. [ Google Scholar ]
36. Schaefer R, Zoboli EL, Vieira MM. Psychometric evaluation of the moral distress risk scale: a methodological study . Nurs Ethics . 2019; 26 ( 2 ):434-442. [ PubMed ] [ Google Scholar ]
37. Sporrong SK, Höglund AT, Arnetz B. Measuring moral distress in pharmacy and clinical practice . Nurs Ethics . 2006; 13 ( 4 ):416-427. [ PubMed ] [ Google Scholar ]
38. Wocial LD, Weaver MT. Development and psychometric testing of a new tool for detecting moral distress: the moral distress thermometer . J Adv Nurs . 2013; 69 ( 1 ):167-174. [ PubMed ] [ Google Scholar ]
39. Rice EM, Rady MY, Hamrick A, Verheijde JL, Pendergast DK. Determinants of moral distress in medical and surgical nurses at an adult acute tertiary care hospital . J Nurs Manage . 2008; 16 ( 3 ):360-373. [ PubMed ] [ Google Scholar ]
40. Elpern EH, Covert B, Kleinpell R. Moral distress of staff nurses in a medical intensive care unit . Am J Crit Care . 2005; 14 ( 6 ):523-530. [ PubMed ] [ Google Scholar ]
41. Fruet IMA, Dalmolin GL, Barlem ELD, Silva RMD, Andolhe R. Applicability of the adapted moral distress scale in the context of nursing in hemato-oncology services . Rev Gaucha Enfermagem . 2018; 38 ( 4 ):e63060. [ PubMed ] [ Google Scholar ]
42. Colville GA, Dawson D, Rabinthiran S, Chaudry-Daley Z, Perkins-Porras L. A survey of moral distress in staff working in intensive care in the UK . J Intensive Care Soc . 2019; 20 ( 3 ):196-203. [ PMC free article ] [ PubMed ] [ Google Scholar ]
43. Vincent H, Jones DJ, Engebretson J. Moral distress perspectives among interprofessional intensive care unit team members . Nurs Ethics . 2020; 27 ( 6 ):1450-1460. [ PMC free article ] [ PubMed ] [ Google Scholar ]
44. Trautmann J, Epstein E, Rovnyak V, Snyder A. Relationships among moral distress, level of practice independence, and intent to leave of nurse practitioners in emergency departments: results from a national survey . Adv Emerg Nurs J . 2015; 37 ( 2 ): 134-145. [ PubMed ] [ Google Scholar ]
45. Karagozoglu S, Yildirim G, Ozden D, Çınar Z. Moral distress in Turkish intensive care nurses . Nurs Ethics . 2017; 24 ( 2 ):209-224. [ PubMed ] [ Google Scholar ]
46. Lamiani G, Setti I, Barlascini L, Vegni E, Argentero P. Measuring moral distress among critical care clinicians: validation and psychometric properties of the italian moral distress scale-revised . Crit Care Med . 2017; 45 ( 3 ):430-437. [ PubMed ] [ Google Scholar ]
47. Burston A, Eley R, Parker D, Tuckett A. Validation of an instrument to measure moral distress within the Australian residential and community care environments . Int J Older People Nurs . 2017; 12 ( 2 ):12144. [ PubMed ] [ Google Scholar ]
48. Luo Y, Yao SH, Yu GF, Bao GJ, Chen RM. Reliability and validity of Chinese version of moral distress scale [in Chinese] . Chin J Prac Nurs . 2017; 33 ( 34 ):2704-2708. [ Google Scholar ]
49. Chae Y, Yu SJ, Lee EJ, Kang KJ, Yu M. Validity and reliabilityof the Korean version of the moral distress scale-revised for Korean hospital nurses . J Korean Acad Soc Nurs Educ . 2016; 22 ( 2 ):228-239. [ Google Scholar ]
50. Ramos AM, Barlem ELD, Barlem JGT, Rocha LP, Dalmolin GL, Figueira AB. Cross-cultural adaptation and validation of the Moral Distress Scale-Revised for nurses . Rev Bras Enfermagem . 2017; 70 ( 5 ):1011-1017. [ PubMed ] [ Google Scholar ]
51. Pergert P, Bartholdson C, Blomgren K, Af Sandeberg M. Moral distress in paediatric oncology: contributing factors and group differences . Nurs Ethics . 2019; 26 ( 7-8 ):2351-2363. [ PubMed ] [ Google Scholar ]
52. Soleimani MA, Sharif SP, Yaghoobzadeh A, Panarello B. Psychometric evaluation of the moral distress scale-revised among Iranian nurses . Nurs Ethics . 2019; 26 ( 4 ):1226-1242. [ PubMed ] [ Google Scholar ]
53. Nia HS, Shafipour V, Allen KA, Heidari MR, Yazdani-Charati J, Zareiyan A. A second-order confirmatory factor analysis of the moral distress scale-revised for nurses . Nurs Ethics . 2017; 24 :1-12. [ PubMed ] [ Google Scholar ]
54. Henrich NJ, Dodek PM, Alden L, Keenan SP, Reynolds S, Rodney P. Causes of moral distress in the intensive care unit: a qualitative study . J Crit Care . 2016; 35 :57-62. [ PubMed ] [ Google Scholar ]
55. Penny NH, Bires SJ, Bonn EA, Dockery AN, Pettit NL. Moral distress scale for occupational therapists: part 1. Instrument development and content validity . Am J Occup Ther . 2016; 70 ( 4 ):7004300020. [ PubMed ] [ Google Scholar ]
56. Schaefer R, Zoboli EL, Vieira M. Identification of risk factors for moral distress in nurses: basis for the development of a new assessment tool . Nurs Inq . 2016; 23 ( 4 ):346-357. [ PubMed ] [ Google Scholar ]
57. Ghasemi E, Negarandeh R, Janani L. Moral distress in Iranian pediatric nurses . Nurs Ethics . 2019; 26 ( 3 ):663-673. [ PubMed ] [ Google Scholar ]
58. Sandeberg M, Wenemark M, Bartholdson C, Lützén K, Pergert P. To change or not to change - translating and culturally adapting the paediatric version of the moral distress scale-revised (MDS-R) . BMC Med Ethics . 2017; 18 ( 1 ):14. [ PMC free article ] [ PubMed ] [ Google Scholar ]
59. Cavaliere TA, Daly B, Dowling D, Montgomery K. Moral distress in neonatal intensive care unit RNs . Adv Neonat Care . 2010; 10 ( 3 ):145-156. [ PubMed ] [ Google Scholar ]
60. Ramos FRS, Barlen ELD, Brito MJM, Vargas MA, Schneider DG, de Farias Brehmer LC. Validation of the Brazilian moral distress scale in nurses . J Nurs Meas . 2019; 27 ( 2 ):335-357. [ PubMed ] [ Google Scholar ]
61. Barth PO, Ramos FRS, Barlem ELD, Dalmolin GL, Schneider DG. Validation of a moral distress instrument in nurses of primary health care . Rev Latino-Am Enfermagem . 2018; 26 :e3010. [ PMC free article ] [ PubMed ] [ Google Scholar ]
62. Dodek PM, Culjak A, Cheung EO, et al. High prevalence of burnout despite low levels of moral distress and high resilience scores in medical students . Am J Respir Crit Care Med . 2019; 199 :9. [ Google Scholar ]
63. Gibson E, Duke G, Alfred D. Exploring the relationships among moral distress, moral courage, and moral resilience in undergraduate nursing students . J Nurs Educ . 2020; 59 ( 7 ):392-395. [ PubMed ] [ Google Scholar ]
64. Chiafery MC, Hopkins P, Norton SA, Shaw MH. Nursing ethics huddles to decrease moral distress among nurses in the intensive care unit . J Clin Ethics . 2018; 29 ( 3 ):217-226. [ PubMed ] [ Google Scholar ]
65. Wocial L, Ackerman V, Leland B, et al. Pediatric ethics and communication excellence (PEACE) rounds: decreasing moral distress and patient length of stay in the PICU . HEC Forum . 2017; 29 ( 1 ):75-91. [ PubMed ] [ Google Scholar ]
66. Hanna DR. Moral Distress Redefined: The Lived Experience of Moral Distress of Nurses Who Participated in Legal, Elective, Surgically Induced Abortions . Boston College; 2002. [ Google Scholar ]
67. Wands BA. A survey of moral distress in certified registered nurse anesthetists: a theoretical perspective for change in ethics education for advance practice nurses . Int J Nurs Sci . 2018; 5 ( 2 ):121-125. [ PMC free article ] [ PubMed ] [ Google Scholar ]
68. Dumouchel M, Boytim M, Gorman N, Weismuller P. Does moral distress differ between california certified registered nurse anesthetists in independent versus medically supervised practice: an exploratory study . AANA J . 2015; 83 ( 3 ):203-209. [ PubMed ] [ Google Scholar ]
69. From I, Wilde-Larsson B, Nordström G, Johansson I. Formal caregivers’ perceptions of quality of care for older people: associating factors . BMC Res Notes . 2015; 8 :623. [ PMC free article ] [ PubMed ] [ Google Scholar ]
70. Ahlin J, Ericson-Lidman E, Norberg A, Strandberg G. Revalidation of the perceptions of conscience questionnaire (PCQ) and the stress of conscience questionnaire (SCQ) . Nurs Ethics . 2012; 19 ( 2 ):220-232. [ PubMed ] [ Google Scholar ]
71. Yucel SC, Ergin E, Orgun F, Gokçen M, Eser I. Validity and reliability study of the moral distress questionnaire in Turkish for nurses . Rev Latino-Am Enfermagem . 2020; 28 :e3319. [ PMC free article ] [ PubMed ] [ Google Scholar ]
72. Devellis RF. Scale Development: Theory and Applications . 3rd ed. Sage; 2012. [ Google Scholar ]
73. Radhakrishna RB. Tips for developing and testing questionnaires/instruments . J Ext . 2007; 45 ( 1 ):1-4. [ Google Scholar ]

Articles from Inquiry: A Journal of Medical Care Organization, Provision and Financing are provided here courtesy of SAGE Publications