How Much Does the Aggregation Method Matter? Rank Reversal and Method Sensitivity in Multi-Criteria FMEA for Rural Hospital Sustainability
Keywords:
Failure mode and effects analysis, Multi-criteria decision making, Rank reversal, Method sensitivity, Rural healthcare, Kendall rank correlationAbstract
Failure mode and effects analysis (FMEA) is routinely coupled with multi-criteria decision-making (MCDM) methods, yet the sensitivity of the resulting risk priorities to the choice of aggregation method is seldom quantified on a fixed expert dataset. This study holds the data constant and varies the analysis. Using a four-factor FMEA questionnaire completed by twenty experts on twelve failure modes threatening sustainable healthcare in rural hospitals, risk rankings are computed under twenty-five configurations that cross eight ranking methods (TOPSIS, VIKOR, MARCOS, CoCoSo, MAIRCA, EDAS, WASPAS, and ARAS) with three weighting schemes (equal, questionnaire-derived, and entropy), plus a four-factor risk priority number baseline, with the classical three-factor index as an external reference. Agreement is high overall (mean Kendall tau-b of 0.931), but disagreement is systematic rather than random: it concentrates in the detection criterion, which correlates negatively with the other three factors, and in the compensation assumption, with regret-based VIKOR demoting the consensus top risk from first to fifth under equal weights. Removing any single expert perturbs rankings less (mean tau-b of 0.972) than switching methods does, so the analyst is a larger source of ranking variance than any individual panelist. The findings are distilled into a reporting protocol that treats method sensitivity as a declarable component of FMEA uncertainty.
Downloads
References
Strasser, R. (2003). Rural health around the world: Challenges and solutions. Family Practice, 20(4), 457–463. https://doi.org/10.1093/fampra/cmg422
Weinhold, I., & Gurtner, S. (2014). Understanding shortages of sufficient health care in rural areas. Health Policy, 118(2), 201–214. https://doi.org/10.1016/j.healthpol.2014.07.018
DeRosier, J., Stalhandske, E., Bagian, J. P., & Nudell, T. (2002). Using health care failure mode and effect analysis: The VA National Center for Patient Safety’s prospective risk analysis system. The Joint Commission Journal on Quality Improvement, 28(5), 248–267. https://doi.org/10.1016/s1070-3241(02)28025-6
Liu, H.-C. (2019). Improved FMEA methods for proactive healthcare risk analysis. Springer. https://doi.org/10.1007/978-981-13-6366-5
Bowles, J. B., & Pelaez, C. E. (1995). Fuzzy logic prioritization of failures in a system failure mode, effects and criticality analysis. Reliability Engineering & System Safety, 50(2), 203–213. https://doi.org/10.1016/0951-8320(95)00068-d
Liu, H.-C., Liu, L., & Liu, N. (2013). Risk evaluation approaches in failure mode and effects analysis: A literature review. Expert Systems with Applications, 40(2), 828–838. https://doi.org/10.1016/j.eswa.2012.08.010
Huang, J., You, J.-X., Liu, H.-C., & Song, M.-S. (2020). Failure mode and effect analysis improvement: A systematic literature review and future research agenda. Reliability Engineering & System Safety, 199, 106885. https://doi.org/10.1016/j.ress.2020.106885
Braglia, M. (2000). MAFMA: Multi-attribute failure mode analysis. International Journal of Quality & Reliability Management, 17(9), 1017–1033. https://doi.org/10.1108/02656710010353885
Kutlu, A. C., & Ekmekcioglu, M. (2012). Fuzzy failure modes and effects analysis by using fuzzy TOPSIS-based fuzzy AHP. Expert Systems with Applications, 39(1), 61–67. https://doi.org/10.1016/j.eswa.2011.06.044
Wang, L.-E., Liu, H.-C., & Quan, M.-Y. (2016). Evaluating the risk of failure modes with a hybrid MCDM model under interval-valued intuitionistic fuzzy environments. Computers & Industrial Engineering, 102, 175–185. https://doi.org/10.1016/j.cie.2016.11.003
Tian, Z.-P., Wang, J.-Q., & Zhang, H.-Y. (2018). An integrated approach for failure mode and effects analysis based on fuzzy best-worst, relative entropy, and VIKOR methods. Applied Soft Computing, 72, 636–646. https://doi.org/10.1016/j.asoc.2018.03.037
Huang, J., Liu, H.-C., Duan, C.-Y., & Song, M.-S. (2022). An improved reliability model for FMEA using probabilistic linguistic term sets and TODIM method. Annals of Operations Research, 312(1), 235–258. https://doi.org/10.1007/s10479-019-03447-0
Zanakis, S. H., Solomon, A., Wishart, N., & Dublish, S. (1998). Multi-attribute decision making: A simulation comparison of select methods. European Journal of Operational Research, 107(3), 507–529. https://doi.org/10.1016/s0377-2217(97)00147-1
Ceballos, B., Lamata, M. T., & Pelta, D. A. (2016). A comparative analysis of multi-criteria decision-making methods. Progress in Artificial Intelligence, 5(4), 315–322. https://doi.org/10.1007/s13748-016-0093-1
Salabun, W., Watrobski, J., & Shekhovtsov, A. (2020). Are MCDA methods benchmarkable? A comparative study of TOPSIS, VIKOR, COPRAS, and PROMETHEE II methods. Symmetry, 12(9), 1549. https://doi.org/10.3390/sym12091549
Belton, V., & Gear, T. (1983). On a short-coming of Saaty’s method of analytic hierarchies. Omega, 11(3), 228–230. https://doi.org/10.1016/0305-0483(83)90047-6
Triantaphyllou, E. (2001). Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP. Journal of Multi-Criteria Decision Analysis, 10(1), 11–25. https://doi.org/10.1002/mcda.284
Wang, X., & Triantaphyllou, E. (2008). Ranking irregularities when evaluating alternatives by using some ELECTRE methods. Omega, 36(1), 45–63. https://doi.org/10.1016/j.omega.2005.12.003
Aires, R. F. de F., & Ferreira, L. (2018). The rank reversal problem in multi-criteria decision making: A literature review. Pesquisa Operacional, 38(2), 331–362. https://doi.org/10.1590/0101-7438.2018.038.02.0331
Lin, S.-W. (2026). Strategic interactions and operational efficiency in supply chain networks: A game-theoretic network directional distance function approach. International Journal of Production Economics, 301, 110157. https://doi.org/10.1016/j.ijpe.2026.110157
Lin, S.-W., & Lu, W.-M. (2025). A multi-period decision framework for mutual fund efficiency: Integrating range directional DEA with machine learning for dynamic investment optimization. INFOR: Information Systems and Operational Research, 64(3), 649–683. https://doi.org/10.1080/03155986.2025.2572139
Lin, S.-W., & Lu, W.-M. (2026). An integrated data envelopment analysis–machine learning framework for evaluating pension fund management efficiency. Applied Soft Computing, 197, 115154. https://doi.org/10.1016/j.asoc.2026.115154
Lin, S.-W., & Lu, W.-M. (2026). A strategic framework for supplier performance assessment and anomaly detection in high-tech supply chains: Integrating shared-resource DEA with machine learning. IEEE Transactions on Engineering Management, 73, 2470–2484. https://doi.org/10.1109/tem.2026.3675293
Lin, S.-W., Lo, H.-W., & Lu, W.-M. (2026). Mapping the evolution of low-carbon supply chain research: A systematic review with non-negative matrix factorization-assisted topic modeling. Process Integration and Optimization for Sustainability. Advance online publication. https://doi.org/10.1007/s41660-026-00858-y
Lin, S.-W. (2026). Evaluating renewable energy policy effectiveness: A quantitative approach for decision support and risk mitigation. Annals of Operations Research. Advance online publication. https://doi.org/10.1007/s10479-026-07296-6
Lin, S.-W., & Lu, W.-M. (2026). Proactive supplier performance reliability prediction: A chance-constrained network DEA and machine learning integration. Annals of Operations Research. Advance online publication. https://doi.org/10.1007/s10479-026-07371-y
Lo, H.-W., Shiue, W., Liou, J. J. H., & Tzeng, G.-H. (2020). A hybrid MCDM-based FMEA model for identification of critical failure modes in manufacturing. Soft Computing, 24(20), 15733–15745. https://doi.org/10.1007/s00500-020-04903-x
Lo, H.-W., Chen, T.-H., Fang, T.-Y., & Lin, S.-W. (2025). Airport sustainability and risk assessment using interval-valued Fermatean fuzzy MCDM network analysis. Research in Transportation Business & Management, 62, 101454. https://doi.org/10.1016/j.rtbm.2025.101454
Leeftink, A. G., Visser, J., de Laat, J. M., van der Meij, N. T. M., Vos, J. B. H., & Valk, G. D. (2021). Reducing failures in daily medical practice: Healthcare failure mode and effect analysis combined with computer simulation. Ergonomics, 64(10), 1322–1332. https://doi.org/10.1080/00140139.2021.1910734
Tartaglia, R., Parretti, C., Candido, G., La Regina, M., Scaldaferri, F., Rumi, G., Napolitano, D., Vetrugno, G., & Barach, P. (2026). Implementing a televisiting program in a tertiary university hospital: Failure modes and effect analysis (FMEA) and solutions for improving patient safety and sustainable care. Safety Science, 196, 107096. https://doi.org/10.1016/j.ssci.2025.107096
Triantaphyllou, E. (2000). Multi-criteria decision making methods: A comparative study. Springer. https://doi.org/10.1007/978-1-4757-3157-6
Watrobski, J., Jankowski, J., Ziemba, P., Karczmarczyk, A., & Ziolo, M. (2019). Generalised framework for multi-criteria method selection. Omega, 86, 107–124. https://doi.org/10.1016/j.omega.2018.07.004
Cinelli, M., Kadzinski, M., Miebs, G., Gonzalez, M., & Slowinski, R. (2022). Recommending multiple criteria decision analysis methods with a new taxonomy-based decision support system. European Journal of Operational Research, 302(2), 633–651. https://doi.org/10.1016/j.ejor.2022.01.011
Chiu, Y.-H., Lo, H.-W., & Lin, S.-W. (2026). An integrated CRITIC–TOPSIS framework for warehouse personnel performance evaluation: Evidence from an electronics manufacturing enterprise. Journal of Intelligent Decision Making and Granular Computing, 2(1), 30–45. https://doi.org/10.31181/jidmgc21202633
Lo, H.-W., & Lin, S.-W. (2025). A hybrid Bayesian BWM–machine learning framework for university digital transformation assessment: Integrating expert clustering and predictive validation. Technology in Society, 85, 103170. https://doi.org/10.1016/j.techsoc.2025.103170
Lo, H.-W., & Lin, S.-W. (2026). A Shapley value-based argumentation framework integrating CRITIC and MARCOS for sustainable supplier evaluation: Evidence from a global semiconductor supply chain. Argumentation Based Systems Journal, 2, 304–323. https://doi.org/10.59543/cg894606
Forman, E., & Peniwati, K. (1998). Aggregating individual judgments and priorities with the analytic hierarchy process. European Journal of Operational Research, 108(1), 165–169. https://doi.org/10.1016/s0377-2217(97)00244-0
Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49–57. https://doi.org/10.1016/j.omega.2014.11.009
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
Hwang, C.-L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. Springer. https://doi.org/10.1007/978-3-642-48318-9
Opricovic, S., & Tzeng, G.-H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2), 445–455. https://doi.org/10.1016/s0377-2217(03)00020-1
Opricovic, S., & Tzeng, G.-H. (2007). Extended VIKOR method in comparison with outranking methods. European Journal of Operational Research, 178(2), 514–529. https://doi.org/10.1016/j.ejor.2006.01.020
Stevic, Z., Pamucar, D., Puska, A., & Chatterjee, P. (2020). Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to compromise solution (MARCOS). Computers & Industrial Engineering, 140, 106231. https://doi.org/10.1016/j.cie.2019.106231
Yazdani, M., Zarate, P., Zavadskas, E. K., & Turskis, Z. (2019). A combined compromise solution (CoCoSo) method for multi-criteria decision-making problems. Management Decision, 57(9), 2501–2519. https://doi.org/10.1108/md-05-2017-0458
Pamucar, D., Vasin, L., & Lukovac, V. (2014). Selection of railway level crossings for investing in security equipment using hybrid DEMATEL-MAIRCA model. Proceedings of the XVI International Scientific-Expert Conference on Railways, 89–92.
Keshavarz Ghorabaee, M., Zavadskas, E. K., Olfat, L., & Turskis, Z. (2015). Multi-criteria inventory classification using a new method of evaluation based on distance from average solution (EDAS). Informatica, 26(3), 435–451. https://doi.org/10.15388/informatica.2015.57
Zavadskas, E. K., Turskis, Z., Antucheviciene, J., & Zakarevicius, A. (2012). Optimization of weighted aggregated sum product assessment. Elektronika ir Elektrotechnika, 122(6), 3–6. https://doi.org/10.5755/j01.eee.122.6.1810
Zavadskas, E. K., & Turskis, Z. (2010). A new additive ratio assessment (ARAS) method in multicriteria decision-making. Technological and Economic Development of Economy, 16(2), 159–172. https://doi.org/10.3846/tede.2010.10
Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1–2), 81–93. https://doi.org/10.1093/biomet/30.1-2.81
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Sheng-Wei Lin (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
