A multi-dimensional approach to improve validation practices for qualitative models of marine social-ecological systems

Abstract

Current modeling practices for social-ecological systems (SES) are often qualitative and use causal loop diagrams (CLDs), as these models promote an evaluation of the systems loops and variable connectivity. Our literature review demonstrated that quality assurance of these models often lacks a consistent validation procedure. Therefore, a guide to improving the validation of qualitative models is presented. The presumed utility protocol is a multi-dimensional protocol with 26 criteria, organized into four dimensions, designed to assess specific parts of the modeling process and provide recommendations for improvement. This protocol was applied to three demonstration cases, located in the Arctic Northeast Atlantic Ocean, Macaronesia, and the Tuscan archipelago. The “Specific Model Tests” dimension, which focuses on the structure of the model, revealed positive evaluations of its structure, boundaries, and capacity to be scaled up. “Guidelines and Processes”, which focuses on the meaning and representativeness of the process, showed positive results regarding purpose, usefulness, presentation, and meaningfulness. “Policy Insights and Spillovers”, a dimension focused on the policy recommendations, revealed a high number of “not apply”, indicating that several criteria are too advanced for the status of the models tested. The “Administrative, Review, and Overview” dimension, which focused on the managerial overview, showed the models needed improvement in the documentation and replicability, while time and cost constraints were positively evaluated. The presumed utility protocol has shown to be a useful tool providing quantitative and qualitative evaluations for an intermediate evaluation of the model-building process, helping to substantiate confidence, with recommendations for improvements and applications elsewhere.