Academic

Improving Model Performance by Adapting the KGE Metric to Account for System Non-Stationarity

arXiv:2604.03906v1 Announce Type: new Abstract: Geoscientific systems tend to be characterized by pronounced temporal non-stationarity, arising from seasonal and climatic variability in hydrometeorological drivers, and from natural and anthropogenic changes to land use and cover. As has been pointed out, such variability renders "the assumption of statistical stationarity obsolete in water management", and requires us to "account for, rather than ignore, non-stationary trends" in the data. However, metrics used for model development are typically based on the implicit and unjustifiable assumption that the data generating process is time-stationary. Here, we introduce the JKGE_ss metric (adapted from KGE_ss) that detects and accounts for dynamical non-stationarity in the statistical properties of the data and thereby improves information extraction and model performance. Unlike NSE and KGE_ss, which use the long-term mean as a benchmark against which to evaluate model efficiency, JKG

M
M Jawad, HV Gupta, YH Wang, MA Farmani, A Behrangi, GY Niu · · 1 min read · 17 views
#cs.LG #physics.ao-ph

arXiv:2604.03906v1 Announce Type: new Abstract: Geoscientific systems tend to be characterized by pronounced temporal non-stationarity, arising from seasonal and climatic variability in hydrometeorological drivers, and from natural and anthropogenic changes to land use and cover. As has been pointed out, such variability renders "the assumption of statistical stationarity obsolete in water management", and requires us to "account for, rather than ignore, non-stationary trends" in the data. However, metrics used for model development are typically based on the implicit and unjustifiable assumption that the data generating process is time-stationary. Here, we introduce the JKGE_ss metric (adapted from KGE_ss) that detects and accounts for dynamical non-stationarity in the statistical properties of the data and thereby improves information extraction and model performance. Unlike NSE and KGE_ss, which use the long-term mean as a benchmark against which to evaluate model efficiency, JKGE_ss emphasizes reproduction of temporal variations in system storage. We tested the robustness of the new metric by training physical-conceptual and data-based catchment-scale models of varying complexity across a wide range of hydroclimatic conditions, from recent-precipitation-dominated to snow-dominated to strongly arid. In all cases, the result was improved reproduction of system temporal dynamics at all time scales, across wet to dry years, and over the full range of flow levels (especially recession periods). Since traditional metrics fail to adequately account for temporal shifts in system dynamics, potentially resulting in misleading assessments of model performance under changing conditions, we recommend the adoption of JKGE_ss for geoscientific model development.

Executive Summary

This article introduces a novel metric, JKGE_ss, designed to address the issue of non-stationarity in geoscientific systems. By adapting the KGE metric to account for system non-stationarity, the authors aim to improve model performance in hydroclimatic modeling. The new metric focuses on reproducing temporal variations in system storage, rather than relying on long-term means. The study demonstrates the effectiveness of JKGE_ss in improving model performance across various hydroclimatic conditions. The authors recommend adopting JKGE_ss for geoscientific model development, highlighting its potential to provide more accurate assessments of model performance under changing conditions.

Key Points

  • ▸ JKGE_ss is a novel metric designed to address non-stationarity in geoscientific systems
  • ▸ The new metric adapts the KGE metric to focus on reproducing temporal variations in system storage
  • ▸ JKGE_ss demonstrates improved model performance across various hydroclimatic conditions

Merits

Strengths in Addressing Non-Stationarity

JKGE_ss effectively addresses the limitations of traditional metrics in accounting for temporal shifts in system dynamics.

Demerits

Potential Over-Reliance on Temporal Variations

JKGE_ss may prioritize reproducing temporal variations over capturing other essential aspects of system behavior.

Expert Commentary

The article presents a significant contribution to the field of hydroclimatic modeling, addressing a critical issue in model development. The introduction of JKGE_ss offers a more nuanced understanding of system dynamics, enabling more accurate model performance. However, it is essential to consider the potential trade-offs between reproducing temporal variations and capturing other essential aspects of system behavior. Further research is needed to explore the robustness and applicability of JKGE_ss across diverse hydroclimatic contexts. The article's findings have significant implications for water management and policy-making, highlighting the need for more adaptive and responsive approaches to hydroclimatic modeling.

Recommendations

  • ✓ Future research should focus on integrating JKGE_ss with existing model calibration and validation techniques to further improve model performance.
  • ✓ The development of JKGE_ss should be extended to other geoscientific systems, such as atmospheric and terrestrial models, to explore its broader applicability.

Sources

Original: arXiv - cs.LG

Related Articles