research software and quality: Research Software Quality Dimensions

We distinguish between abstract quality dimensions (high-level categories like reliability or maintainability defined by standards such as ISO/IEC 25010 and concrete quality indicators (measurable proxies like test coverage or community engagement). While indicators provide empirical evidence, they cannot fully capture the subjective, context-dependent nature and dimensions of “good” software.

Assessing research software for quality requires balancing “fit for purpose” (the ability of a tool to solve a specific scientific problem) against long-term goals of reusability and sustainability as championed by the FAIR4RS principles. This assessment is further complicated by the non-linear relationship between technical quality and scientific impact, as well as the potential for software to be repurposed for unintended uses, which may shift the relevant quality criteria entirely.

We build upon the nine top-level [ISO software quality dimensions][ISO-IEC-25010] to bridge the gap between the nature or research software quality (e.g. community around research software) and engineering standards.

Quality Dimension Definitions

EVERSE project is working to formally define a number of research software quality dimensions along with a formal schema to be used in machine readable metadata embedded in various resources.

Each of the quality dimensions will be an umbrella for a number of quality indicators, representing a specific software quality aspect that can be measured.

Current research software quality dimensions are described below. This is still work in progress - we expect indicators in particular to be updated in the near future.

Compatibility

Degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions while sharing the same common environment and resources. This characteristic is composed of the following sub-characteristics:

Co-existence

Degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on any other product.

Interoperability

Degree to which a system, product or component can exchange information with other products and mutually use the information that has been exchanged.

Read more about compatibility dimension on TechRadar.

FAIRness

FAIRness refers to the degree to which research software adheres to the FAIR principles: Findable, Accessible, Interoperable, and Reusable. These principles, adapted for research software, aim to enhance the discoverability, accessibility, interoperability, and reusability of software, thereby maximizing its value and impact in scientific research.