How do you use AI to develop quality research software?
This section of RSQKit is focussed on guidance and practices using AI to creating high quality research software. When considering where to start - always start from the work, not the tool or practice.
Most pages include multiple different approaches for improving quality. If you do not have the time to apply all of them, practicality beats purity - focus on the key aspects:
- Research correctness of output
- Guardrails
- Ability to rollback
- Reproducibility
Guidance
Guidance pages focus on overview, high level, and cross cutting concerns.
Practices
Practice pages focus on specific guidance for a given style of usage, ordered by intensity level. Each practice has guidance on the following topics
- Overview - what is the practice, when is it useful?
- Getting Started - how to get started
- Sample/Example Appropriate Practice - example scenarios
- Concerns/Risks - What risks are related to this practice?
- Quality Practice - What quality practices can you apply to this practice to help/assist with research software quality at this level
- AI Intensity Considerations - Why might you consider using more or less AI?
Spectrum of Intensity of AI Usage
The organising principle used in this section is a Spectrum of Intensity of AI Usage. The purpose of the spectrum is to assist in deciding what level of intensity is suitable for your work.
This correlates with how much context, authority, and autonomy a tool has. It is not a maturity scale. Higher intensity does not mean better practice. It does not mean higher skill. It does not mean better quality.
It is also worth remembering that a key part of any evaluation infrastructure for AI-assisted practice is your manual practice. AI doesn’t replace your skills - it requires your skills more than ever.
Intensity correlates with a number of other issues including:
- Level of AI use/autonomy in the workflow
- Complexity of the AI setup
- Cost, Environmental impacts
- Severity of consequences (blast radius) of failure
- Volume, complexity and novelty of output
Questions to consider of your project and of yourself:
- Where does your project sit? Where should it sit?
- Where do you sit? Where do you want to be? Where should you be?
Guidance
Guidance pages focus on overview, high level, and cross cutting concerns.
tasks ai
How do you use AI to develop quality research software
tasks ai
Overview and comparison of tools useful at different levels of the AI spectrum
Practices
Practice pages focus on specific guidance for a given style of usage, ordered by intensity level.
tasks ai
Practice Overview for Conversational artefact creation
tasks ai
Practice Overview for Conversational Creation with Zip Files (etc)
tasks ai
Practice Overview for IDE autocomplete and inline suggestions
tasks ai
Practice Overview for Editor-integrated local assistant
tasks ai
Practice Overview for Repository-aware assistance
tasks ai
Practice Overview for Constrained local tool-using agents
tasks ai
Practice Overview for Managed or cloud coding agents
tasks ai
Practice Overview for Open-ended autonomous agents
Training
Ahmad Alam
The University of Manchester
Aleksandra Nenadic
The University of Manchester / Software Sustainability Institute
Michael Sparks
The University of Manchester / Software Sustainability Institute
Shoaib Sufi
The University of Manchester / Software Sustainability Institute