Research Data and Digital Tooling

Introduction

Artificial intelligence and machine-learning research is increasingly conducted through digital ecosystems rather than isolated computers. A project may involve data collected from sensors, public repositories, interviews, application programming interfaces (APIs), or organisational systems; annotation performed by distributed workers; code maintained in version-control platforms; experiments recorded in notebooks; and outputs deposited in institutional or disciplinary repositories. These activities create both opportunities and responsibilities. Poorly documented data can make a result irreproducible, insecure storage can expose participants, unclear licensing can prevent lawful reuse, and ungoverned generative-AI tools can introduce confidential-data leakage or fabricated content.

Research data management (RDM) is the organised planning, collection, description, storage, protection, preservation, sharing, and eventual disposal of research data and related materials. Digital tooling supports RDM, but tools do not replace methodological judgement. A data-management plan (DMP) connects research questions with practical decisions about what data will exist, who may access it, how quality will be assessed, and how evidence will remain interpretable over time.

For AI and ML students, these issues are especially important because model performance is inseparable from data provenance, annotation quality, sampling choices, preprocessing, and versioning. A technically sophisticated model trained on undocumented or unlawfully obtained data remains a weak research contribution.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the data life cycle and the components of an effective DMP.

2. Design collection, annotation, metadata, storage, security, and backup procedures.

3. Distinguish data ownership, licensing, consent, privacy, and access governance.

4. Select appropriate notebooks, repositories, version-control systems, and collaboration platforms.

5. Apply reproducibility and FAIR principles to AI/ML research.

6. Use generative-AI tools responsibly while protecting integrity, privacy, and confidentiality.

7. Analyse a realistic research-data scenario and justify governance decisions.

1. Research Data as an Evidence System

Research data are recorded observations, measurements, responses, files, labels, logs, or derived values used to answer a research question. In an ML project, data may include raw images, cleaned tables, feature matrices, model predictions, annotation instructions, software environments, and evaluation results. A useful distinction is:

• Raw data: Data captured or received before substantive transformation.
• Processed data: Data cleaned, transformed, filtered, or reformatted.
• Derived data: Data generated from other data, such as embeddings, features, or predictions.
• Reference data: External datasets, ontologies, pretrained models, or documentation used for interpretation.
• Research artefacts: Code, configuration files, notebooks, protocols, model weights, and reports that explain or reproduce analysis.

Data are not automatically neutral facts. Collection instruments, sampling frames, annotation categories, missing-value policies, and preprocessing decisions influence what can be observed and what conclusions are possible. Consequently, data management is part of research design, not merely an administrative activity.

A data life cycle commonly includes planning, collection or acquisition, processing and quality assurance, analysis, preservation, sharing or publication, and disposal or restricted retention. Stages can overlap, and projects may return to earlier stages when errors are discovered.

FAIR and responsible data

The FAIR principles state that research objects should be Findable, Accessible, Interoperable, and Reusable. Findability requires persistent identifiers and discoverable metadata. Accessibility requires a clear access mechanism, although access may be controlled rather than open. Interoperability involves standard formats, vocabularies, and relationships. Reusability requires adequate documentation, provenance, quality information, and licensing.

FAIR does not mean unrestricted publication. Data containing personal, confidential, commercially sensitive, or culturally restricted information may require controlled access, aggregation, anonymisation, or non-disclosure. Responsible management balances openness with privacy, safety, intellectual property, justice, and the legitimate interests of data providers and communities.

2. Constructing a Data-Management Plan

A DMP is a living document that records decisions and responsibilities. Many institutions require it before data collection begins and expect updates when the protocol changes. A strong DMP normally answers the following questions:

1. What data will be collected, generated, reused, or destroyed?

2. What file formats, volumes, quality standards, and naming conventions will be used?

3. Who owns or controls the data, and what consent or permissions apply?

4. How will data be annotated, validated, documented, stored, backed up, and secured?

5. Who can access which data, under what conditions?

6. How will data, code, metadata, and outputs be preserved and shared?

7. What resources, costs, roles, and review points are required?

Data inventory and specification

Begin with a data inventory. For each dataset or artefact, record its purpose, source, sensitivity, expected size, format, responsible person, retention period, and relationship to other objects. A data dictionary should define variables, units, allowed values, missing-value codes, and transformations. For images or audio, relevant fields may include resolution, sampling rate, device, language, and collection context.

Naming conventions should be predictable and machine-readable. A useful filename may include project, object, date, and version, but dates should use an unambiguous format such as `2025-03-08`. Avoid names such as `final_final2.csv`; use version control or structured release identifiers instead. File formats should favour open, documented standards where practical: CSV or Parquet for tabular data, TIFF or PNG for suitable images, WAV for uncompressed audio, and plain text or Markdown for documentation. Format choice remains dependent on analytical requirements and preservation policies.

Roles and accountability

A DMP should identify the principal investigator, data steward, annotators, system administrator, repository manager, and data-protection or ethics contact where relevant. Responsibility must be explicit: the person who collects data may not be the person authorised to approve sharing. Access reviews, incident reporting, retention decisions, and release approval should each have an owner.

3. Collection, Sampling, and Annotation

Data collection and provenance

Collection protocols should specify the population, sampling method, instruments, timing, location, inclusion and exclusion criteria, and quality checks. For reused data, record the original creator, date obtained, source URL or identifier, version, licence, and any transformations. Provenance is the trace of where data came from and what happened to them. A provenance record might link raw data to a cleaning script, an annotation release, a training split, a model run, and a published result.

Sampling decisions affect external validity and fairness. A dataset collected from one geographic region, device type, language group, or platform may not represent deployment conditions. Researchers should document under-representation and avoid claiming that a convenient sample represents a wider population without justification.

Annotation as measurement

Annotation converts observations into labels or structured judgements. It should therefore be treated as a measurement process. An annotation protocol defines the construct, label definitions, decision rules, examples, escalation procedures, and treatment of uncertainty. Ambiguous categories produce noisy labels and make model evaluation difficult.

A pilot annotation phase is valuable. Annotators independently label a shared sample, discuss disagreements, revise the codebook, and then begin the main task. Quality assurance may include hidden test items, duplicate annotation, expert adjudication, range checks, and periodic calibration. Annotator training, compensation, workload, and exposure to disturbing material are ethical and operational concerns.

Agreement statistics can be useful, but they do not replace substantive review. For two annotators assigning categorical labels, Cohen’s kappa is commonly defined as

κ = (pₒ − pₑ) / (1 − pₑ),

where pₒ is observed agreement and pₑ is agreement expected from the annotators’ marginal label proportions. The statistic assumes categorical ratings and a defined pairing of ratings; it can be misleading when categories are highly imbalanced or when more than two annotators are involved. In such cases, alternatives such as weighted kappa or multi-rater measures may be considered, with the choice justified. Researchers should report the agreement measure, its assumptions, the sample used, and how disagreements were resolved.

4. Metadata, Documentation, and Reproducibility

Metadata are data about a research object. Descriptive metadata identify what an object is; structural metadata explain relationships among files; administrative metadata record ownership, rights, access, and preservation information; and provenance metadata describe origin and transformations. A dataset without metadata may be technically available but practically unusable.

At minimum, a dataset record should include title, creator, description, dates, keywords, methodology, population or scope, variables, units, file formats, version, licence, identifier, limitations, and contact information. Sensitive datasets should also describe access conditions without exposing protected information.

A README should explain project purpose, folder structure, software requirements, execution steps, known limitations, and citation instructions. A codebook defines variables and labels. A CHANGELOG records important modifications. A research protocol describes procedures before analysis, while an analysis report explains decisions and results. These documents reduce dependence on tacit knowledge.

Reproducibility has several levels:

• Repeatability: the same team obtains consistent results using the same data, code, and conditions.
• Reproducibility: another team obtains consistent results using the original materials and documented procedures.
• Replicability: an independent study reaches compatible findings using new data or procedures.

Exact numerical replication may be difficult in stochastic ML systems because of hardware, library versions, random seeds, and nondeterministic operations. Researchers should therefore record environment specifications, package versions, random seeds where meaningful, hardware, configuration parameters, and train/validation/test split logic. Reproducibility is strengthened by containers or environment-lock files, but containers do not solve undocumented data access or licensing restrictions.

5. Storage, Backup, and Security

Storage architecture should reflect data sensitivity, scale, access needs, and retention requirements. A practical arrangement may use encrypted institutional storage for sensitive raw data, a controlled project workspace for active analysis, and a repository for approved publication materials. Local laptops should not be the sole location of research data.

The commonly taught 3-2-1 backup rule recommends at least three copies, on at least two different types of media or storage systems, with at least one copy geographically separate. This is a resilience heuristic, not a guarantee. Backups must be tested by restoration, protected against accidental deletion and ransomware, and governed by retention policies. Synchronisation is not identical to backup: synchronisation may replicate corruption or deletion immediately.

Security controls include:

• Authentication: verifying identity, preferably with multi-factor authentication.
• Authorisation: granting only the permissions required for a role.
• Encryption in transit and at rest: protecting data during transfer and storage.
• Key management: controlling encryption keys separately from ordinary files.
• Logging: recording access and important changes.
• Patching: maintaining operating systems and software.
• Incident response: containing, assessing, reporting, and learning from breaches.

The principle of least privilege limits users and services to the minimum access needed. Access should be reviewed when people join, change roles, or leave a project. Sensitive identifiers should be separated from research data where possible, with a controlled linkage key. Pseudonymisation replaces direct identifiers but does not make data anonymous; re-identification may remain possible when datasets are combined.

6. Ethics, Privacy, Licensing, and Governance

Ethical review may be required for research involving people, personal data, sensitive communities, animals, or potentially harmful applications. Consent should be informed, voluntary, specific enough for the proposed use, and compatible with future sharing claims. Researchers should not promise unlimited reuse when future uses cannot reasonably be described.

Privacy regulation differs by jurisdiction, but common principles include purpose limitation, data minimisation, accuracy, storage limitation, security, and accountability. Legal compliance is necessary but not sufficient: a lawful use may still be ethically inappropriate or socially harmful. Researchers should assess risks of re-identification, discrimination, surveillance, and unintended secondary use.

Licensing determines what others may legally do with data or code. Public availability does not imply public-domain status. A licence may require attribution, preserve the same licence for derivatives, restrict commercial use, or prohibit redistribution. Dataset, software, image, and model licences may differ. Researchers should retain licence evidence, comply with attribution conditions, and check whether combining resources creates incompatible obligations. Where no licence is stated, permission should not be assumed.

Governance is the system of policies, decision rights, controls, and review processes through which data are managed. A governance board or designated review group may approve access, evaluate requests, monitor compliance, and resolve conflicts. Community governance is important when data concern Indigenous peoples or other groups with collective interests that are not captured by individual consent alone.

7. Digital Research Tooling

Version control and collaboration

A distributed version-control system such as Git records changes to code and text files, supports branching, and enables review before merging. It is not a complete data-management system: large or sensitive datasets should not automatically be committed to a public code repository. Use appropriate large-file storage, data repositories, or restricted institutional services. Commit messages should explain substantive changes, and releases should be tagged so that published results refer to a stable state.

Issue trackers and project boards turn informal requests into visible tasks. They support ownership, priorities, milestones, and decisions. Collaboration platforms should be selected according to institutional approval, data location, access controls, export capability, and retention policy—not merely popularity.

Electronic laboratory notebooks and computational notebooks

Electronic laboratory notebooks preserve hypotheses, protocols, observations, decisions, and dates. Computational notebooks combine code, narrative, visualisation, and output, making exploratory analysis accessible. However, notebooks can hide execution order, depend on undeclared state, and become difficult to review. Good practice includes restarting and running all cells from top to bottom, avoiding hidden manual edits, recording software versions, separating exploration from final pipelines, and exporting results with provenance.

A notebook should not be the only record of a production workflow. Reusable analysis should be moved into tested scripts or packages, with parameters stored in configuration files. Data-processing pipelines should make inputs, outputs, and transformations explicit.

Repositories and persistent identification

Institutional and disciplinary repositories support preservation, discovery, citation, and controlled access. A repository record should include metadata, a licence, a version or release date, and a persistent identifier such as a DOI where appropriate. Deposit decisions should distinguish raw restricted data from shareable derivatives, documentation, code, and synthetic or aggregated data. Repository selection should consider preservation policy, size limits, access controls, community standards, and whether the repository is recognised by the relevant discipline.

8. Responsible Use of Generative AI

Generative-AI systems can assist with brainstorming, code explanation, documentation drafts, translation, test-case generation, and interface design. They can also produce fabricated references, insecure code, biased classifications, leaked confidential information, or text that obscures authorship and accountability.

A responsible workflow includes:

1. Check institutional, funder, publisher, and course policies.

2. Do not upload confidential, personal, proprietary, or restricted research data to an unapproved service.

3. Treat generated text, code, citations, and analyses as unverified drafts.

4. Test code, inspect dependencies, and review for security and licensing issues.

5. Preserve a record of significant AI assistance, including tool, date, purpose, and human verification.

6. Disclose use when required and retain human responsibility for the final work.

Prompting a model with a sensitive dataset may create a disclosure risk even if the output appears harmless. Local or institutionally governed models may reduce some risks but still require access control, evaluation, and policy compliance. AI-generated synthetic data are not automatically private: they may reproduce memorised examples or encode biases. Their generation process, quality checks, and limitations must be documented.

9. Worked Example: Planning an Urban Mobility Dataset

The following is an illustrative case. A university team proposes an ML system to predict bicycle congestion at major intersections. It plans to combine bicycle-counter data, weather observations, road images, and short voluntary travel surveys.

Background and problem

The team initially proposes to store all data in a shared cloud folder, ask students to label images informally, and publish the complete dataset with the final paper. This plan creates risks: images may contain faces and vehicle plates; survey responses may be identifiable through time and route; labels may be inconsistent; and the cloud service may not meet institutional requirements.

Analysis

The research questions require temporal and spatial information, but not direct identity. The team therefore separates data into sensitivity classes. Counter and weather observations are low risk after quality checks. Images are potentially personal data. Survey responses require consent, minimisation, and restricted access. The project also identifies a licensing question for third-party weather data and a governance question concerning publication of images from public places.

Approach and solution

The team creates a DMP with the following controls:

• Raw images are stored in encrypted institutional storage with access limited to trained researchers.
• A preprocessing pipeline detects and blurs faces and plates before creation of a shareable derivative.
• Original images are retained only for a justified period and then securely deleted or placed under approved restricted retention.
• Survey identifiers are separated from responses; the linkage file is held by the data steward.
• A codebook defines congestion categories and examples. Two annotators label a pilot sample, disagreements are reviewed, and the protocol is revised before full annotation.
• Git stores code and documentation, while large files and data releases use an approved repository.
• A README, data dictionary, provenance record, environment specification, and change log accompany the release.
• Access is role-based, multi-factor authentication is required, and restoration of backups is tested.
• The public release contains aggregated counter data, de-identified survey summaries, blurred image derivatives where lawful, and all permissible code. Restricted materials are described in metadata with a request-based access procedure.
• The team records external-data licences and cites each source. Any generative-AI assistance used for code explanation is reviewed, documented, and excluded from handling restricted data.

Outcome

The resulting dataset is less complete in a superficial sense because some raw images cannot be openly released. Nevertheless, it is more defensible, reusable, and trustworthy. External researchers can reproduce the published analysis from approved derivatives and code, while the access process protects participants and third-party rights.

Learning points

The case demonstrates that openness is not binary; controlled access can satisfy accessibility while protecting people. Annotation is a measurement activity requiring a protocol. Security, metadata, licences, and reproducibility should be designed together rather than added after analysis. Most importantly, the DMP follows the research questions and risks instead of treating every file identically.

10. Contemporary Developments and Professional Implications

Research infrastructures are moving toward machine-actionable metadata, automated provenance capture, data-quality monitoring, privacy-preserving computation, federated analysis, and reproducible workflow platforms. Cloud services can provide scalable computation, but they introduce questions about jurisdiction, cost, vendor dependence, and data egress. Federated approaches may allow analysis to move to data rather than copying sensitive records, although they require compatible infrastructure and careful security evaluation.

Employers increasingly expect graduates to understand data stewardship, version control, secure collaboration, documentation, and auditability. In regulated or high-impact AI settings, the ability to explain data lineage and access decisions may be as important as model accuracy. A well-managed project also reduces duplicated work, shortens onboarding, supports peer review, and improves the credibility of technical claims.

Challenges and Limitations

No DMP can eliminate uncertainty. Long-term preservation costs money, file formats become obsolete, repositories may change policy, and metadata can become detached from data. Anonymisation is difficult for high-dimensional or linked datasets. Open publication can conflict with privacy or community expectations, while excessive restriction can prevent useful scrutiny. Automated annotation may scale rapidly but reproduce systematic errors. Digital tools can create concentration of power in commercial platforms and can fail during outages.

Researchers should therefore review plans at defined milestones, involve ethics and information-security specialists when needed, document deviations, and be candid about what cannot be shared or reproduced. Governance is a continuing practice, not a form completed once.

Best-Practice Checklist

Before collection, define the research purpose, lawful and ethical basis, roles, risks, formats, consent, and retention. During collection and analysis, maintain provenance, validate data, use controlled access, version code, document transformations, and test backups. Before sharing, verify licences, remove or protect identifiers, prepare metadata and README files, record the release version, and obtain appropriate approval. Throughout the project, treat external services and generative-AI tools as governed components of the research environment.

Chapter Summary

Research data management connects research design, ethical responsibility, security, reproducibility, and scholarly communication. A DMP should describe data and artefacts across their life cycle, including collection, annotation, quality assurance, metadata, storage, backup, access, licensing, preservation, sharing, and disposal. FAIR principles improve discoverability and reuse, but FAIR does not require unrestricted access. Provenance, codebooks, version control, notebooks, repositories, and persistent identifiers make computational work more understandable and auditable. Security requires authentication, least privilege, encryption, logging, tested backups, and incident planning. Privacy, consent, licensing, and community governance constrain what may be collected and released. Generative AI can support research tasks, but it must not replace verification, authorship accountability, confidentiality controls, or scholarly judgement. Effective data stewardship is therefore both a technical competence and a professional obligation.

Key Terms

Data-management plan: A document specifying how research data will be handled throughout a project.

Data life cycle: The stages through which data pass from planning and collection to preservation, sharing, or disposal.

Provenance: Information about the origin, history, and transformations of a research object.

Metadata: Structured information describing a dataset, file, process, or access condition.

FAIR principles: Findability, Accessibility, Interoperability, and Reusability principles for research objects.

Data dictionary: A reference defining variables, values, units, and missing-data conventions.

Codebook: Documentation defining annotation categories and decision rules.

Annotation: The assignment of labels or structured judgements to data.

Pseudonymisation: Replacing direct identifiers while retaining a controlled possibility of linkage.

Anonymisation: Processing intended to prevent identification, subject to residual re-identification risk.

Least privilege: Providing only the access permissions necessary for a role or task.

Encryption: Transforming information so that unauthorised parties cannot readily read it.

Version control: A system for recording and managing changes to files and collaborative work.

Repository: A managed service for storing, preserving, describing, and sharing research objects.

Persistent identifier: A durable reference to a digital object, such as a DOI.

Reproducibility: The ability of another researcher to obtain consistent results using documented original materials and procedures.

Licence: Legal terms specifying permitted uses, conditions, and restrictions.

Governance: Policies, responsibilities, decision rights, and controls for managing data.

Generative AI: An AI system that produces content such as text, code, images, or audio in response to inputs.

Review Questions

Short Answer Questions

1. What are the main purposes of a data-management plan?

2. Distinguish raw, processed, and derived data.

3. Why is a README important in a computational research project?

4. Explain why synchronisation is not equivalent to backup.

5. What does the principle of least privilege require?

Long Answer Questions

1. Explain how FAIR principles can be applied without requiring unrestricted access to sensitive research data.

2. Discuss the role of annotation protocols, pilot annotation, and quality assurance in an ML project.

3. Compare the functions of version-control systems, computational notebooks, and repositories.

4. Analyse the ethical, legal, and technical factors that should influence data sharing.

5. Develop a structured DMP outline for a project using personal or commercially sensitive data.

Analytical / Application Questions

1. A research team wants to upload identifiable interview transcripts to a free online AI service for summarisation. Evaluate the risks and propose a responsible alternative workflow.

2. An image dataset has high annotation agreement but contains very few examples from one demographic group. Analyse why agreement alone is insufficient and recommend further quality and fairness checks.

3. Design an access, storage, backup, and sharing strategy for a multi-institutional ML project involving restricted raw data and openly shareable derived data.

Multiple Choice Questions with Answers

Question 1

Which statement best describes FAIR data principles?

A. All research data must be published without restrictions

B. Data should be findable, accessible, interoperable, and reusable

C. Data should be stored only in commercial cloud systems

D. Data should be collected exclusively through automated tools

Correct Answer: B. Data should be findable, accessible, interoperable, and reusable

Question 2

What is the primary purpose of a data dictionary?

A. To encrypt research files

B. To assign repository permissions

C. To define variables, units, values, and missing-data conventions

D. To generate synthetic observations

Correct Answer: C. To define variables, units, values, and missing-data conventions

Question 3

Which practice best supports reproducibility in an ML experiment?

A. Recording software versions, configuration, and split logic

B. Changing preprocessing manually for each run

C. Keeping the final dataset without its processing code

D. Relying on memory to report parameter choices

Correct Answer: A. Recording software versions, configuration, and split logic

Question 4

Why is pseudonymisation not necessarily anonymisation?

A. Pseudonymised data cannot be encrypted

B. Pseudonymised data may still be linked to identities using additional information

C. Pseudonymisation removes all metadata

D. Pseudonymised data cannot be used for research

Correct Answer: B. Pseudonymised data may still be linked to identities using additional information

Question 5

Under the 3-2-1 backup heuristic, which arrangement is recommended?

A. Three copies on one laptop

B. Two copies on one removable drive

C. Three copies, two storage types, and one geographically separate copy

D. One encrypted copy with three passwords

Correct Answer: C. Three copies, two storage types, and one geographically separate copy

Question 6

What does least privilege mean in data security?

A. Every team member receives administrator access

B. Users receive only the permissions needed for their roles

C. Data are made publicly accessible by default

D. Passwords are shared within the research group

Correct Answer: B. Users receive only the permissions needed for their roles

Question 7

Which item is most directly an annotation codebook function?

A. Defining label categories and decision rules

B. Scheduling repository maintenance

C. Encrypting database backups

D. Calculating cloud-computing costs

Correct Answer: A. Defining label categories and decision rules

Question 8

Why should a public code repository not automatically contain all project data?

A. Repositories cannot store text files

B. Data may be too large, sensitive, or subject to licence restrictions

C. Version control prevents collaboration

D. Public repositories cannot contain documentation

Correct Answer: B. Data may be too large, sensitive, or subject to licence restrictions

Question 9

What is an appropriate first response to AI-generated research code?

A. Use it without testing because it is generated by AI

B. Publish it immediately under any licence

C. Review, test, and check its security and licensing implications

D. Delete all documentation about its use

Correct Answer: C. Review, test, and check its security and licensing implications

Question 10

Which statement about data sharing is most accurate?

A. Public availability always means unrestricted reuse

B. Controlled access can be appropriate for sensitive but valuable data

C. Licences apply only to software, not datasets

D. Consent is unnecessary when data are technically easy to collect

Correct Answer: B. Controlled access can be appropriate for sensitive but valuable data