Curation and Validation

Automated curation and validation of mitogenome annotations is a central goal of MitoPilot. In the Annotate module, annotations are first generated by MITOS2 (rRNA and protein-coding genes) and tRNAscan-SE (tRNAs), optionally supplemented by MitoFinder, ARWEN, and ARAGORN. Those raw annotations are then passed through curation and validation.

This page describes what those two steps do, how to configure them, and how to read the warnings they produce. It applies to every clade; ray-finned fishes are used throughout as the worked example because that is the default ruleset.

Curation changes, validation only flags

These are two separate steps and the distinction matters.

Curation edits your annotations. It moves gene boundaries, trims stop codons that run into the next gene, applies the punctuation model, and labels exons. Every change it makes is recorded in the Notes column of the annotation table, so you can always see what was altered and by how much. Typical notes read extending start 3 bp, trimming end 12 bp, stop codon trimmed by 2 bp, or Applied punctuation model - moved pos1 by 1 bp.

Validation changes nothing. It compares the curated annotations against the expectations in the ruleset and writes the Warnings column. A warning is a prompt to look, not a statement that something is broken.

Note. Because curation modifies coordinates, running the wrong ruleset for your clade does not merely produce bad warnings, it produces edited gene models. Set curate_target to match your organism before you run the Annotate module.

Choosing a ruleset

The ruleset determines expected gene content, allowed start and stop codons, length limits, permitted overlaps, gene naming, and the genetic code. You can set it either when you create the project or later in the app.

At project initialization, pass curate_target. This becomes the default for every sample in the project:

new_project(..., curate_target = "octocoral_mito")

In the app, open the Curate Opts. window from the Annotate module and use the Target dropdown, which lists every available ruleset as “Scientific name (common name)”. As with the other options windows, nothing is editable until you tick Edit, and changes apply to the rows you have selected. Because curation options are saved as named sets, different samples in the same project can use different rulesets, which is what you want for a mixed run.

Warning. Changing the ruleset only affects samples the next time they are curated. Samples already curated under the old ruleset keep their existing gene models until you reset their state and re-run the Annotate module.

The curation ruleset browser lists every ruleset with its testing status and parameter values.

If there is no ruleset for your clade

There is no generic fallback ruleset, and building one yourself is not recommended: getting the expected gene content, start and stop codons, length limits, and genetic code right for a new clade takes real curation experience, and a ruleset that is subtly wrong will silently edit your gene models.

If MitoPilot does not cover your group, please open an issue or contact Dan MacGuigan at macguigand@si.edu. We are actively adding rulesets and are glad to work with you on one for your taxa.

In the meantime you can select the closest available clade to get annotations you can review by hand, but treat the automated curation and the validation warnings with caution, since both are being applied under the wrong expectations.

Curation parameters

Every ruleset is produced by a params_*() function, for example params_fish_mito(). Customize it either by supplying a complete named list, or by passing just the modifications you want. The example below raises the expected trnW count to 2 and lowers the accepted reference similarity to 85%:

new_project(
  ...,
  curate_params = params_fish_mito(
    list(
      hit_threshold = 85,
      rules = list(
        trnW = list(
          count = 2
        )
      )
    )
  )
)

What the parameters mean

Top-level parameters apply to the whole ruleset:

Parameter	Meaning
`genetic_code`	NCBI translation table used to translate coding sequences and to define which start and stop codons are valid for this clade.
`hit_threshold`	Minimum percent amino-acid similarity to a reference for a BLAST hit to be used. Protein-coding genes with no hit at or above this value are skipped during curation and flagged `low reference similarity`. Default 90.
`max_overlap`	Global cap on how much an annotation may overlap a neighboring gene on the same strand, as a fraction of its length. Default 0.25.
`rules`	A named list, one entry per feature, holding the per-gene settings below.

Per-gene entries inside rules:

Field	Meaning
`count`	Expected number of copies. Finding fewer reports the gene as missing; finding more triggers `possible duplicate`.
`min_len` / `max_len`	Expected length bounds in base pairs. `NA` means unenforced.
`overlap`	Permitted overlap with adjacent genes. `start` is the maximum base pairs the feature’s start may overlap a neighbor before a warning; `stop: yes` lets the stop codon overlap the next gene, so it is neither trimmed nor flagged.
`start_codons`	Codons accepted as valid starts. Curation snaps the start to the nearest of these.
`stop_codons`	Codons accepted as valid stops, including truncated `T` or `TA` stops.
`intron`	Whether the gene may contain introns. If yes, multiple fragments are labeled as exons rather than flagged as duplicates.

Why per-gene overlaps exist. Overlapping genes are a real feature of mitochondrial genomes, not an annotation error. In many animals atp8 and atp6 overlap, as do nad4l and nad4, and several genes end in a truncated T or TA stop that is completed by polyadenylation after transcription rather than being encoded. The per-gene overlap and stop_codons settings are what stop curation from “fixing” this real biology.

The complete default parameter set for any ruleset can be viewed in the MitoPilot app under Curate Opts., and edited there per sample. The fish defaults are shown below.

Reference databases

Curation refines protein-coding gene boundaries by comparing each gene against a reference database of amino acid sequences, one BLAST database per gene. The reference database is selected independently of the MITOS2 annotation database. Bundled options are Metazoa_RefSeq235 (the default, and the only one carrying rRNA BLAST references), Metazoa_RefSeq231, Metazoa_RefSeq89, and Chordata.

MitoPilot also folds each sample’s own assembly BLAST hit into the curation references automatically, so the closest published relative of your sample is always among the sequences its boundaries are compared against.

To supply your own, use custom_curation_db(), described in building custom databases.

The ref_db property of the curation parameters sets the paths directly, if you need per-gene control. Its default is list(default = "/ref_dbs/Mitos2/Chordata/featureProt/{gene}.fas"), where {gene} is substituted per gene during processing. Gene-specific overrides can be added as named entries, for example list(default = "...", atp8 = "/ref_dbs/curation/atp8.fas"). These paths are resolved inside the execution environment, so any file you point at must be visible from within the container.

Validation warnings

After curation, each annotation is checked against the ruleset and any problems are written to the Warnings column. Use the “Warnings column includes” dropdown in the app to filter to one warning type and work through them as a batch.

Gene content and structure

Warning	Meaning
`possible duplicate`	The annotation occurs more often than the ruleset’s `count`. Every copy is flagged; you decide which, if any, to remove.
`exceeds max length`	Longer than the gene’s `max_len`.
`below min length`	Shorter than the gene’s `min_len`.
`tRNA within PCG or rRNA`	A tRNA falls entirely inside a protein-coding or ribosomal RNA gene, which usually means a spurious tRNA call.
`low-confidence tRNA (NNN anticodon)`	tRNAscan-SE could not resolve the anticodon. These only appear if you enabled `retain_low_conf_trna`.

Overlaps

Warning	Meaning
`exceeds max overlap`	Overlaps a neighbor on the same strand by more than `max_overlap` of its own length.
`exceeds max start overlap`	The start of a protein-coding gene overlaps a neighbor on the same strand more than the gene’s `overlap$start` allows.
`exceeds max stop overlap`	The stop of a protein-coding gene overlaps a neighbor on the same strand more than expected.

Codons

Warning	Meaning
`internal stop codon`	A stop codon was found inside a protein-coding gene.
`non-standard start codon`	The start codon is not in the gene’s `start_codons`.
`non-standard stop codon`	The stop codon is not in the gene’s `stop_codons`.

Reference agreement

Warning	Meaning
`low reference similarity`	No reference reaches `hit_threshold` amino-acid similarity, so curation could not refine this gene’s boundaries.
`check reference start alignment`	The start does not align exactly with the majority of top reference hits.
`check reference stop alignment`	The stop does not align exactly with the majority of top reference hits.

Assembly quality

Warning	Meaning
`low coverage region`	More than 5% of the region under this annotation has less than 10x coverage.
`high error region`	More than 5% of the region has a raw read error rate above 5%, meaning more than 5% of aligned reads disagree on the base call.
`<n> ambiguous bases in CDS`	The protein-coding gene contains bases that are not A, C, G, or T. Translation still resolves a codon when the ambiguous base does not change the amino acid, and writes `X` only where it cannot, so the protein may show no `X` at all. The bases may be `N`s MitoPilot inserted when joining fragments, ambiguity codes from a consensus, or bases the assembly arrived with. Check the affected region before submitting.

Note. The coverage and error thresholds in this last group are fixed at 5% / 10x and 5% / 5%. Unlike everything else on this page they are not configurable through the curation parameters. The ambiguous-bases warning appears in the same Warnings column and filters with the rest.

Warning. Warnings do not clear themselves after you edit an annotation by hand. They are written when validation runs, so re-run validation for the sample to refresh them. See the FAQ for the procedure.

What MitoPilot does not check

Validation targets problems that commonly cause GenBank submissions to be rejected or returned, but it is not a substitute for NCBI’s own checks. It does not screen for contamination or adaptor sequence, does not verify that your declared organism matches the sequence, and does not validate submission metadata. Expect NCBI to run its own screens after you submit.