This page collects common error messages and how to resolve them. For general usage questions, see the FAQ.
Nextflow process exit codes
When a step fails, Nextflow reports the process and an exit status, for example:
Caused by:
Process `WF2:ANNOTATE:annotate (SAMPLE_ID)` terminated with an error exit status (140)The number identifies the cause. Whether the step is retried depends on which step it is:
| Step | On failure |
|---|---|
ASSEMBLE |
Retried on exit 137-140 only, up to
assemble.maxRetries. Each attempt requests more
memory. |
SCAFFOLD_JOIN, BLAST_GENBANK,
BLAST_REF_FETCH
|
Retried up to 3 times (these fail mostly on transient network or resource problems). |
PREPROCESS, COVERAGE,
ANNOTATE, CURATE, ORF,
VALIDATE, BLAST_REF_ALIGN
|
Not retried. The sample is skipped and the run continues without it. |
For the steps that are not retried there is no free second attempt: the sample keeps whatever state it had before, and the only way forward is to change the options (usually memory) and rerun. Because the run continues past a skipped sample and then exits with a non-zero status at the end, a red exit code alongside a handful of per-sample warnings is the normal shape of a partly failed run, not a sign that the whole batch died.
“exit status (137)”
Exit 137 means the process was killed with SIGKILL (137
= 128 + 9). The usual cause is running out of memory: the container or
the cluster scheduler enforces a memory limit, and a process that
exceeds it is killed outright rather than allowed to finish. It can also
appear when a job hits a scheduler time limit, or when a container is
stopped manually.
For ASSEMBLE, each retry requests more memory (the
request is multiplied by the attempt number), so a one-off 137 that then
succeeds needs no action.
For every other step a 137 is final. Increase the requested memory in
the R Shiny GUI for the process that failed, then rerun the workflow
(the same fix as exit status (140) below). If raising
memory does not help, check your cluster’s walltime limit: a job killed
at the time limit also reports 137.
“exit status (140)”
If your Nextflow log contains an error that looks something like this:
Caused by:
Process `WF2:ANNOTATE:annotate (SAMPLE_ID)` terminated with an error exit status (140)Usually this indicates that the process ran out of memory (RAM). Sometimes this will happen for all samples, other times only certain difficult samples will need more RAM. Try using the R Shiny GUI to increase the requested memory for the process that failed, then rerun the workflow.
“exit status (127)”
This indicates that MitoPilot was unable to find a file, directory, or executable. Double check that the paths you specified when creating your project are correct. Also make sure you’re using the correct execution environment. This error is commonly caused by using the “local” executor when running MitoPilot on a computing cluster.
Other errors
Inspecting the project database directly
MitoPilot stores sample metadata, processing parameters, and results
in a .sqlite database in the project directory, created
automatically when the project is initialized. The app is the normal way
to read it, but when troubleshooting it is often quicker to query it
yourself.
From R, use {DBI} and {dbplyr}, which the package itself relies on heavily. Outside R, a graphical client such as DB Browser for SQLite works well. Close the app first if you intend to write anything.
“database disk image is malformed”
You may encounter an error message in your R Studio console that looks something like this:
Listening on http://127.0.0.1:4329
Warning: Couldn't set synchronous mode: database disk image is malformed
Use `synchronous` = NULL to turn off this warning.
Database attached: /pool/public/genomics/macguigand/MitoPilot/testing/turtle_test/run_01/.sqlite
Warning: Error in db_query_fields.DBIConnection: Can't query fields.
ℹ Using SQL: SELECT * FROM `pre_opts` AS `q02` WHERE (0 = 1)
Caused by error:
! database disk image is malformed
...This indicates that your .sqlite database is corrupted.
The exact cause of this error can be difficult to determine.
Fortunately, you may be able to recover your database by running the
following commands in your project directory.
Then try relaunching the MitoPilot app.
“Insufficient memory for the Java Runtime Environment”
This suggests that the Nextflow head process (which controls all other jobs) is running out of Java heap space. To fix, run the following command in the console or add it to your submission submission script before launching Nextflow.
Alternatively, submit the Nextflow job with fewer samples at time.