Error while running dada2

Hello!

I keep getting this error massage while running dada2:

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

The command I ran was: (QIIME2 06.2020)

qiime dada2 denoise-paired \
  --i-demultiplexed-seqs demux-paired-end.qza \
  --p-trim-left-f 20 \
  --p-trim-left-r 20 \
  --p-trunc-len-f 280 \
  --p-trunc-len-r 250 \
--p-n-threads 4 \
  --o-table table.qza \
  --o-representative-sequences rep-seqs.qza \
  --o-denoising-stats denoising-stats.qza --verbose

Can you please help me?

Thank!

Hello @ShayF. I see you ran the command with --verbose which should give you a full debug traceback, but that doesn’t look like the full traceback. When your command failed, was there any additional information along with

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1),
please inspect stdout and stderr to learn more.

See above for debug info.

Thank you.

1 Like

Hi! Thanks for the reply!
The full traceback:

qiime dada2 denoise-paired \
>   --i-demultiplexed-seqs demux-paired-end.qza \
>   --p-trim-left-f 20 \
>   --p-trim-left-r 20 \
>   --p-trunc-len-f 280 \
>   --p-trunc-len-r 250 \
> --p-n-threads 4 \
>   --o-table table.qza \
>   --o-representative-sequences rep-seqs.qza \
>   --o-denoising-stats denoising-stats.qza --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmp_wp6lcsw/forward /tmp/tmp_wp6lcsw/reverse /tmp/tmp_wp6lcsw/output.tsv.biom /tmp/tmp_wp6lcsw/track.tsv /tmp/tmp_wp6lcsw/filt_f /tmp/tmp_wp6lcsw/filt_r 280 250 20 20 2.0 2.0 2 independent consensus 1.0 4 1000000

R version 3.5.1 (2018-07-02) 
Loading required package: Rcpp
DADA2: 1.10.0 / Rcpp: 1.0.4.6 / RcppParallel: 5.0.0 
1) Filtering 
 *** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: readBin(con, raw(), n)
 2: reader(con, readerBlockSize)
 3: x$yield(...)
 4: yield(fR)
 5: yield(fR)
 6: withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))
 7: suppressWarnings(fqR <- yield(fR))
 8: (function (fn, fout, maxN = c(0, 0), truncQ = c(2, 2), truncLen = c(0,     0), maxLen = c(Inf, Inf), minLen = c(20, 20), trimLeft = c(0,     0), trimRight = c(0, 0), minQ = c(0, 0), maxEE = c(Inf, Inf),     rm.phix = c(TRUE, TRUE), matchIDs = FALSE, orient.fwd = NULL,     id.sep = "\\s", id.field = NULL, n = 1e+06, OMP = TRUE, compress = TRUE,     verbose = FALSE, ...) {    if (!OMP) {        ompthreads <- .Call(ShortRead:::.set_omp_threads, 1L)        on.exit(.Call(ShortRead:::.set_omp_threads, ompthreads))    }    if (!is.character(fn) || length(fn) != 2)         stop("Two paired input file names required.")    if (!is.character(fout) || length(fout) != 2)         stop("Two paired output file names required.")    if (any(duplicated(c(fn, fout)))) {        stop("The output and input file names must be different.")    }    for (var in c("maxN", "truncQ", "truncLen", "maxLen", "minLen",         "trimLeft", "trimRight", "minQ", "maxEE", "rm.phix")) {        if (length(get(var)) == 1) {            assign(var, c(get(var), get(var)))        }        if (length(get(var)) != 2) {            stop(paste("Input variable", var, "must be length 1 or 2 (Forward, Reverse)."))        }    }    startF <- max(1, trimLeft[[1]] + 1, na.rm = TRUE)    startR <- max(1, trimLeft[[2]] + 1, na.rm = TRUE)    endF <- truncLen[[1]]    if (endF < startF) {        endF = NA    }    endF <- endF - startF + 1    endR <- truncLen[[2]]    if (endR < startR) {        endR = NA    }    endR <- endR - startR + 1    fF <- FastqStreamer(fn[[1]], n = n)    on.exit(close(fF))    fR <- FastqStreamer(fn[[2]], n = n)    on.exit(close(fR), add = TRUE)    if (file.exists(fout[[1]])) {        if (file.remove(fout[[1]])) {            if (verbose)                 message("Overwriting file:", fout[[1]])        }        else {            stop("Failed to overwrite file:", fout[[1]])        }    }    if (file.exists(fout[[2]])) {        if (file.remove(fout[[2]])) {            if (verbose)                 message("Overwriting file:", fout[[2]])        }        else {            stop("Failed to overwrite file:", fout[[2]])        }    }    first = TRUE    remainderF <- ShortReadQ()    remainderR <- ShortReadQ()    casava <- "Undetermined"    inseqs = 0    outseqs = 0    while (TRUE) {        suppressWarnings(fqF <- yield(fF))        suppressWarnings(fqR <- yield(fR))        if (length(fqF) == 0 && length(fqR) == 0) {            break        }        inseqs <- inseqs + length(fqF)        if (matchIDs) {            if (first) {                if (is.null(id.field)) {                  id1 <- as.character(id(fqF)[[1]])                  id.fields <- strsplit(id1, id.sep)[[1]]                  ncolon <- sapply(gregexpr(":", id.fields),                     length)                  ncoltab <- table(ncolon)                  if (max(ncolon) == 6 && ncoltab["6"] == 1) {                    casava <- "Current"                    id.field <- which(ncolon == 6)                  }                  else if (max(ncolon) == 4 && ncoltab["4"] ==                     1) {                    casava <- "Old"                    id.field <- which(ncolon == 4)                  }                  else {                    stop("Couldn't automatically detect the sequence identifier field in the fastq id string.")                  }                }            }            else {                fqF <- append(remainderF, fqF)                fqR <- append(remainderR, fqR)            }        }        else {            if (length(fqF) != length(fqR))                 stop("Mismatched forward and reverse sequence files: ",                   length(fqF), ", ", length(fqR), ".")        }        if (matchIDs) {            idsF <- sapply(strsplit(as.character(id(fqF)), id.sep),                 `[`, id.field)            idsR <- sapply(strsplit(as.character(id(fqR)), id.sep),                 `[`, id.field)            if (casava == "Old") {                idsF <- sapply(strsplit(idsF, "#"), `[`, 1)            }            lastF <- max(c(0, which(idsF %in% idsR)))            lastR <- max(c(0, which(idsR %in% idsF)))            if (lastF < length(fqF)) {                remainderF <- fqF[(lastF + 1):length(fqF)]            }            else {                remainderF <- ShortReadQ()            }            if (lastR < length(fqR)) {                remainderR <- fqR[(lastR + 1):length(fqR)]            }            else {                remainderR <- ShortReadQ()            }            fqF <- fqF[idsF %in% idsR]            fqR <- fqR[idsR %in% idsF]        }        if (!is.null(orient.fwd)) {            if (!C_isACGT(orient.fwd))                 stop("Non-ACGT characters detected in orient.fwd")            barlen <- nchar(orient.fwd)            keepF <- narrow(sread(fqF), 1, barlen) == orient.fwd            keepR <- (narrow(sread(fqR), 1, barlen) == orient.fwd) &                 !keepF            fq <- ShortReadQ(sread = c(sread(fqF[keepF]), sread(fqR[keepR])),                 quality = c(quality(quality(fqF[keepF])), quality(quality(fqR[keepR]))),                 id = c(id(fqF[keepF]), id(fqR[keepR])))            fqR <- ShortReadQ(sread = c(sread(fqR[keepF]), sread(fqF[keepR])),                 quality = c(quality(quality(fqR[keepF])), quality(quality(fqF[keepR]))),                 id = c(id(fqR[keepF]), id(fqF[keepR])))            fqF <- fq            rm(fq)        }        if (is.finite(maxLen[[1]]) || is.finite(maxLen[[2]])) {            keep <- width(fqF) <= maxLen[[1]] & width(fqR) <=                 maxLen[[2]]            fqF <- fqF[keep]            fqR <- fqR[keep]        }        keep <- (width(fqF) >= startF & width(fqR) >= startR)        fqF <- fqF[keep]        fqF <- narrow(fqF, start = startF, end = NA)        fqR <- fqR[keep]        fqR <- narrow(fqR, start = startR, end = NA)        if (trimRight[[1]] > 0) {            keep <- width(fqF) > trimRight[[1]]            fqF <- fqF[keep]            fqR <- fqR[keep]            fqF <- narrow(fqF, start = NA, end = width(fqF) -                 trimRight[[1]])        }        if (trimRight[[2]] > 0) {            keep <- width(fqR) > trimRight[[2]]            fqF <- fqF[keep]            fqR <- fqR[keep]            fqR <- narrow(fqR, start = NA, end = width(fqR) -                 trimRight[[2]])        }        encF <- encoding(quality(fqF))        encR <- encoding(quality(fqR))        if (is.numeric(truncQ)) {            indF <- which(encF == truncQ[[1]])            indR <- which(encR == truncQ[[2]])            if (!(length(indF) == 1 && length(indR) == 1))                 stop("Encoding for this truncQ value not found.")            truncQ <- c(names(encF)[[indF]], names(encR)[[indR]])        }        if (length(fqF) > 0) {            rngF <- trimTails(fqF, 1, truncQ[[1]], ranges = TRUE)            fqF <- narrow(fqF, 1, end(rngF))        }        if (length(fqR) > 0) {            rngR <- trimTails(fqR, 1, truncQ[[2]], ranges = TRUE)            fqR <- narrow(fqR, 1, end(rngR))        }        truncQ <- c(encF[truncQ[1]], encR[truncQ[2]])        keep <- (width(fqF) > 0 & width(fqR) > 0)        fqF <- fqF[keep]        fqR <- fqR[keep]        keep <- rep(TRUE, length(fqF))        if (!is.na(endF)) {            keep <- keep & (width(fqF) >= endF)        }        if (!is.na(endR)) {            keep <- keep & (width(fqR) >= endR)        }        fqF <- fqF[keep]        fqR <- fqR[keep]        fqF <- narrow(fqF, start = 1, end = endF)        fqR <- narrow(fqR, start = 1, end = endR)        keep <- width(fqF) >= minLen[[1]] & width(fqR) >= minLen[[2]]        fqF <- fqF[keep]        fqR <- fqR[keep]        suppressWarnings(keep <- nFilter(maxN[[1]])(fqF) & nFilter(maxN[[2]])(fqR))        fqF <- fqF[keep]        fqR <- fqR[keep]        keep <- rep(TRUE, length(fqF))        qmat <- as(quality(fqF), "matrix")        if (minQ[[1]] > truncQ[[1]])             suppressWarnings(keep <- keep & (apply(qmat, 1, min,                 na.rm = TRUE) > minQ[[1]]))        if (maxEE[[1]] < Inf)             keep <- keep & C_matrixEE(qmat) <= maxEE[[1]]        qmat <- as(quality(fqR), "matrix")        if (minQ[[2]] > truncQ[[2]])             suppressWarnings(keep <- keep & (apply(qmat, 1, min,                 na.rm = TRUE) > minQ[[2]]))        if (maxEE[[2]] < Inf)             keep <- keep & C_matrixEE(qmat) <= maxEE[[2]]        fqF <- fqF[keep]        fqR <- fqR[keep]        rm(qmat)        if (length(fqF) != length(fqR))             stop("Filtering caused mismatch between forward and reverse sequence lists: ",                 length(fqF), ", ", length(fqR), ".")        if (rm.phix[[1]] && rm.phix[[2]]) {            is.phi <- isPhiX(as(sread(fqF), "character"), ...)            is.phi <- is.phi | isPhiX(as(sread(fqR), "character"),                 ...)        }        else if (rm.phix[[1]] && !rm.phix[[2]]) {            is.phi <- isPhiX(as(sread(fqF), "character"), ...)        }        else if (!rm.phix[[1]] && rm.phix[[2]]) {            is.phi <- isPhiX(as(sread(fqR), "character"), ...)        }        if (any(rm.phix)) {            fqF <- fqF[!is.phi]            fqR <- fqR[!is.phi]        }        outseqs <- outseqs + length(fqF)        if (first) {            writeFastq(fqF, fout[[1]], "w", compress = compress)            writeFastq(fqR, fout[[2]], "w", compress = compress)            first = FALSE        }        else {            writeFastq(fqF, fout[[1]], "a", compress = compress)            writeFastq(fqR, fout[[2]], "a", compress = compress)        }    }    if (outseqs == 0) {    }    if (verbose) {        outperc <- round(outseqs * 100/inseqs, 1)        outperc <- paste(" (", outperc, "%)", sep = "")        message("Read in ", inseqs, " paired-sequences, output ",             outseqs, outperc, " filtered paired-sequences.",             sep = "")    }    if (outseqs == 0) {        message(paste("The filter removed all reads:", fout[[1]],             "and", fout[[2]], "not written."))        file.remove(fout[[1]])        file.remove(fout[[2]])    }    return(invisible(c(reads.in = inseqs, reads.out = outseqs)))})(dots[[1L]][[1L]], dots[[2L]][[1L]], truncQ = 2L, truncLen = c(280L, 250L), trimLeft = c(20L, 20L), trimRight = 0, maxLen = Inf, minLen = 20,     maxN = 0, minQ = 0, maxEE = c(2, 2), rm.phix = TRUE, orient.fwd = NULL,     matchIDs = FALSE, id.sep = "\\s", id.field = NULL, n = 1e+05,     OMP = FALSE, compress = TRUE, verbose = FALSE)
 9: .mapply(FUN, dots, MoreArgs)
10: FUN(X[[i]], ...)
11: lapply(X = S, FUN = FUN, ...)
12: doTryCatch(return(expr), name, parentenv, handler)
13: tryCatchOne(expr, names, parentenv, handlers[[1L]])
14: tryCatchList(expr, classes, parentenv, handlers)
15: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
16: try(lapply(X = S, FUN = FUN, ...), silent = TRUE)
17: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE))
18: FUN(X[[i]], ...)
19: lapply(seq_len(cores), inner.do)
20: mclapply(seq_len(n), do_one, mc.preschedule = mc.preschedule,     mc.set.seed = mc.set.seed, mc.silent = mc.silent, mc.cores = mc.cores,     mc.cleanup = mc.cleanup, affinity.list = affinity.list)
21: mcmapply(fastqPairedFilter, mapply(c, fwd, rev, SIMPLIFY = FALSE),     mapply(c, filt, filt.rev, SIMPLIFY = FALSE), MoreArgs = list(truncQ = truncQ,         truncLen = truncLen, trimLeft = trimLeft, trimRight = trimRight,         maxLen = maxLen, minLen = minLen, maxN = maxN, minQ = minQ,         maxEE = maxEE, rm.phix = rm.phix, orient.fwd = orient.fwd,         matchIDs = matchIDs, id.sep = id.sep, id.field = id.field,         n = n, OMP = OMP, compress = compress, verbose = verbose),     mc.cores = ncores, mc.silent = TRUE)
22: filterAndTrim(unfiltsF, filtsF, unfiltsR, filtsR, truncLen = c(truncLenF,     truncLenR), trimLeft = c(trimLeftF, trimLeftR), maxEE = c(maxEEF,     maxEER), truncQ = truncQ, rm.phix = TRUE, multithread = multithread)
23: withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))
24: suppressWarnings(filterAndTrim(unfiltsF, filtsF, unfiltsR, filtsR,     truncLen = c(truncLenF, truncLenR), trimLeft = c(trimLeftF,         trimLeftR), maxEE = c(maxEEF, maxEER), truncQ = truncQ,     rm.phix = TRUE, multithread = multithread))
An irrecoverable exception occurred. R is aborting now ...
Error in names(answer) <- names1 : 
  'names' attribute [80] must be the same length as the vector [60]
Execution halted
Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 264, in denoise_paired
    run_commands([cmd])
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmp_wp6lcsw/forward', '/tmp/tmp_wp6lcsw/reverse', '/tmp/tmp_wp6lcsw/output.tsv.biom', '/tmp/tmp_wp6lcsw/track.tsv', '/tmp/tmp_wp6lcsw/filt_f', '/tmp/tmp_wp6lcsw/filt_r', '280', '250', '20', '20', '2.0', '2.0', '2', 'independent', 'consensus', '1.0', '4', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-499>", line 2, in denoise_paired
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2020.6/lib/python3.6/site-packages/q2_dada2/_denoise.py", line 279, in denoise_paired
    " and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

Hello @ShayF. Thanks for the traceback. That error suggests that the code is attempting to apply 80 names to 60 data points, and it is erroring because it wants as many data points as there are names. It isn’t immediately clear to me why this would be happening, but I’ll look into it.

Thank you! I really appreciate your help.
If it helps, I tried running some older files with the commands I previously used, but suddenly I started getting the same error… Very strange. Is there any chance it is related to R? as the last line of the traceback is-

Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Yes the error is definitely happening while running dada2 in R. One of the other admins pointed me towards this post, and it looks like this error has to do with a multi threading issue. Try reducing the number of threads you’re using (all the way down to one if you have to) and see if that fixes it. If you’re using more than one thread and you encounter an error a good rule of thumb is probably to try re-running the command with one thread and seeing if the error persists. Multi threading can be great when it works, but as we’ve seen here it can also cause some pretty difficult to track down errors.

1 Like

@Oddant1
I tried running it without multi-threading. Still got the sample error.
I started R Studio, and got this error:

During startup - Warning message:
Setting LC_CTYPE= failed

Is there any chance it’s related?

Thanks!

I’m not sure. Does that message prevent you from opening R Studio? Additionally, what kind of environment are you running QIIME 2 in (native install on Linux/Mac, Virtualbox, etc.).