2021: An error was encountered while running DADA2 in R (return code -9)

bettya · April 6, 2021, 8:53pm

Hi Chloe

Thanks for your response.
I used both 'p-rev-comp-barcodes' and 'p-rev-comp-mapping-barcodes' and it worked.
The command:

qiime demux emp-paired \

--m-barcodes-file sample-metadata.tsv
--m-barcodes-column BarcodeSequence
--p-rev-comp-barcodes
--p-rev-comp-mapping-barcodes
--p-no-golay-error-correction
--i-seqs emp-paired-end-sequences.qza
--o-per-sample-sequences demux-full.qza
--o-error-correction-details demux-details.qza
Hurray!
DENOISING PAIR-END READS
I have an issue with the denoising command. I do not want to trim my reads so I used this command:
qiime dada2 denoise-paired
--i-demultiplexed-seqs demux-full.qza
--p-trim-left-f 0
--p-trim-left-r 0
--p-trunc-len-f 151
--p-trunc-len-r 151
--o-table table.qza
--o-representative-sequences rep-seqs.qza
--o-denoising-stats denoising-stats.qza
Error:
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code -9), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-mk4sd79d.log
What am I missing please?

bettya · April 8, 2021, 7:39am

Hi Chloe

These are my plots and table summaries from the demux.qzv:

The region is V4, the primers are 515/806.
I have reads that comprises of 12 samples (12 forwards and 12 reverse) and no sample must be lost due to the type of analysis. Samples were run in 3 batches. I have 8nts in my reads. I believe I have a very good reads as shown by the bar plot.
What is the basis of trimming? How can I trim and merge the reads and still get a good cluster for my samples?
Thank you for your timely response.

cherman2 · April 9, 2021, 4:19pm

Hello @bettya ,
Thank you for all the information
Unfortunately I am not sure what your error is to be honest.
Could you less /tmp/qiime2-q2cli-err-mk4sd79d.log and send me a screenshot of the result so that I can better understand the error? Or you could run the same command with the --verbose. parameter that would also give us more information. Currently I don't understand fully what the error is and so I don't know what the problem is.

The reason for trimming is to remove your primers if they are still in the sequences. :qiime2:

You should be able to trim at 0 if you don't have a primer in your sequences. I do it all the time! Again I think I just need more information about the original error.
Thank you
Chloe

cherman2 · April 9, 2021, 7:38pm

Hey @bettya,
I think your issue is actually lack of memory.
If DADA2 throws a -9 error code it means that DADA2 was not able to complete the command with the amount of memory given.
So I would run this command on a computer that has more memory.
I hope this helps!
Chloe

bettya · April 17, 2021, 11:17am

Hi Chloe
It did work, thanks.

Can I ask on what basis do I have to cut part of my sample sequence during diversity analysis?
Why do I have to apply the sampling depth to my sequence? I do not understand why.
Can you please explain?

Regards.

cherman2 · April 20, 2021, 11:38pm

Hello @bettya,
I am not sure that I fully understand your question.I think you asking why you need a sampling depth and why it is worth it to throw out some samples and reads to accommodate the sampling depth?

So the short and sweet answer is that we can't investigate samples at different sampling depths. So if a sample doesn't have enough reads to accommodate the sampling depth they will be thrown out. And if a sample has too many reads for the set sampling depth you will not look at all of them. So that all the samples are comparable.

If they are not comparable. It is like going to a rain forest and marking off a 10 x 10 mile section and counting all the plants in that section, then going to another rainforest and marking a 1 x1 mile section and counting all the plants. You would probably find more plants in the bigger section simply because you are investigating one place more than the other.

So that is why we need a sampling depth.

Choosing the sampling depth is a balance between how many samples you are willing to not use and much you want to investigate each sample. For example if I have a really large sampling depth I will be able to investigate each sample more but I might not be able to use all my samples.

Hope that helps!

bettya · April 29, 2021, 8:41am

Thank you Chloe for your assistance.
I did get the explanation very well. I used 3 levels of sampling depth and I am considering using the smallest in order to retain all the samples. I believe I will get more clarity if am able to plot the rearefaction curve but meanwhile I was given this error when I tried the visualization command. Can you please help? The commands are below:

(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2$ qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv
Plugin error from diversity:

Metadata does not contain any columns that satisfy this visualizer's requirements. There must be at least one metadata column that contains categorical data, isn't empty, doesn't consist of unique values, and doesn't consist of exactly one value.

Debug info has been saved to /tmp/qiime2-q2cli-err-pd6h6h04.log
(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2$ qiime diversity alpha-group-significance --i-alpha-diversity core-metrics-results/faith_pd_vector.qza --m-metadata-file sample-metadata.tsv --o-visualization core-metrics-results/faith-pd-group-significance.qzv
Plugin error from diversity:

Metadata does not contain any columns that satisfy this visualizer's requirements. There must be at least one metadata column that contains categorical data, isn't empty, doesn't consist of unique values, and doesn't consist of exactly one value.

Debug info has been saved to /tmp/qiime2-q2cli-err-tq0ci02r.log.

iime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/evenness_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/evenness-group-significance.qzv
Plugin error from diversity:

Metadata does not contain any columns that satisfy this visualizer's requirements. There must be at least one metadata column that contains categorical data, isn't empty, doesn't consist of unique values, and doesn't consist of exactly one value.

Debug info has been saved to /tmp/qiime2-q2cli-err-zp3o5oen.log
Can you please help with these errors?

Regards.

bettya · April 29, 2021, 8:48am

The error log:

(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2$ qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv
--verbose
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in alpha_group_significance
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/action.py", line 452, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/q2_diversity/_alpha/_visualizer.py", line 52, in alpha_group_significance
"Metadata does not contain any columns that satisfy this "
ValueError: Metadata does not contain any columns that satisfy this visualizer's requirements. There must be at least one metadata column that contains categorical data, isn't empty, doesn't consist of unique values, and doesn't consist of exactly one value.

Does this means something is wrong with my metadata table?

Regards.

cherman2 · May 3, 2021, 7:32pm

Hello @bettya,
I am very sorry for my late response.
So the issue that I see is that the Metadata does not have a categorical data for it to group your samples into and compare with alpha group significance.

Alpha group significance compares categorical groups from your metadata and investigates if there is significant differences between the groups. For example, if I wanted to see if the microbiome was different between males and females then I would have a categorical column in my metadata that marked if the sequence was from a male or a female and then alpha group significance could see if there was a statistically significant difference between males and females.

My advice would be to check out you metadata and make sure it has categorical columns for you to investigate.
And if you want to validate you metadata and make sure it is compatible with QIIME2. I would recommend the Keemei
Hope that helps!
Chloe

bettya · May 17, 2021, 6:58pm

Hi Cherman2
Thanks for your reply. I have been battling with the metadata file.
First, I mistakingly opened it with the wrong app, I later changed it back to open with notepad.
Now, I want to run the associations between categorical metadata columns and alpha diversity data. I ran this command:
qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv

I had this error with verbose:
qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv
There was an issue with loading the file sample-metadata.tsv as metadata:

Metadata file path doesn't exist, or the path points to something other than a file. Please check that the path exists, has read permissions, and points to a regular file (not a directory): sample-metadata.tsv

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: https://keemei.qiime2.org

Find details on QIIME 2 metadata requirements here: Metadata in QIIME 2 — QIIME 2 2020.11.1 documentation

(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2-sample$ qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv
--verbose
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/click/type.py", line 169, in _convert_metadata
artifact = qiime2.Artifact.load(fp)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/sdk/result.py", line 66, in load
archiver = archive.Archiver.load(filepath)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/core/archive/archiver.py", line 299, in load
archive = cls.get_archive(filepath)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/core/archive/archiver.py", line 259, in get_archive
raise ValueError("%s does not exist." % filepath)
ValueError: sample-metadata.tsv does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/q2cli/click/type.py", line 172, in _convert_metadata
metadata = qiime2.Metadata.load(fp)
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/metadata/metadata.py", line 309, in load
return MetadataReader(filepath).read(into=cls,
File "/home/qiime2/miniconda/envs/qiime2-2020.11/lib/python3.6/site-packages/qiime2/metadata/io.py", line 47, in init
"(not a directory): %s" % filepath)
qiime2.metadata.io.MetadataFileError: Metadata file path doesn't exist, or the path points to something other than a file. Please check that the path exists, has read permissions, and points to a regular file (not a directory): sample-metadata.tsv

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: https://keemei.qiime2.org

Find details on QIIME 2 metadata requirements here: Metadata in QIIME 2 — QIIME 2 2020.11.1 documentation

There was an issue with loading the file sample-metadata.tsv as metadata:

Metadata file path doesn't exist, or the path points to something other than a file. Please check that the path exists, has read permissions, and points to a regular file (not a directory): sample-metadata.tsv

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: https://keemei.qiime2.org

Find details on QIIME 2 metadata requirements here: Metadata in QIIME 2 — QIIME 2 2020.11.1 documentation

Please, kindly note that I have validated the metadata with Keemei as suggested by you.
I also edited the metadata file before using it by opening it with notepad. Can you advised what is wrong with this command?
Can I skip this step as I need the Biom table for R Studio analysis?
I also need a good tutorial for ITS 1 and 2 analysis, can you please recommend one?

Thank you for your help.

Mehrbod_Estaki · May 18, 2021, 6:39am

Hi @bettya ,
In case this issue hasn't been resolved yet, this is the line you want to pay attention to:

Can you double check that the sample-metadata.tsv file a) exists, and b) is in the same folder that you are running that command from? If it has been moved, renamed, or you are running the code from a different directory then you'll want to make sure you provide the full path to the metadata file. Right now it just can't find it.

bettya · May 23, 2021, 10:30pm

Hi Estaki
Thank you for the advice. The problem persisted. I did all the suggested actions.
The command and the output:

(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2-sample$ qiime diversity alpha-group-significance \

--i-alpha-diversity core-metrics-results/faith_pd_vector.qza
--m-metadata-file sample1-metadata.tsv
--o-visualization core-metrics-results/faith-pd-group-significance.qzv
Usage: qiime diversity alpha-group-significance [OPTIONS]

Visually and statistically compare groups of alpha diversity values.

Inputs:
--i-alpha-diversity ARTIFACT SampleData[AlphaDiversity]
Vector of alpha diversity values by sample. [required]
Parameters:
--m-metadata-file METADATA...
(multiple The sample metadata.
arguments will
be merged) [required]
Outputs:
--o-visualization VISUALIZATION
[required]
Miscellaneous:
--output-dir PATH Output unspecified results to a directory
--verbose / --quiet Display verbose output to stdout and/or stderr during
execution of this action. Or silence output if
execution is successful (silence is golden).
--examples Show usage examples and exit.
--citations Show citations and exit.
--help Show this message and exit.

              There were some problems with the command:

(1/4) Missing option '--i-alpha-diversity'.
(2/4) Missing option '--m-metadata-file'.
(3/4) Missing option '--o-visualization'. ("--output-dir" may also be used)
(4/4) Got unexpected extra arguments (core-metrics-
results/faith_pd_vector.qza sample1-metadata.tsv core-metrics-results/faith-
pd-group-significance.qzv).

All these files are present in the folder:
(qiime2-2020.11) qiime2@qiime2core2020-11:~/Desktop/shared/qiime2-sample$ ls
--i-alpha-diversity demux-subsample.qza rep-seqs.qza
--i-demultiplexed-seqs demux-subsample.qzv rep-seqs.qzv
--m-metadata-file demux.qzv rooted-tree.qza
--o-visualization denoising-stats.qza sample-metadata.tsv
aligned-rep-seqs.qza denoising-stats.qzv sample1-metadata.tsv
core-metrics-results emp-paired-end-sequences table.qza
demux-details.qza emp-paired-end-sequences.qza table.qzv
demux-full.qza masked-aligned-rep-seqs.qza unrooted-tree.qza

I don't know why qiime2 is not reading the files.

Any suggestion or help?

cherman2 · May 24, 2021, 4:17pm

Hello @bettya,
So before the issue with your metadata was that you were telling qiime2 that the metadata file was named "sample-metadata.tsv" and it's named "sample1-metadata.tsv". My advice would be to make sure that you are 100% sure that the name is correct.

So the error that you are getting now is a new error. To be honest, your post is very busy and has a lot of information in it. I am not sure that I am getting all the information. The error seems to think that you aren't handing in the "-- i-alpha-diversity","--m-metadata-file" and "--o-visualization" commands? My advice would be to run the command again and be very careful to make sure you are typing in the command right.

If the problem persists please send a screenshot of the command you ran to help us visualize what you are running.
I hope that helps!
Chloe

Mehrbod_Estaki · May 25, 2021, 4:56am

I agree with @cherman2 completely, in my experience those errors you reported tend to come from just incorrect syntax, most often from when people copy & paste from a word document that does some weird formatting to the characters for example turning 2 dashes-- into an ndash – ,or mdash — which are going to cause errors like you see. I'd recommend keeping notes in plain text format like on a notepad. So if you were to copy & paste the below exactly as is, I think it should work.

qiime diversity alpha-group-significance \
  --i-alpha-diversity core-metrics-results/faith_pd_vector.qza \
  --m-metadata-file sample1-metadata.tsv \
  --o-visualization core-metrics-results/faith-pd-group-significance.qzv

thermokarst · June 6, 2021, 9:44pm

An off-topic reply has been split into a new topic: A mapping file is different from a metadata file, is that true?

Please keep replies on-topic in the future.

system · July 8, 2021, 3:45am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.