About importing files of qc_mock_observed and qc_mock_expected

[Jia (Jia Zhou)]

QC/benchmark Qiime2 on Mock community

Hi @hotblast, Note that a bug was found in quality-control evaluate-composition that caused TAR and TDR scores to be reversed. See this announcement for more details if you are affected.

Hi, Nicholas

I have read a few posts about this topic and they are very helpful to my question. However, I still could not import the files properly.

I have written qc_mock_observed and qc_mock_expected and saved in .txt files.

#OTU_ID	expected
k__Bacteria; p__Firmicutes; c__Bacilli; o__Bacillales; f__Staphylococcaceae; g__Staphylococcus; s__aureus	15.5
k__Bacteria; p__Proteobacteria; c__Gammaproteobacteria; o__Enterobacteriales; f__Enterobacteriaceae; g__Salmonella; s__enterica	10.4
k__Bacteria; p__Firmicutes; c__Bacilli; o__Bacillales; f__Bacillaceae; g__Bacillus; s__subtilis	17.4
k__Bacteria; p__Firmicutes; c__Bacilli; o__Bacillales; f__Listeriaceae; g__Listeria; s__monocytogenes	14.1
k__Bacteria; p__Proteobacteria; c__Gammaproteobacteria; o__Enterobacteriales; f__Enterobacteriaceae; g__Escherichia; s__coli	10.1
k__Bacteria; p__Firmicutes; c__Bacilli; o__Lactobacillales; f__Lactobacillaceae; g__Lactobacillus; s__fermentum	18.4
k__Bacteria; p__Firmicutes; c__Bacilli; o__Lactobacillales; f__Enterococcaceae; g__Enterococcus; s__faecalis	9.9
k__Bacteria; p__Proteobacteria; c__Gammaproteobacteria; o__Pseudomonadales; f__Pseudomonadaceae; g__Pseudomonas; s__aeruginosa	4.2

#OTU_ID	observed
k__Archaea; p__Euryarchaeota; c__Methanomicrobia; o__Methanomicrobiales; f__Methanocorpusculaceae; g__Methanocorpusculum; s__	0
k__Bacteria; p__Acidobacteria; c__Acidobacteriia; o__Solibacterales; f__Solibacteraceae(Subgroup3); g__Bryobacter; s__	0
k__Bacteria; p__Acidobacteria; c__Acidobacteriia; o__Solibacterales; f__Solibacteraceae(Subgroup3); g__; s__	0

I am trying to convert the files into . qza, but got the following error messages.
qiime tools import --input-path qc-mock-expected.txt --output-path qc-mock-expected.qza --type DistanceMatrix
There was a problem importing qc-mock-expected.txt:

qc-mock-expected.txt is not a(n) LSMatFormat file

Could you please give me some suggestions on it?

Thanks,

Jia

[Jia (Jia Zhou)]

Sorry that the content in the .txt files might be a bit confusing. In the file of expected mock community, there are all eight species and the values are out of 100. In the file of observed mock community, I only listed three species, there are counts in OTU table.

[Jia (Jia Zhou)]

Hi, again

Just realised that the import file type was not right. I tried
qiime tools import
--input-path qc-mock-observed.txt
--output-path qc-mock-observed.qza
--type FeatureTable

But got another error message.
TypeError: 0 cannot be used as a field to FeatureTable[{content}] (not a type).
An unexpected error has occurred:
0 cannot be used as a field to FeatureTable[{content}] (not a type).

Thanks,

Jia

[Nicholas_Bokulich]

Hi @Jia,
You have the wrong type again. You want FeatureTable[RelativeFrequency], but ONLY after you have converted to biom-table format. See the final section of this tutorial for a very similar example:

Fungal ITS analysis tutorial

Fungal ITS analysis, mock communities, and more fun NOTE: This tutorial was written in a QIIME 2 2018.11 environment. It is not guaranteed to work with earlier or later versions of QIIME 2. This tutorial was compiled as a working exercise for a QIIME 2 workshop in December 2018, and does not represent the only possible fungal ITS workflow with QIIME 2, or even a benchmarked protocol recommendation. See other tutorials, e.g., the q2-itsxpress tutorial for other fungal ITS analysis options in QII…

[Jia (Jia Zhou)]

Hi, Nicholas

Thanks for the reply. It works and I can get the .biom files and .qza files. However, when I run the qiime quality-control evaluate-composition, I always got the following error message.

qiime quality-control evaluate-composition \

> --i-expected-features qc-mock-expected.qza \

> --i-observed-features qc-mock-observed.qza \

> --o-visualization qc-mock-comparison.qzv

Plugin error from quality-control:

"None of [Index(['observed'], dtype='object')] are in the [index]"

Debug info has been saved to /var/folders/v9/gy5c6y6j3b9cz3f2ccj5lj0m0000gn/T/qiime2-q2cli-err-6z0scyt1.log

I tried to do the below two commands, I can get results. I guess the names of taxa have to be exactly the same in two files (qc-mock-expected.qza and qc-mock-observed.qza).
qiime quality-control evaluate-composition
--i-expected-features qc-mock-expected.qza
--i-observed-features qc-mock-expected.qza
--o-visualization qc-mock-comparison.qzv

qiime quality-control evaluate-composition
--i-expected-features qc-mock-observed.qza
--i-observed-features qc-mock-observed.qza
--o-visualization qc-mock-comparison.qzv

I wanted to get a results mentioned in this tutorial, docs/source/tutorials/quality-control.rst at master · qiime2/docs · GitHub and I want to get the results as shown in attached file qc-mock-3-comparison.qzv (339.7 KB)

It seems that I can get the differences of taxa in 7 taxonomic levels. However, I have very limited import information for qc-mock-expected.qza. So, I am wondering whether it will work for this case.
I have modified the expected mock community and the observed mock community files to match the OTU_ID, here attached the two files
qc-mock-expected.qza (6.8 KB)
qc-mock-observed.qza (6.8 KB).

Thanks,

Jia

[Nicholas_Bokulich]

@Jia

I guess the names of taxa have to be exactly the same in two files

Yes, you need to have the same sample IDs, or use use a metadata file and column to indicate how the same names map between observed and expected files.