Hi,
I am new to Qiime and have only spent a week or so pulling my hair.
I am trying to generate some beta diversity plots relating some 16S gut samples.
I executing the following commands:
qiime diversity core-metrics-phylogenetic
--i-phylogeny rooted-tree_exp1_2_3.qza
--i-table Table_Freq_EXP11_12_13.qza
--p-sampling-depth 1000
--m-metadata-file Metadata_exp11_12_13_Valid.txt
--output-dir core_metrics_results
I get the following error:
/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/sklearn/metrics/pairwise.py:1735: DataConversionWarning: Data was converted to boolean for metric jaccard
warnings.warn(msg, DataConversionWarning)
/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:152: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.05259711795033266 and the largest is 7.863737098882202.
RuntimeWarning
/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:152: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.24217353423261714 and the largest is 8.119482582194188.
RuntimeWarning
Traceback (most recent call last):
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_diversity/_alpha/_method.py", line 54, in alpha_phylogenetic
tree=phylogeny)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/skbio/diversity/_driver.py", line 170, in alpha_diversity
counts, otu_ids, tree, validate, single_sample=False)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/skbio/diversity/alpha/_faith_pd.py", line 136, in _setup_faith_pd
_validate_otu_ids_and_tree(counts[0], otu_ids, tree)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/skbio/diversity/_util.py", line 104, in _validate_otu_ids_and_tree
" ".join(missing_tip_names)))
skbio.tree._exception.MissingNodeError: All otu_ids
must be present as tip names in tree
. otu_ids
not corresponding to tip names (n=557): 1160 1240 717 72 514 876 825 747 621 822 858 453 851 208 207 383 673 922 84 1237 830 674 1209 366 719 10 73 893 33 1115 601 872 1095 96 182 646 395 1143 1135 1107 1079 1096 452 1219 356 908 549 14 175 1057 52 407 1011 1148 1027 17 577 527 268 1264 956 424 467 1047 415 1072 190 1162 511 1068 1249 1051 382 768 874 1086 1097 217 1164 884 507 103 380 1030 1167 495 174 128 429 1025 252 980 362 712 1262 29 30 655 647 910 1247 186 303 421 767 404 663 282 665 97 623 641 274 107 0 210 74 115 191 644 12 1112 9 28 78 600 40 161 784 901 1033 1141 1099 257 561 1127 585 676 1136 304 92 285 231 944 1191 1175 277 126 917 1120 473 170 502 1253 921 83 1165 70 414 111 713 443 619 1083 995 589 1172 920 493 794 399 155 446 875 288 795 863 195 94 11 62 1125 295 656 812 1198 567 700 1223 134 868 1074 1186 730 731 168 750 279 1016 894 1013 124 476 571 269 89 938 1038 1218 159 45 81 202 1192 223 197 243 1222 651 1111 86 393 392 16 528 587 791 693 862 701 69 697 1202 435 1084 127 188 34 675 997 1133 888 233 1022 401 218 834 213 1138 7 823 235 566 163 575 1019 857 136 440 698 1094 192 664 289 370 953 586 579 643 594 457 524 1085 501 171 1034 125 1056 850 559 529 1109 1217 273 329 433 439 363 318 491 229 374 466 503 965 723 671 912 272 151 932 71 271 183 300 432 899 283 526 66 627 255 1062 605 381 821 149 1031 1169 88 1073 203 164 344 630 961 1174 322 123 313 106 118 172 861 1113 537 121 556 287 576 114 598 616 833 354 755 327 1005 582 215 840 442 918 1076 31 1061 889 1001 530 764 219 438 458 702 517 800 808 1014 778 242 290 680 548 553 679 845 1254 1225 18 509 926 1026 686 999 1193 241 1046 1078 201 705 624 1242 954 892 883 142 572 225 21 205 198 578 909 1075 1029 112 809 291 745 541 46 1236 24 1063 143 311 709 22 607 756 640 369 687 90 211 504 945 376 981 787 947 618 915 426 1155 116 64 15 65 196 765 963 762 375 216 555 240 138 342 769 42 305 706 782 397 1060 412 1118 1035 1119 456 1088 51 849 135 610 817 714 76 385 1117 162 236 614 102 1070 131 262 199 461 890 405 652 583 1166 87 253 666 455 933 653 265 710 148 19 35 990 720 987 725 1250 337 328 275 542 193 120 340 2 154 436 1080 481 1055 986 881 551 428 425 1228 1146 781 82 454 220 683 716 437 1132 609 101 852 886 1221 510 263 336 667 1173 1201 505 1087 93 626 992 450 293 1 1102 1255 608 775 525 129 44 1054 855
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in call
results = action(**arguments)
File "</Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-386>", line 2, in core_metrics_phylogenetic
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 484, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_diversity/_core_metrics.py", line 57, in core_metrics_phylogenetic
metric='faith_pd')
File "</Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-477>", line 2, in alpha_phylogenetic
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
output_types, provenance)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in callable_executor
output_views = self._callable(**view_args)
File "/Users/xxx/anaconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_diversity/_alpha/_method.py", line 58, in alpha_phylogenetic
raise skbio.tree.MissingNodeError(message)
skbio.tree._exception.MissingNodeError: All feature_ids
must be present as tip names in phylogeny
. feature_ids
not corresponding to tip names (n=557): 1160 1240 717 72 514 876 825 747 621 822 858 453 851 208 207 383 673 922 84 1237 830 674 1209 366 719 10 73 893 33 1115 601 872 1095 96 182 646 395 1143 1135 1107 1079 1096 452 1219 356 908 549 14 175 1057 52 407 1011 1148 1027 17 577 527 268 1264 956 424 467 1047 415 1072 190 1162 511 1068 1249 1051 382 768 874 1086 1097 217 1164 884 507 103 380 1030 1167 495 174 128 429 1025 252 980 362 712 1262 29 30 655 647 910 1247 186 303 421 767 404 663 282 665 97 623 641 274 107 0 210 74 115 191 644 12 1112 9 28 78 600 40 161 784 901 1033 1141 1099 257 561 1127 585 676 1136 304 92 285 231 944 1191 1175 277 126 917 1120 473 170 502 1253 921 83 1165 70 414 111 713 443 619 1083 995 589 1172 920 493 794 399 155 446 875 288 795 863 195 94 11 62 1125 295 656 812 1198 567 700 1223 134 868 1074 1186 730 731 168 750 279 1016 894 1013 124 476 571 269 89 938 1038 1218 159 45 81 202 1192 223 197 243 1222 651 1111 86 393 392 16 528 587 791 693 862 701 69 697 1202 435 1084 127 188 34 675 997 1133 888 233 1022 401 218 834 213 1138 7 823 235 566 163 575 1019 857 136 440 698 1094 192 664 289 370 953 586 579 643 594 457 524 1085 501 171 1034 125 1056 850 559 529 1109 1217 273 329 433 439 363 318 491 229 374 466 503 965 723 671 912 272 151 932 71 271 183 300 432 899 283 526 66 627 255 1062 605 381 821 149 1031 1169 88 1073 203 164 344 630 961 1174 322 123 313 106 118 172 861 1113 537 121 556 287 576 114 598 616 833 354 755 327 1005 582 215 840 442 918 1076 31 1061 889 1001 530 764 219 438 458 702 517 800 808 1014 778 242 290 680 548 553 679 845 1254 1225 18 509 926 1026 686 999 1193 241 1046 1078 201 705 624 1242 954 892 883 142 572 225 21 205 198 578 909 1075 1029 112 809 291 745 541 46 1236 24 1063 143 311 709 22 607 756 640 369 687 90 211 504 945 376 981 787 947 618 915 426 1155 116 64 15 65 196 765 963 762 375 216 555 240 138 342 769 42 305 706 782 397 1060 412 1118 1035 1119 456 1088 51 849 135 610 817 714 76 385 1117 162 236 614 102 1070 131 262 199 461 890 405 652 583 1166 87 253 666 455 933 653 265 710 148 19 35 990 720 987 725 1250 337 328 275 542 193 120 340 2 154 436 1080 481 1055 986 881 551 428 425 1228 1146 781 82 454 220 683 716 437 1132 609 101 852 886 1221 510 263 336 667 1173 1201 505 1087 93 626 992 450 293 1 1102 1255 608 775 525 129 44 1054 855
I would note that I have imported a number of partially processed files to get to this point. Qiime seemed satisfied with my imported files:
- a .biom table with my OTU frequencies per sample formatted to match the metadata. (BIOMV210Format)
- A fasta file of unaligned sequences converted to an .fna file formatted to match the biom table data, especially the #OTU_ID column consists of otus labelled by number only from 0 to 1265
values are delineated by tabs in all the precursor files.
The only thing off here I can think of is that the fasta file includes the representative sequence which is not included in the biom table. I used the format of thesequences.fna / sequences.qza
example. - a metadata file was formatted against the qiime sample-metadata.tsv format, except a txt.
I have painstakingly removed all the "_" which I thought was the problem before.
These commands were executed before the error commands successfully:
qiime tools import
--input-path collapsed_ref_OTUs_97_pct_rep_seqs_Silva_v108_untrimmed2.fna
--output-path RepSeq_Exp1_2_3.qza
--type 'FeatureData[Sequence]'
biom convert -i Table_Match_Metadata_countsMatrix_collapsed_ref_OTUs_97_pct_rep_seqs_Silva_v108_untrimmed22.txt -o EXP11_12_13.biom --to-hdf5 --table-type="OTU table" --process-obs-metadata taxonomy
qiime tools import
--input-path EXP11_12_13.biom
--type 'FeatureTable[Frequency]'
--input-format BIOMV210Format
--output-path Table_Freq_EXP11_12_13.qza
qiime feature-table summarize
--i-table Table_Freq_EXP11_12_13.qza
--o-visualization FreqTable_exp1_2_3.qzv
--m-sample-metadata-file Metadata_exp11_12_13_Valid.txt
qiime feature-table tabulate-seqs
--i-data RepSeq_Exp1_2_3.qza
--o-visualization seq_exp1_2_3.qzv
qiime phylogeny align-to-tree-mafft-fasttree
--i-sequences RepSeq_Exp1_2_3.qza
--o-alignment aligned-rep-seqs_exp1_2_3.qza
--o-masked-alignment masked-aligned-rep-seqs_exp1_2_3.qza
--o-tree unrooted-tree_exp1_2_3.qza
--o-rooted-tree rooted-tree_exp1_2_3.qza
runnning qiime2-2020.2
Huge thanks to anyone who can tell me where my wrong turn is. At this point I'm thinking I need to start with the sequencer raw outputs.
Thanks!