No model_stats.qza file

ceraker · January 25, 2024, 8:24pm

Hi all,

I've been working on analyzing some data using qiime2 and mmvec, and I have what may be a silly question.

When I fun mmvec on its own, not as a plugin, it generates the ranks, ordination, and embedding files. Code below:

mmvec paired-omics \
        --microbe-file microbiome_done.biom \
        --metabolite-file metabolites_done.biom \
        --summary-dir summary

But when I run the basic analysis as a QIIME2 plugin, it only generates the conditional_biplot.qza and conditionals.qza files, NOT the model_stats.qza file. Code below:

qiime mmvec paired-omics \
        --i-microbes microbiome.qza \
        --i-metabolites metabolites.qza \
        --p-summary-interval 1 \
        --output-dir model_summary

I've been searching through the forum and the package documentation, and cannot find anyone with a similar issue. Is this a problem?

I am using QIIME2/2023.5, and mmvec/1.0.5, and I have attached my mmvec output files below, as well as the qiime artifacts I used to run the analysis through the plugin.

Thank you so much!

latent_dim_3_input_prior_1.00_output_prior_1.00_beta1_0.90_beta2_0.95_ranks.txt (2.2 MB)
latent_dim_3_input_prior_1.00_output_prior_1.00_beta1_0.90_beta2_0.95_ordination.txt (62.6 KB)
latent_dim_3_input_prior_1.00_output_prior_1.00_beta1_0.90_beta2_0.95_embedding.txt (184.8 KB)

metabolites.qza (340.1 KB)
microbiome.qza (339.5 KB)

gregcaporaso · January 26, 2024, 4:44pm

Hi @ceraker, Welcome to the QIIME 2 Forum! I ping'ed the developer of the mmvec plugin to help out with this one, so we should have some input for you soon.

mortonjt · January 26, 2024, 5:35pm

No, I'm not sure why it is the case -- I wonder if you didn't train your model long enough (i.e. it completed within 1 second). It logs every second, so if you train too fast, that could lead to no logged outputs.

ceraker · January 26, 2024, 9:54pm

Thank you so much for responding!

And my model definitely took longer than 1 second, but shorter than five minutes, so it is possible that I did not train it for long enough. I will change some parameters and get back to you.

ceraker · January 26, 2024, 11:14pm

Okay I ran the model again, this time with these parameters:

qiime mmvec paired-omics \
          --i-microbes microbiome2.qza \
          --i-metabolites metabolites2.qza \
          --m-metadata-file meta_samp_met.txt \
          --p-learning-rate 1e-1 \
          --p-latent-dim 1 \
          --p-epochs 1000 \
          --p-summary-interval 1 \
          --output-dir model_summary5

It took about an hour to run, which was definitely much longer than before. However, it still did not generate a model_stats file!

Out of curiosity, I tried to generate an emperor biplot using those output files (attached below), and received this error:

Plugin error from emperor: Ordinations with less than two dimensions are not supported.

I have ~225 samples, so I don't think that's the issue.

conditional_biplot5.qza (33.4 KB)
conditionals5.qza (1.0 MB)

cherman2 · January 30, 2024, 3:47pm

Hi @ceraker,

I have reach out to the developer to see if he has any further ideas.

In the mean time have you tried to use the tutorial data on the mmvec github? GitHub - biocore/mmvec: Neural networks for microbe-metabolite interaction analysis

I would be interested to see if you can get other data to sucessfully run or if its an install issue.

Thanks!

mortonjt · January 30, 2024, 4:08pm

RE ordination - you can't visualization 1D PCoAs. You need at least 2 dimensions for that.

Regarding why no model_stats is outputted, I'm really not sure -- this is the first time this error has occurred. The only thing that could explain this is that something with the time readouts is broken (which would be bizarre).

Have you tried the non-qiime2 version? It would be another way to sanity check it.

Feel free to provide the raw readouts. I won't have time to look at this in the near future, but will try to look when time frees up.

ceraker · February 2, 2024, 1:31am

Okay, on ordination: I thought I HAD more than two dimensions! If I don't, then I'm not sure what's going on with my data.

As for install issues, we (my university's computing office and myself) had issues installing the qiime2 plugin version, and were finally able to make it work in a container. So that could be part of it? I have never worked with containers before, so I'm not very familiar with troubleshooting.

And I have run the non-qiime2 version, which outputted the ranks, ordination, and embedding files I attached in my initial post.

Thank you so much for looking into this, I really appreciate it!

ceraker · February 6, 2024, 7:37pm

I finally got a chance to run the soil tutorial data, and it did not generate a model_stats.qza file either! So it appears to be an install issue.

I will chat more with our computing office, and in the meantime focus on the non-qiime2 version of mmvec.

Thank you for all your help!

ceraker · February 15, 2024, 8:51pm

Sorry, back again. I am attempting to essentially re-do this analysis from the beginning, and something seems to have gone wrong with my input files. My code to convert things to the .biom format and incorporate metadata runs fine, but then when I try to run a basic mmvec paired-omics analysis I get the error: ValueError: 'a' cannot be empty unless no samples are taken. I'm really not sure why this is happening!

For reference, here is the code I am using to create the metabolome .biom file:
module load mmvec/1.0.5

# convert metabolite aboundance to .biom
biom convert -i metabolites.txt -o metabolites_json.biom --table-type="OTU table" --to-json

# incorporate sample metadata (same for both)
biom add-metadata -i metabolites_json.biom -o metabolites_meta2.biom -m meta_samp_met.txt

# incorporate observational metadata
biom add-metadata -i metabolites_meta2.biom -o metabolites_done2.biom -m met_obs_meta.txt

And the microbiome file:

#convert output from qiime
qiime tools export --input-path table-filtered_blue.qza --output-path exported-feature-table

biom convert -i exported-feature-table/feature-table.biom -o microbes.tsv --to-tsv

# convert microbiome aboundance to .biom
biom convert -i microbes.tsv -o microbiome_json.biom --table-type="OTU table" --to-json

# incorporate sample metadata (same for both)
biom add-metadata -i microbiome_json.biom -o microbiome_meta2.biom --sample-metadata-fp meta_samp_met.txt

# incorporate observational metadata
biom add-metadata -i microbiome_meta2.biom -o microbiome_done2.biom --observation-metadata-fp mic_obs_meta.txt --sc-separated taxonomy

Apologies if I'm doing something very silly with the file conversions: I've tried a couple different methods (particularly with the microbiome data, which is from qiime), but continue getting the same error. Input files are attached.
table-filtered_blue.qzv (3.4 MB)
meta_samp_met.txt (8.7 KB)
metabolites.txt (841.0 KB)
met_obs_meta.txt (53.5 KB)

system · March 18, 2024, 2:52am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

ceraker · April 11, 2024, 7:50pm

Hi all,

Back again now that everything is installed properly (hooray!) and I can finally get back to actually analyzing my data.

I have metabolomic data from an outside source, and microbiome data from qiime2 v 2023.5, and am now trying to analyze it using the qiime2 2020.6 mmvec plugin. I am running into issues with I believe my microbiome data.

I have tried multiple ways to convert it and upload it, but when I run the analysis I get the error:

Plugin error from mmvec:

  'a' cannot be empty unless no samples are taken

I get the same error when I run mmvec on its own (not as a plugin). I've searched various forums, but continue to be baffled. I definitely have samples, and looking at the example data from the Knight lab, mine seems to be formatted the same way.

I've attached a link to the files I've been using, and the code to convert things to biom below:

Code to convert files:

# convert metabolite aboundance to .biom
biom convert -i metabolites_b.txt -o metabolites_hdf5.biom --table-type="OTU table" --to-hdf5

# incorporate sample metadata (same for both)
biom add-metadata -i metabolites_hdf5.biom -o metabolites_meta.biom -m meta_samp_met.txt

# incorporate observational metadata
biom add-metadata -i metabolites_meta.biom -o metabolites_done.biom -m met_obs_meta.txt

# convert microbiome aboundance to .biom
biom convert -i microbes_b.txt -o microbiome_hdf5.biom --table-type="OTU table" --to-hdf5

# incorporate sample metadata (same for both)
biom add-metadata -i microbiome_hdf5.biom -o microbiome_meta.biom --sample-metadata-fp meta_samp_met.txt

# incorporate observational metadata
biom add-metadata -i microbiome_meta.biom -o microbiome_done.biom --observation-metadata-fp mic_obs_meta.txt --sc-separated taxonomy

Code to import files to qiime:

qiime tools import \
        --input-path metabolites_done.biom \
        --output-path metabolites.qza \
        --type FeatureTable[Frequency]

qiime tools import \
        --input-path microbiome_done.biom \
        --output-path microbes.qza \
        --type FeatureTable[Frequency]

Running basic model:

qiime mmvec paired-omics \
        --i-microbes microbes.qza \
        --i-metabolites metabolites.qza \
        --p-summary-interval 1 \
        --output-dir model_summary1

Link to files: Box

Thank you so much for any advice!