process killed, memory usage of qiime longitudinal nmit

I am trying to run the following command and ubuntu "Killed" the process because it was taking >600GB of RAM. My machine only has 500G.

$ qiime longitudinal nmit --i-table 240202_260x205_featureTable_freq.qza --m-metadata-file 240206-metadata-time_combine_patient_location.tsv --p-individual-id-column patient --p-corr-method pearson --o-distance-matrix nmit-dm.qza

It isn't a particularly large dataset ~97M reads across 185 samples. Is there a way that I can cap the memory usage? It doesn't appear to in the docs. There is also a possibility that my -i--table artifact is malformed. It looks a bit weird when I create a visualization of it.

Any help or ideas?


Hi @waalkes,

Welcome to the :qiime2: forum!

That does sound funky!
Could you send me the feature table(.qza and qzv) that you think is malformed and I can take a look?

Also, what commands did you run on your feature table prior to this?

What is the best way to get these to you? Here is how they were created:

qiime tools import --type 'SampleData[PairedEndSequencesWithQuality]' --input-path 240201_read_manifest.tsv --output-path 240201_imported_reads.qza --input-format PairedEndFastqManifestPhred33V2

qiime cutadapt trim-paired --i-demultiplexed-sequences 240201_imported_reads.qza --p-cores 40 --p-front-f GGAGGCAGCAGTRRGGAAT --p-front-r CTACCRGGGTATCTAATCC --verbose --p-discard-untrimmed --o-trimmed-sequences 240201_imported_reads_trimmed.qza

qiime demux summarize --i-data 240201_imported_reads_trimmed.qza --o-visualization 240201_imported_reads_trimmed.qzv
qiime dada2 denoise-paired --i-demultiplexed-seqs ../final/240201_imported_reads_trimmed.qza --p-n-threads 20 --p-trunc-len-f 260 --p-trunc-len-r 205 --verbose --o-table 240202_260x205_featureTable.qza --o-representative-sequences 240202_260x205_representative-sequences.qza --o-denoising-stats 240202_260x205_denoising-stats.qza

qiime feature-table relative-frequency --i-table 240202_260x205_featureTable.qza --o-relative-frequency-table 240202_260x205_featureTable_freq.qza

Hi @waalkes

Those steps seem reasonable!

Whatever works best for you! You could share them here publically if you are comfortable or you can DM me them!

Here they are. Thanks
240202_260x205_featureTable_freq.qzv (1.6 MB)
240202_260x205_featureTable_freq.qza (2.3 MB)

Hi @waalkes,
After talking with other mods it looks like you need to filter more aggressively. Right now, you are correlating over 20K features.The q2-longitudial tutorial talks alittle about their filtering steps if you read the text above the NMIT command.

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.