I recently started working on my last project on 16s amplicon on Qiime 2 with a lot of data.
So, I filtered my samples via Geneious and I went ahead and did dada2 for quality filtering regardless. However, when I received the dada2 files. None of the computers would run it right and the weirdest thing is that even with --verbose, it would not spit out the problem but rather, the computer would freeze and shut off all the time. I tried to do run this in a supercomputer offered by the university with 40GB RAM but still would not work.
I am attaching the table.qza, rep-set.qza and denoising.qza files.
I don't see any problem with table files or rep-set files. So I am very confused.
Hi @Sammi,
I appreciate you sharing your prior Results, but it's not totally clear what you're asking here. As a general practice, please copy/paste exact commands that are failing.
What commands failed to run? How were you running them (e.g. as scheduled jobs on the cluster, or directly from the cluster CLI, without using a scheduler)? Is it possible they're just taking a long time to run? Some QIIME 2 commands are very compute-intensive, and require compute resources and a significant run time.
So I was running this command:
(qiime2-2020.8) Jiyoungs-MacBook-Pro: qiime alignment mafft --i-sequences representative_sequences2.qza --o-alignment aligned-rep-seqs.qza
I had all the files ready. I was running on my Mac Qiime with 8RAM, 250GB SSD
My computer just shut off automatically after running for almost 36 hrs.
I tried the same comment on Virtual box, the computer froze, and stopped running after a couple of hours so I had to forcefully shut my computer off.
Mafft has pretty hefty resource requirements, and 8GB RAM is almost certainly insufficient. A long runtime would be expected, and in 36 hours, something may have terminated your virtualbox or caused your computer to shut down. Sleep/hibernate can do funny things to long-running processes sometimes, IIRC.
When you run on the cluster you mentioned above, are you actively allocating resources? For example, when I log into our institutional cluster and run a command, it runs on 1 core with little memory by default. In order to access supercompute resources, I need to schedule a job and request resources for it.
I am pretty new about running on certain system before.. I tried to run it on the institutional cluster previously, but it would just not run it... I think it was something to do with the file. I tried to run on 4 node with RAM of 40 GB..
It sounds like you might be experiencing two separate issues. I suspect your local machine doesn't have sufficient resources to run qiime alignment mafft. Not much you can do about that, so probably best to do this part of your analysis on your institutional cluster, or on a server.
Clusters are all different, so you might want to talk to your cluster admin about the issues you're having in that environment. They will be better able to answer questions about their system. If you continue having trouble after seeking their support, please feel free to open a new topic here, with specific commands and error messages we can use to help you troubleshoot.
