Import "Killed" Error

[carte494 (Drew Carter)]

Hi,
I'm importing both 16S and WGS paired end reads to be processed on a supercomputer. The WGS data is ~150GB and the 16S is ~18GB. The import will run for ~20 minutes and then say "Killed" with no further information. It says there's no such command as "--verbose" for some reason when I try adding that. Is this a memory problem? Here are the details for the supercomputer: About 17,700 total cores, on Intel Haswell processors. 24 cores and 62gb per node in the large primary queues. Here is my code for what it's worth:

#!/bin/bash -1
#PBS -1 walltime=24:00:00,nodes=4:ppn=8,pmem=10GB
#PBS -m abe
#PBS -M [username]

module load qiime2/2018.6

cd [Directory Location]

qiime tools import
--type SampleData[PairedEndSequencesWithQualty]
--input-path WGS_Manifest.txt
--output-path data.qza
--source-format PairedEndFastqManifestPhred33

Any ideas or would you need more info?

[ebolyen (Evan Bolyen)]

Hi @carte494!

Sorry for the very delayed response on my part.

@carte494

The WGS data is ~150GB and the 16S is ~18GB.

That's fantastic!

@carte494

About 17,700 total cores, on Intel Haswell processors. 24 cores and 62gb per node in the large primary queues. Here is my code for what it’s worth:

#!/bin/bash -1
#PBS -1 walltime=24:00:00,nodes=4:ppn=8,pmem=10GB
#PBS -m abe
#PBS -M [username]

module load qiime2/2018.6

cd [Directory Location]

qiime tools import
–type SampleData[PairedEndSequencesWithQualty]
–input-path WGS_Manifest.txt
–output-path data.qza
–source-format PairedEndFastqManifestPhred33

That all looks good, I suspect the problem is you are running out of disk space in your TMPDIR.

You can override the directory used by setting the TMPDIR environment variable:

export TMPDIR=/somewhere/with/lots/of/disk/

Hopefully that fixes the issue, but if not, please let me know!

---

Incidentally, what is your plan for analyzing the WGS data? I don't think we have any good analysis steps for that in QIIME 2 (yet!). Are there some plugins your are using to work with this?