Feature classifier : issue loading file

[SophieD]

Hello,

I'm running qiime feature-classifier classify-sklearn with qiime2 version 2022.11 installed with conda.

I ran :

#!/usr/bin/env bash
#PBS -q sequentiel
#PBS -l ncpus=1
#PBS -l mem=20gb
#PBS -l walltime=01:00:00

. /appli/bioinfo/qiime/2022.11/env.sh

database=/home/ref-bioinfo/ifremer/sebimer/tool-specific-banks/qiime2/2022.11/SILVA/v138.1/target_region/16S_V4_515f-806r_Parada-Apprill/SILVA_v138.1_ref_V4_515f-806r_Parada-Apprill.qza
sequences=/home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/04_dbotu_seqs.qza
confidence=0.9 #default
taxoqza=/home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/05_taxonomy.qza
taxoqzv=/home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/05_taxonomy.qzv
taxo_output=/home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/05_taxonomy_output

qiime feature-classifier classify-sklearn \
 --i-reads ${sequences} \
 --i-classifier ${database} \
 --p-confidence ${confidence} \
 --o-classification ${taxoqza}

qiime metadata tabulate \
 --m-input-file ${sequences} \
 --m-input-file ${taxoqza} \
 --o-visualization ${taxoqzv}

qiime tools export \
 --input-path ${taxoqzv} \
 --output-path ${taxo_output}

And I get the following error:

/var/spool/PBS/mom_priv/jobs/5503593.datarmor0.SC: line 25: 40721 Killed                  
qiime feature-classifier classify-sklearn --i-reads ${sequences} --i-classifier ${database} --p-confidence ${confidence} --o-classification ${taxoqza}
There was an issue with loading the file /home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/05_taxonomy.qza as metadata:

  Metadata file path doesn't exist, or the path points to something other than a file. Please check that the path exists, has read permissions, and points to a regular file (not a directory): /home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/05_taxonomy.qza

I don't understand why the error mentions a metadata format since this file is supposed to be the output.
And last week my code worked just fine with another dataset...

Thanks for any help you could provide me.
Have a good day,
Sophie

[colinvwood (Colin Wood)]

Hello @SophieD,

Can you run ls -ld /home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/ and share the output? It might be a permission issue?

[SophieD]

@colinvwood

ls -ld /home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/

Hello Colin,

here's the result :

drwxr-xr-x 6 sdarinot cip 4096 Jun  6 18:52 /home1/datawork/sdarinot/08_P3M_Maeva/qiime_results_selected/

[colinvwood (Colin Wood)]

Hello @SophieD,

I think I see what's going on. The error is not from the first command, but from the second command where it's trying to read in as metadata a file (05_taxonomy.qza) that hasn't been created yet. It hasn't been created yet because the jobs aren't executing sequentially because there is a typo in #PBS -q sequentiel -- should be "sequential". My guess is that the default is parallel execution.

[SophieD]

Thank you for your suggestion Colin, I tried changing it to ''sequential'' but it doesn't recognize the queue anymore:

qsub: Unknown queue

My workplace is french that's why it works using ''sequentiel'' haha

[SophieD]

Hello again Colin,

You were right about the meaning of the error message!
The second command is indeed starting without the first one being completed. Not because of a parallel execution problem but because the first command was killed. It was even stated in the error message.

It was a simple problem of memory allocation! I only asked for 20gb which was enough for my previous dataset but not for this one. Now I asked for 40gb, it seems to work.

Thank you for all your help Colin,
Have a good day,
Sophie