Force overwrite of directory?

[metagenomes (Dan)]

I’m using QIIME2 with snakemake. One of the things snakemake does is create output directories when specified in the snakefile.

When running things like the Moving Pictures tutorial, this becomes problematic:

qiime diversity core-metrics-phylogenetic \
  --i-phylogeny rooted-tree.qza \
  --i-table table.qza \
  --p-sampling-depth 1109 \
  --m-metadata-file sample-metadata.tsv \
  --output-dir core-metrics-results

Nets an error:

Error: --output-dir directory already exists, won't overwrite.
Error in job coreDiversityMetrics while creating output file core-metric-results/weighted_unifrac_pcoa_results.qza.
RuleException:

Because the output directory is created by snakemake before the directory is created by QIIME2.

Is there a way in QIIME2 to force overwrite if a directory already exists? If not it’s an ugly hack: https://stackoverflow.com/questions/40729835/prevent-snakemake-from-making-output-directory

[thermokarst]

Hi @metagenomes! What if instead of using the --output-dir flag, you specified all of the outputs:

qiime diversity core-metrics-phylogenetic \
  --i-phylogeny rooted-tree.qza \
  --i-table table.qza \
  --p-sampling-depth 1109 \
  --m-metadata-file sample-metadata.tsv \
  --o-rarefied-table core-metrics-results/table.qza \
  --o-observed-otus-vector core-metrics-results/observed-otus-vector.qza \ 
  --o-shannon-vector core-metrics-results/shannon-vector.qza \
  --o-evenness-vector core-metrics-results/evenness-vector.qza \
  --o-jaccard-distance-matrix core-metrics-results/jaccard-distance-matrix.qza \
  --o-bray-curtis-distance-matrix core-metrics-results/bray-curtis-distance-matrix.qza \
  --o-jaccard-pcoa-results core-metrics-results/jaccard-pcoa-results.qza \
  --o-bray-curtis-pcoa-results core-metrics-results/bray-curtis-pcoa-results.qza \
  --o-jaccard-emperor core-metrics-results/jaccard-emperor.qzv \
  --o-bray-curtis-emperor core-metrics-results/bray-curtis-emperor.qzv

I know that is a huge pain in the rear, but you can overwrite outputs without an error when you use the --o flags, versus the --output-dir flag. Let me know if that gets you moving - thanks!

[colinbrislawn (Colin J Brislawn)]

Given that this is part of a pipeline, you will only have to list these outputs once and snakemake will validate all of these outputs for you.

In the context of snakemake, I think using explicit outputs is pretty elegant. It’s good that snakemake is controlling this, instead of qiime.

[metagenomes (Dan)]

Thanks guys, explicit outputs is fine - I wasn’t aware I had such fine grained control at my fingertips there, I don’t have an issue with this - it’s definitely more elegant than the linked solution!