"Invalid value for 'input-path' Path does not exist" error on cluster

Hi,

I'm using the qiime2 module on the cluster system my university uses (ComputeCan) and am running into an error when I try to import my data. The error is "Invalid value for 'input-path' Path 'MetaData/manifest4.tsv' does not exist, but I'm using tab to finish the path when typing out the code, so I know it exists. I've also tried the longer path of /home/mkc206/scratch/MetaData/manifest4.tsv, but this doesn't work either. I'm concerned this has something to do with my environment set up.

Full code:

qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path MetaData/manifest4.tsv
--output-path 1_reads.qza
--input-format PairedEndFastqManifestPhred33V2

1 Like

Hi @mkc206,
That is weird!
Can you send in a screenshot of what you get when you ls in home/mkc206/scratch/MetaData/ ?

Thanks for responding!

So that's the "full" path, but because I'm submitting my code within scratch (which includes directory MetaData) I've been using --input-path MetaData/manifest4.tsv.

image

Hi @mkc206,
Still weird, lets keep digging :construction_worker_woman:
I believe that the reason ls home/mkc206/scratch/MetaData/ didn't work is that it should be
ls /home/mkc206/scratch/MetaData/

What happens when you nano the .tsv file? I just want to make sure its not a permissions thing?

Are you currently in the QIIME 2 env?

Sorry forgot to add, yes I am in the qiime environment.

By nano do you mean open the file? I can use 'vim manifest4.tsv' to open and edit. I also ran the suggested path and this is the output:

image

Hi @mkc206

Can you vim the file while in your qiime2 env? Nano is just another text editor!

yes, here is a screenshot of the first few rows:

@mkc206
This is weird! I have never heard of a QIIME 2 env doing this. I think the next test is using test data and see if we can repeat the error?

Can you follow the steps for importing in the parkinsons mouse tutorial? Parkinson’s Mouse Tutorial — QIIME 2 2019.7.0 documentation
Let me know if you are able to successfully find the pd-mouse data!

1 Like

Ok, I tried the first step of the Mouse tutorial by running this in directory mkc206 after loading the qiime2/2023.5 module:

apptainer exec qiime2.sif #the necessary code to make the module work on our server
qiime metadata tabulate \ #the code provided in the tutorial
--m-input-file metadata.tsv \ #this file is in /home/mkc206/projects/def-sanrehan/mkc206
--o-visualization metadata.qzv

and got the error:

Traceback (most recent call last):
File "/opt/conda/envs/qiime2-2023.7/bin/qiime", line 11, in
sys.exit(qiime())
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/click/core.py", line 1157, in call
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/click/core.py", line 1078, in main
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/click/core.py", line 1682, in invoke
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/click/core.py", line 1729, in resolve_command
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 118, in get_command
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/commands.py", line 141, in init
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/util.py", line 231, in example_data_option
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/q2cli/click/type.py", line 11, in
from qiime2.core.type.util import is_collection_type
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/init.py", line 9, in
from qiime2.sdk import Artifact, Visualization, ResultCollection
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/init.py", line 9, in
from .context import Context
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/context.py", line 13, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/qiime2/sdk/parallel_config.py", line 15, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/parsl/init.py", line 22, in
from parsl.app.app import bash_app, join_app, python_app
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/parsl/app/app.py", line 12, in
from parsl.dataflow.dflow import DataFlowKernel
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/parsl/dataflow/dflow.py", line 25, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/parsl/channels/init.py", line 2, in
from parsl.channels.ssh.ssh import SSHChannel
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/parsl/channels/ssh/ssh.py", line 5, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/paramiko/init.py", line 22, in
from paramiko.transport import (
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/paramiko/transport.py", line 93, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/paramiko/dsskey.py", line 37, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/paramiko/pkey.py", line 32, in
File "/opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/bcrypt/init.py", line 34, in
from . import _bcrypt # noqa: I100
ImportError: /opt/conda/envs/qiime2-2023.7/lib/python3.8/site-packages/_cffi_backend.cpython-38-x86_64-linux-gnu.so: failed to map segment from shared object

Okay I may have figured it out - I thought I removed conda because I just learned that we are not supposed to use that on the server because it messes up the environment, but I see here in the error that it's still in my path! Do you know how I might get rid of it? I found instructions online saying to use ' export $PATH=/path/with/conda' but that didn't get rid of it. I also deleted the miniconda directory.

1 Like

Hi Makaylee,

I'm jumping in here because I also use qiime2 on HPC using modules (instead of the typical conda envs)

That would add conda to the path...

That a good start! My HPC team provides a qiime2 module and also a module for anaconda itself, so there may be multiple conda copies on the server.

This is the perfect time to talk to your HPC support team about how to do this.
They would know exactly how to avoid conflicts with conda and call the right version of Qiime2.
(Also, HPC want you to use their big computers so they tend to be nice :hugs: )

4 Likes

Thank you! I did reach out to them and the problem wasn't with conda or qiime, but with apptainer. The -B bind paths need to be as follows below. I still don't quite understand why the file path needs to end in temp since the files are within the scratch directory, but this did solve the problem!

apptainer run -B /scratch/mkc206:/temp qimme2.sif qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path /temp/manifest3.tsv
--output-path /temp/1_reads.qza
--input-format PairedEndFastqManifestPhred33V2

Also thank you again to @cherman2!

4 Likes

5 off-topic replies have been split into a new topic: "Invalid value for 'input-path'

Please keep replies on-topic in the future.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.