Qiime2 installation and run error

User Support
1

I have successfully installed qiime2-2018.6 on the linux server and also qiime2-2018.2 on my laptop using docker.
I was trying to run some commands which failed on server but were running on my laptop,Please suggest!
I ran this command on server which ends up in error, whereas the same command works properly on my laptop.

(qiime2-2018.2) root§3209d9fe45ee:/data£ qiime tools import --type EMPPairedEndSequences --input-path emp-paired-end-sequences --output-path emp-paired-end-sequences.qza
No error.

Error on linux server is below:

[pramod@herman-login2 Qiime]$ qiime tools import --type EMPPairedEndSequences --input-path emp-paired-end-sequences --output-path emp-paired-end-sequences.qza
-bash: qiime: command not found
[pramod@herman-login2 Qiime]$ qiime
-bash: qiime: command not found
[pramod@herman-login2 Qiime]$ pwd
/mnt/BioAdHoc/overflow/Groups/KLey/Qiime
[pramod@herman-login2 Qiime]$ source activate qiime2-2018.6
(qiime2-2018.6) [pramod@herman-login2 Qiime]$ qiime tools import --type EMPPairedEndSequences --input-path emp-paired-end-sequences --output-path emp-paired-end-sequences.qza
Traceback (most recent call last):
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2cli/tools.py", line 116, in import_data
    view_type=source_format)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py", line 219, in import_data
    return cls._from_view(type_, view, view_type, provenance_capture)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py", line 250, in _from_view
    provenance_capture=provenance_capture)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/archiver.py", line 310, in from_data
    Format.write(rec, type, format, data_initializer, provenance_capture)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/format/v1.py", line 26, in write
    prov_dir, [root / cls.METADATA_FILE, archive_record.version_fp])
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/provenance.py", line 315, in finalize
    self.path.rename(final_path)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/pathlib.py", line 1277, in rename
    self._accessor.rename(self, target)
  File "/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/pathlib.py", line 377, in wrapped
    return strfunc(str(pathobjA), str(pathobjB), *args)
FileExistsError: [Errno 17] File exists: '/scratch/qiime2-provenance-4o84xp5y' -> '/scratch/qiime2-archive-dgz17z59/f6712e98-38c9-40c0-9b62-1f56b1524052/provenance'

An unexpected error has occurred:

  [Errno 17] File exists: '/scratch/qiime2-provenance-4o84xp5y' -> '/scratch/qiime2-archive-dgz17z59/f6712e98-38c9-40c0-9b62-1f56b1524052/provenance'

See above for debug info
3

Hey @Pramod!

Well that's not supposed to happen... I've never seen this before, so I don't have a quick answer. Does re-running the import command succeed?

That'll let us know if it's a stochastic issue with the random names generated, or if its a fundamental issue we've never come accross.

If you know, what kind of filesystem is /scratch/?

Thanks!

1 Like
5

hi Evan,

I reinstalled qiime2 :
qiime info | grep ‘quality-control’
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
quality-control: 2018.6.1

and try to run the command again but the same error repeats
-rw-r–r--+ 1 pramod fs_bioinformatics_common 4583 Jun 28 12:03 qiime2-2018.6-py35-linux-conda.yml
(qiime2-2018.6) [pramod@herman-login2 Qiime]$ qiime tools import --type EMPPairedEndSequences --input-path emp-paired-end-sequences --output-path emp-paired-end-sequences.qza
Traceback (most recent call last):
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2cli/tools.py”, line 116, in import_data
view_type=source_format)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py”, line 219, in import_data
return cls.from_view(type, view, view_type, provenance_capture)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py”, line 250, in _from_view
provenance_capture=provenance_capture)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/archiver.py”, line 310, in from_data
Format.write(rec, type, format, data_initializer, provenance_capture)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/format/v1.py”, line 26, in write
prov_dir, [root / cls.METADATA_FILE, archive_record.version_fp])
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/archive/provenance.py”, line 315, in finalize
self.path.rename(final_path)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/pathlib.py”, line 1277, in rename
self._accessor.rename(self, target)
File “/home/pramod/miniconda3/envs/qiime2-2018.6/lib/python3.5/pathlib.py”, line 377, in wrapped
return strfunc(str(pathobjA), str(pathobjB), *args)
FileExistsError: [Errno 17] File exists: ‘/scratch/qiime2-provenance-difz5eeh’ -> ‘/scratch/qiime2-archive-isgo3hiy/b7f5c095-1f9f-480d-837b-aba392d161c4/provenance’

An unexpected error has occurred:

[Errno 17] File exists: ‘/scratch/qiime2-provenance-difz5eeh’ -> ‘/scratch/qiime2-archive-isgo3hiy/b7f5c095-1f9f-480d-837b-aba392d161c4/provenance’

See above for debug info.

and the file system info for scratch is as follows:

df -TH /scratch/
Filesystem Type Size Used Avail Use% Mounted on
beegfs_nodev beegfs 43T 24T 19T 57% /mnt/beegfs

Please suggest!

7

Thanks so much for the info! Sorry for the delayed response, I've been out of the office for a bit :frowning:

I don't think I've ever run across this filesystem before.

I'm going to have to do some research here as this issue really shouldn't be happening at all, but perhaps there's a known race-condition with Python's temporary filenames and this particular filesystem.

I'll report back when I know more. Thanks so much!

10

Just an update: I haven’t forgotten about this, but I haven’t had a chance to replicate either, sorry. Please stay tuned!

11

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.