QIIME2 installation with dependencies downloaded locally

Hi, I’m trying to install QIIME2 on a Fedora system, but due to poor internet connectivity, installation fails in the middle. So, I downloaded all the required files from the text file to a local disk individually and tried to divert the command to that path. But it was not successful. It always searches for online repos rather than local drive. Is there a way to achieve this?
All required packages are in the home directory in QIIME2 folder and the text file has been edited to show the path of each file and it was saved in the home directory. Kindly inform me on how to install these dependencies from a local path.

Thank you…

Hey @anwesh,

This isn’t going to be an officially supported installation as conda can often do some strange (and pretty unpredictable things), so there’s no guarantee this is going to work, but the following will probably work:

conda create -n qiime2-unofficial --offline DIRECTORY_TO_YOUR_PACKAGES/*.tar.bz2

You should then probably run the following tests in your new environment:

conda install pytest  # get the test runner first
pytest --pyargs q2_dada2
pytest --pyargs q2_diversity
pytest --pyargs q2_alignment

And also

qiime info

If all of that succeeds, then you likely have a working install!

1 Like

Hello @ebolyen,
Tried the installation, all went well…

> qiime info
showed no errors

However, “pytest” was not successful.
It says dependency is missing (missing _ _init __.py), something similar to that.

Could be something to do with my earlier installation procedure, as I had deviated a bit from what you have suggested…

I’m going to try a clean install.

Thank you…

1 Like

Would you be able to paste in the output and your command if you still run into that?

It sounds a little bit like maybe one of the packages (there's ~200) wasn't downloaded, so an import failed.

Let us know how it goes!

Hi, clean install completed without any errors. Pytest showed some warnings while testing q2_diversity…

pytest --pyargs q2_dada2
==== 11 passed in 155.00 seconds ====

pytest --pyargs q2_alignment
==== 19 passed in 3.44 seconds ====

pytest --pyargs q2_diversity
==== 126 passed, 21 warnings in 32.25 seconds ====

Some of the warnings are like this…

==== warnings summary ====
Warning: alpha of 1.0 is weighted-normalized UniFrac. Weighted-normalized is being used instead as it is more optimized.
Warning: divide by zero encountered in double_scalars
Warning: invalid value encountered in true_divide
Warning: invalid value encountered in less
Warning: Data with input dtype int64 was converted to bool by check_pairwise_arrays.

I guess they are of less concern.

qiime info
No errors…

So, I proceeded further to import the demultiplexed fastq.gz files, where I faced OSErrors 18 and 28.

qiime tools import \
--type 'SampleData[PairedEndSequencesWithQuality]' \
--input-path bst_pe-33_manifest \
--output-path Att1/paired-end-demux.qza \
--source-format PairedEndFastqManifestPhred33

I found that “OSError: [Errno 18] Invalid cross-device link:” is related to filesystem type and it was rectified as per #274.

I have 36 pairs of fastq.gz files each pair about 900mb. When I use all the files these errors occur. If I limit the number to 2 pairs of input files, the process is getting completed. So, I guess this is memory problem of the “tmp” directory. I’ll change the path of “tmp” directory in my next attempt.
Is there a way to select the location to tmp while installation?

Apart from that, offline installation may work without any error, I guess

Thank you.

Awesome!

Yeah those are mostly the tests doing odd things on purpose (ideally). But aren't something you need to worry about.

Nice sleuthing! What version are you running (i.e. what does qiime info say?). That error shouldn't really be happening anymore.

That's honestly very strange. I don't think it should have anything to do with your /tmp directory (but if you want to play with it, you can set the TMPDIR environment variable in your shell to a different directory and QIIME 2 will respect that for a given command call).

It also sounds like you are getting cross-device links when you try to use all of the files, but not when you use just 2 of them? Are any of the files in your manifest on a different hard-drive (or usb stick)?

I'm a little worried that maybe something didn't work quite like we would hope for the offline install. So I'm not sure how much help I can really provide, since this isn't an installation scheme we test or support in any way.

2 Likes

2017.10

Yes, may be...

I checked the space used by /tmp directory, it was 3.9 GB. With this space, I was able to process not more than 3 samples at a time. So, I increased the space relative to the total size of my samples, and it worked.

That being said, the error it showed [Errno 18] Invalid cross-device link in the previous attempt has nothing to do in this scenario, moreover, [Errno 28] No space left on device which was shown at the end, is the one to be concerned about.

I guess, I have a working installation for now...

Thank you very much for the help...

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.