Missing data when importing

I am having a similar problem - same error - with my input-path. I am using the Moving Pictures data. I checked and confirmed that the two fastq.gz files are in the directory emp-single-end-sequences. Any other suggestions as to how to properly format the input-path option? Thank you.

I am also having this problem, with the error:
Error: Invalid value for “–input-path”: Path “TrimmedTaxa.otu_table.biom” does not exist.

When I am in the directory containing that file and type “dir,” I can see the table with that name there. But I cannot get qiime2 to recognize it.

Any suggestions?

Hi @rwagner, thanks for reaching out! We need a bit more info from you to help assess the problem:

  1. What platform are you running QIIME 2 on (e.g. Mac OS X, Linux, Windows)? You mentioned running dir, which I thought was a Windows command. We do not support running QIIME 2 on Windows (yet!), so if you are attempting to run QIIME 2 on Windows we aren’t really in a great place to help you with that at the moment, because we don’t yet know what is involved with making things work on that platform (sorry!).
  2. Can you provide that exact import commands you were attempting to run?
  3. Can you provide directory listings of the files that you are attempting to import (e.g. the output ls emp-single-end-sequences)? And where is that directory relative to where you are executing the import command from?

Thanks so much!

Thanks for getting back to me. I am in Windows, but I had thought (hoped?) that using a terminal and docker was enough to get me around that. I am not particularly saavy when it comes to anything beyond Windows, so I’m out of my league here. Below I’ve listed my commands, but it seems likely that I need to head on over to qiime1 and learn that. Thanks again, and if you see something glaring here that I can fix, do let me know! I appreciate it.

Rachel

The commands and the error:

=================

c:\Users\Workstation>docker run -t -i -v C:/Users/Workstation/data qiime2/core:2017.6 qiime tools import \ --type EMPSingleEndSequences \ --input-path C:/Users/Workstation/qiime2-moving-pictures-tutorial/emp-single-end-sequences \ --output-path C:/Users/Workstation/qiime2-moving-pictures-tutorial/emp-single-end-sequences.qza
Usage: qiime tools import [OPTIONS]

Error: Invalid value for "--input-path": Path "C:/Users/Workstation/qiime2-moving-pictures-tutorial/emp-single-end-sequences" does not exist.

===================

I also tried it with the shorter path while “in” the directory that has the emp-single-end-sequences folder. I tried it with and without the slashes before --type etc, and also without the space between \ and --type (etc).

==================

c:\Users\Workstation\qiime2-moving-pictures-tutorial>docker run -t -i -v C:/Users/Workstation/data qiime2/core:2017.6 qiime tools import \ --type EMPSingleEndSequences \ --input-path emp-single-end-sequences \ --output-path emp-single-end-sequences.qza
Usage: qiime tools import [OPTIONS]

Error: Invalid value for "--input-path": Path "emp-single-end-sequences" does not exist.

==================

Here’s the directory info:

===================

c:\Users\Workstation\qiime2-moving-pictures-tutorial>dir
 Volume in drive C has no label.
 Volume Serial Number is 02EB-C666

 Directory of c:\Users\Workstation\qiime2-moving-pictures-tutorial

07/17/2017  12:55 PM    <DIR>          .
07/17/2017  12:55 PM    <DIR>          ..
07/17/2017  12:57 PM    <DIR>          emp-single-end-sequences
               0 File(s)              0 bytes
               3 Dir(s)  22,665,678,848 bytes free

====================

I also tried out, just FYI, importing a .biom table and I had similar problems.

I hope this can help in the long run with figuring out how to make Windows work out. Thanks for replying to me!

Sincerely,
Rachel

Hi @rwagner!

Ah, I understand! Technically docker is a virtualization environment, so when you run software in docker, it isn’t really running on the host OS (in your case, Windows), but actually in a Linux environment. Anyway, that is a bit of a technicality, and I should have been more explicit in my caveat above: we don’t support native installations of QIIME 2 on Windows! We do support running QIIME 2 on Windows via docker and Virtualbox!

Thanks for posting your commands, that was really helpful! There is a common issue with all of the commands, related to the volume mounting:

$ docker run -t -i -v C:/Users/Workstation/data qiime2/core:2017.6 qiime ...

This should actually be:

$ docker run -t -i -v C:/Users/Workstation/data:/data qiime2/core:2017.6 qiime ...

Note the :/data after your C:/Users/Workstation/data section.

This is what docker calls “volume mounting,” which maps data directories on the host machine (in your case, Windows) to the docker virtual machine! Also, please note this might still require a little bit of tweaking, there are a few versions of docker available for Windows, and they all have different requirements for how to format those file paths (e.g. C:/Users/Workstation/data). So, if the suggestion above doesn’t work, don’t fret, we still have options to work with!

Thanks!

1 Like

Question: Having assumed I was not going to be able to make qiime2 run, I started playing around with “original” qiime in a “virtual box” machine. I have made some but not tremendous progress. Is it your suggestion that I come back and work towards making qiime2 run in docker? Meanwhile, thank you for your tips on the commands, and for letting me know that it’s possible in the docker environment.

Personally I would suggest giving QIIME 2 a shot — we will be ending support for QIIME 1 at the beginning of 2018, so now would be a great time to get geared up. As always, if you have any questions or comments, or get stuck, you know where to find us!

Hi, this still isn’t working for me. I’m trying the EMPSingleEndSequences command for the moving pictures tutorial and it still says it

"C:\Users\Workstation>docker run -t -i -v C:/Users/Workstation/data:/data qiime2/core:2017.6 qiime"
gives me a good output, but

“C:\Users\Workstation>qiime tools import --type EMPSingleEndSequences --input-path emp-single-end-sequences --output-path emp-single-end-sequences.qza” gives me the following:

‘qiime’ is not recognized as an internal or external command,
operable program or batch file.

(No matter what my “parent” directory is.)

If I run it as “docker run -t -i -v C:/Users/Workstation/data:/data qiime2/core:2017.6 qiime tools import --type EMPSingleEndSequences --input-path emp-single-end-sequences --output-path emp-single-end-sequences.qza” then the error I get is as before:
Error: Invalid value for “–input-path”: Path “C:/Users/Workstation/data/qiime2-moving-pictures-tutorial/emp-single-end-sequences” does not exist.

Or, Error: Invalid value for “–input-path”: Path “emp-single-end-sequences” does not exist.

Depending on whether or not I try to give it a full path or just the path for the folder within the directory.

I guess this is probably a docker problem, not a qiime2 problem… but it’s all the same to me…

Thanks.

Hey @rwagner,

Actually the error you’re getting is just from QIIME 2, so in this case it looks like you’ve got docker more or less figured out! As you’ve noticed, every command needs to be prefixed with this:

docker run -t -i -v C:/Users/Workstation/data:/data qiime2/core:2017.6 

Both of these are saying that there isn’t a directory with that name, keep in mind that that the:

 -v C:/Users/Workstation/data:/data

part of the above docker command puts a local directory inside of your docker instance.

So as far as QIIME 2 can tell, the /data directory, which is really C:\Users\Workstation\data, doesn’t have the filepath you’ve given with --input-path.

Would you be able to provide a screenshot (or other listing) for what’s inside the C:\Users\Workstation\data directory?

2 Likes

Here's the directory:

Are the files within the folder “emp-single-end-sequences” supposed to be unzipped, or left in the zipped format for use with the “import” tool and EMPSingleEndSequences?

Hey @rwagner,

Looking at that screenshot, it does look like your volume mount should be:

 -v C:/Users/Workstation/data/qiime2-moving-pictures-tutorial:/data

(this is like cd-ing at the start of the tutorial, but since every docker command is basically a whole new computer, it doesn’t persist, so we mount/link the directory we want to use instead)

Alternatively you could set the input path to look like:

 --input-path qiime2-moving-pictures-tutorial/emp-single-end-sequences

(but that could get tiresome after a while)

They should be fastq.gz files, so you shouldn’t have to do anything different to them.

1 Like

I still can’t get this stuff to import. Any other ideas? I’ve tried on multiple machines (all Windows, using Docker). Any further advice or do I part with qiime2 for now?

Could you provice the exact commands that aren’t working?

In the meanwhile, one option would be to run it interactively which means just running:

docker run -t -i -v C:/Users/Workstation/data:/data qiime2/core:2017.6 

(notice there’s no qiime command)

You’ll then be dropped into a terminal where you’ll see the directories that were mounted, and you can cd and ls around like normal.

If that doesn’t work for you, I would use a VirtualBox image (since you are using Docker, you’ll need to disable Hypervisor to use Oracle VirtualBox). That will give you a “normal” filesystem that doesn’t require quite as much work.

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.