Import issue with Docker

docker
import

(Kim Lam Chiok) #1

Hello folks,

Super new to Qiime. Please be patient with me.
I recently installed the new Qime2 release using docker in my Windows 10 Pro computer. I have 10 paired end read samples (total 20 files) that I’ve used in the past in my Mac and a previous release of Qiime without any issues. I’m trying to import this same data in this new docker setup. I start qiime in docker like this:

PS C:\Users\chiok\desktop\EHDP_Qiime2_KC_2018> docker run -t -i -v ${pwd}:/data qiime2/core:2018.11 bash
(qiime2-2018.11) [email protected]:/data# ls
EHDP 2018 metadata.tsv  fasta  pe-64-manifest.txt

**EHDP_Qiime2_KC_2018 is my current working directory. The metadata file (which needs renaming probably) is the first file on the left. The second “fasta” is the folder where my fasta.gz are. The third file is my manifest pe-64-manifest.txt (1.7 KB). I first check that my fastas are there, like this:

(qiime2-2018.11) [email protected]:/data# cd fasta
(qiime2-2018.11) [email protected]:/data/fasta# ls
EHDP_10_R1.fastq.gz  EHDP_15_R1.fastq.gz  EHDP_19_R1.fastq.gz  EHDP_3_R1.fastq.gz  EHDP_6_R1.fastq.gz
EHDP_10_R2.fastq.gz  EHDP_15_R2.fastq.gz  EHDP_19_R2.fastq.gz  EHDP_3_R2.fastq.gz  EHDP_6_R2.fastq.gz
EHDP_13_R1.fastq.gz  EHDP_17_R1.fastq.gz  EHDP_1_R1.fastq.gz   EHDP_4_R1.fastq.gz  EHDP_7_R1.fastq.gz
EHDP_13_R2.fastq.gz  EHDP_17_R2.fastq.gz  EHDP_1_R2.fastq.gz   EHDP_4_R2.fastq.gz  EHDP_7_R2.fastq.gz

**They’re all there. So I go back to my working directory. I used this command to import my data:

(qiime2-2018.11) [email protected]:/data# qiime tools import \
> --type 'SampleData[PairedEndSequencesWithQuality]' \
> --input-path pe-64-manifest.txt \
> --output-path paired-end-demux.qza \
> --input-format PairedEndFastqManifestPhred33

**And I get the following error:

There was a problem importing pe-64-manifest.txt:

pe-64-manifest.txt is not a(n) PairedEndFastqManifestPhred33 file:

File referenced on line 2 could not be found (C:\Users\chiok\Desktop\EHDP_Qiime2_KC_2018\fasta\EHDP_1_R1.fastq.gz).

**I’ve been canvassing the Forum trying to find answers. I checked the files were there and I am executing the command in my working directory, outside the folder that contains my fastas. Is it because of the \ symbols in the file paths specified in the manifest file? I tried changing to / without any luck. I also tried removing the C: from file paths in my manifest. Is it Docker? I am going insane trying to figure out what I’m missing.
Any help is greatly appreciated.

Thank you,

K


(Matthew Ryan Dillon) #2

(Matthew Ryan Dillon) #3

Hey there @Kim_Lam_Chiok!

Fortunately, this is an easy fix!

First off, all of your data needs to be in your working directory (according to your first code block above, that is the case).

Then, you need to update your manifest to

sample-id,absolute-filepath,direction
EHDP-1,$PWD/fasta/EHDP_1_R1.fastq.gz,forward
EHDP-3,$PWD/fasta/EHDP_3_R1.fastq.gz,forward
EHDP-4,$PWD/fasta/EHDP_4_R1.fastq.gz,forward
...

Note, the $PWD could also say /data/ — the important takeaway is that Docker _has no idea about C:\Users\chiok\Desktop — the running container has its own filesystem.

Make those changes and give it a whirl — keep us posted! :qiime2: :t_rex:


(Matthew Ryan Dillon) #4

(Kim Lam Chiok) #5

You are a genius, thermokarst. It created the qza object and now I will try the rest of the pipeline. Hopefully, things will go smoother. We are also getting the VirtualBox version for now, but will likely get us Linux and run things there. I just like the many options we have for Qiime. Maximum effort!

K