# Importing demultiplexed fastq data into qiime2-2017.7

**URL:** https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064
**Category:** User Support
**Tags:** miseq, pending-development, demux, import
**Created:** [August 21, 2017, 6:55pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064 "2017-08-21T18:55:40Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![eDNA](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/e/dec6dc/32.png) [@eDNA](https://forum.qiime2.org/u/eDNA)
#### Post date: [August 21, 2017, 6:55pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/1 "2017-08-21T18:55:40Z")

</div>

Hello,

I did Miseq pair-ended sequencing. I used other softwares to merge and demultiplex the sequences. Now, I am trying to import sequence data (each sample has one fastq file) into QIIME2.

My command:

```bash
qiime tools import \
  --type 'SampleData[SequencesWithQuality]' \
  --input-path /Volumes/xxxx/V4_QIIME2test \
  --output-path /Volumes/xxxx/V4_single-end-demux.qza \
  --source-format SingleEndFastqManifestPhred33

```

The manifest file:

```auto
sample-id,absolute-filepath,direction
BS_0_1_1,/Volumes/xxxx/BS_P1_V4/BS_0_1_1.fastq,forward
......

```

```bash
Error:
ValueError: InPath('/var/folders/zr/vqccwhr90rl1hdcykhvdly100000gp/T/q2-SingleLanePerSampleSingleEndFastqDirFmt-x_5h6kva/BS_0_2_2_4_L001_R1_001.fastq.gz') is not formatted as a FastqGzFormat file.

```

All my sequence files are fastq format. Does anybody know how to fix it? Thanks.

---

<div class="post-metadata">

### Author: ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)
#### Post date: [August 22, 2017, 5:09pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/2 "2017-08-22T17:09:09Z")

</div>

Hi @eDNA! The file that QIIME 2 is complaining about has the name `BS_0_2_2_4_L001_R1_001.fastq.gz`, which looks like a pretty different naming scheme compared to the one example sample you posted above (`BS_0_1_1.fastq`). We typically see filenames that are pretty consistently named --- are you sure your manifest file has the right filenames in it? Can you please post your complete manifest file, and a listing of the files in the data directory (could be a screenshot, or `ls` or `tree`), that will help us with troubleshooting this problem. Thanks!

---

<div class="post-metadata">

### Author: ![eDNA](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/e/dec6dc/32.png) [@eDNA](https://forum.qiime2.org/u/eDNA)
#### Post date: [August 22, 2017, 5:46pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/3 "2017-08-22T17:46:41Z")

</div>

Hey

Yes, my manifest file has the right filenames in it.

The listing of files in the data directory  
$ ls  
BS\_0\_1\_1.fastq BS\_15\_1\_3.fastq BS\_30\_2\_1.fastq  
BS\_0\_1\_2.fastq BS\_15\_2\_1.fastq BS\_30\_2\_2.fastq  
BS\_0\_1\_3.fastq BS\_15\_2\_2.fastq BS\_30\_2\_3.fastq  
BS\_0\_2\_1.fastq BS\_15\_2\_3.fastq BS\_500\_1\_1.fastq  
BS\_0\_2\_2.fastq BS\_30\_1\_1.fastq BS\_500\_1\_2.fastq  
BS\_0\_2\_3.fastq BS\_30\_1\_2.fastq BS\_500\_1\_3.fastq  
BS\_15\_1\_1.fastq BS\_30\_1\_2\_2.fastq BS\_Blank1.fastq  
BS\_15\_1\_2.fastq BS\_30\_1\_3.fastq BS\_NC1.fastq

I follow the “Fastq manifest” formats tutorial. The "BS\_0\_2\_2\_4\_L001\_R1\_001.fastq.gz" looks like a name in the "Casava 1.8 single-end demultiplexed fastq" tutorial. Did the command thought I was following the "Casava 1.8 single-end demultiplexed fastq" tutorial? It mentioned "q2-SingleLanePerSampleSingleEndFastqDirFmt" in the error.

---

<div class="post-metadata">

### Author: ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)
#### Post date: [August 22, 2017, 6:53pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/4 "2017-08-22T18:53:40Z")

</div>

Hi @eDNA, can you please also post your complete manifest file? Thanks!

---

<div class="post-metadata">

### Author: ![eDNA](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/e/dec6dc/32.png) [@eDNA](https://forum.qiime2.org/u/eDNA)
#### Post date: [August 22, 2017, 7:02pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/5 "2017-08-22T19:02:57Z")

</div>

Hi Matthew,

Can the reason be the sequence problem? My sequences are in lower case. Thanks.

---

<div class="post-metadata">

### Author: ![eDNA](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/e/dec6dc/32.png) [@eDNA](https://forum.qiime2.org/u/eDNA)
#### Post date: [August 23, 2017, 4:33pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/6 "2017-08-23T16:33:58Z")

</div>

Hello Matthew,

I used one sample to test. I think the error was caused by sequence data (lower case in DNA sequence).  
--------------- 1st test ---------------------  
My command:  
qiime tools import   
--type 'SampleData[SequencesWithQuality]'   
--input-path ImportTest2   
--output-path Test2\_single-end-demux.qza   
--source-format SingleEndFastqManifestPhred33

ImportTest2 file:  
sample-id,absolute-filepath,direction  
VP\_15\_2\_3,/Users/xxx/Test2/VP\_15\_2\_3.fastq,forward

The 1st sequence in VP\_15\_2\_3.fastq:  
@M\_M03670:6:000000000-BCN4V:1:1101:17050:1464\_SUB\_SUB\_CMP reverse\_score=80.0; forward\_score=76.0; direction=reverse; avg\_quality=39.7032967033; forward\_tag=gcatcatc; seq\_length\_ori=182; seq\_length=124; sample=VP\_15\_2\_3; forward\_match=aagggcaccacaagaacgc; experiment=eDNA; status=full; location=VP\_15; tail\_quality=35.4; reverse\_match=ccacctatcacacaatcatg; reverse\_tag=gcatcatc; forward\_primer=aagggcaccacaagaacgc; grab=VP\_15\_2; head\_quality=35.2; reverse\_primer=ccacctatcacayaatcatg; mid\_quality=40.2469135802; 1:N:0:4  
gtggagcatgtggcttaatttgactcaacgcagggaatcttaccgggtccggacacactgaggattgacagattaaagcggttgtcagtcttatgactggctccgttgaaagttacagctcttt  
+  
GGJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJHH

Error: ValueError: InPath('/var/folders/zr/vqccwhr90rl1hdcykhvdly100000gp/T/q2-SingleLanePerSampleSingleEndFastqDirFmt-87too7ky/VP\_15\_2\_3\_0\_L001\_R1\_001.fastq.gz') is not formatted as a FastqGzFormat file.

-------------- 2nd test-----------  
qiime tools import   
--type 'SampleData[SequencesWithQuality]'   
--input-path ImportTest1   
--output-path Test1\_single-end-demux.qza   
--source-format SingleEndFastqManifestPhred33

ImportTest1:  
sample-id,absolute-filepath,direction  
VP\_P1\_all,/Users/xxx/Test/VP\_P1\_assigned.fastq,forward

The 1st sequence in the sequence file:  
@M\_M03670:6:000000000-BCN4V:1:1101:15780:1331\_SUB\_SUB\_CMP seq\_length=132; sample=VP\_500\_1\_1; experiment=eDNA; location=VP\_500; grab=VP\_500\_1; forward\_score=76.0; forward\_tag=cgatgaca; reverse\_tag=cgatgaca; forward\_match=aagggcaccacaagaacgc; forward\_primer=aagggcaccacaagaacgc; reverse\_score=80.0; status=full; direction=reverse; reverse\_match=ccacctatcacataatcatg; reverse\_primer=ccacctatcacayaatcatg; head\_quality=34.6; avg\_quality=39.5052083333; seq\_length\_ori=192; tail\_quality=34.6; mid\_quality=40.0755813953; 1:N:0:4  
GTGGAGCATGTGGCTTAATTTGACTCAACGTGGGAAATCTTACCGGGTCCGGACATACTGAGGATTGACAGGCAATTGATGATTGCTTCGGTGTTAAAACCAGGCTTTCATCGCTAAATATGCTAGTCCTTT  
+  
GGGGGHHHHHHHJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJHHHHHHHHHH

## The command run successfully.

@jairideout I found that you mentioned lower case issue in DNA sequence in other posters.

My demultiplexed data (upper case) is in one fastq file (e.g. the VP\_P1\_assigned.fastq I used for 2nd test). I split the file based on the "sample" attribute in each sequence to obtain a fastq file for each sample so that I can import them into QIIME2, but the sequences are in lower case. Does anyone have any idea how to import my data in QIIME2? I am very keen to use QIIME 2 to analyze my data. Thank you.

---

<div class="post-metadata">

### Author: ![eDNA](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/e/dec6dc/32.png) [@eDNA](https://forum.qiime2.org/u/eDNA)
#### Post date: [August 23, 2017, 8:50pm UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/8 "2017-08-23T20:50:28Z")

</div>

Hello,

I did another test. I converted the sequences in VP\_15\_2\_3.fastq to uppercase and it can be imported. I will have to convert each of my 280 fastq files before I use QIIME2.  
My problem has been solved so far. Looking forward to getting results from QIIME2.

Thanks.  
Tom.

---

<div class="post-metadata">

### Author: ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)
#### Post date: [August 24, 2017, 3:15am UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/9 "2017-08-24T03:15:20Z")

</div>

Great, thanks for following up @eDNA. We have an existing [open issue](https://github.com/qiime2/q2-types/issues/91) on one of our bug trackers about the lowercase sequence issue. We will update this thread when a solution has been implemented, but at this time we have no ETA on when that will happen (I would assumed before for the end of 2018). Sounds like you have a workaround in place to get you moving forward. Thanks!

---

<div class="post-metadata">

### Author: ![system](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/2/1/21af5fe23cb6f4579467c66a9ed94e55274ca7bd.svg) [@system](https://forum.qiime2.org/u/system)
#### Post date: [September 24, 2017, 9:15am UTC](https://forum.qiime2.org/t/importing-demultiplexed-fastq-data-into-qiime2-2017-7/1064/11 "2017-09-24T09:15:39Z")

</div>

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.
