# FASTQ Files Importing Error

**URL:** https://forum.qiime2.org/t/fastq-files-importing-error/6412
**Category:** User Support
**Created:** [October 11, 2018, 7:53pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412 "2018-10-11T19:53:14Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 11, 2018, 7:53pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/1 "2018-10-11T19:53:14Z")

</div>

I'm trying to import FASTQ files but I keep getting an error that my .csv file isn't the correct file format. Also, the sequences I'm importing have already been demultiplexed so I'm unsure if the importing method I'm using is incorrect (I'm using paired end Phred offset score 64). I'm unable to attach my FASTQ files but one of the sequences is:

CCGCCAATTTCTTTGAGTTTCAACCTTGCGGTCGTACTCCCCAGGCGGATTGCTTATTGCGTTAGCTGCGGCACTGATCCCCGGAAAGGTTCCAACCCCTAGCCTTCTTCTTTTACGGCTTGGACTACCAGGTTTCTTAATCTTTTTCCCTCCCCACGTTTTCGATCCTCACCTTCTGTTCAAGACCTGAACCCCGTTTTCCCCCCCGGTTTCCCCCCTTTTTTTTCCCCTTTTAACCGCTCCCCTTGGATTTCCCCTTCCCCCTCTTCCCTCCAATTTTGACATTTTCCAACGTAACCTT

- 

-ACCCEEEGGGGGGGCGGGFF@FGDFGEFFGGGCFEGCFGF:-@6\<C@@+8,,CCF9\<C,,@,,,,,:6@@F++C,C,9FF7C=@FFFE=,C\<E,\<+B=++44,,,,,9,,\<F,557+++8\>@=,5,A,,5\>5:,,,3:,35,:,C,,,,,8@@+,3++7D\>\>8+,8\*,@@,,,,,,,7;,,,5,62\>,4,1_4__4;1=\<0_=5=_9):+;0.5=)2_\*/)/))_0:):F:_)20(,()((9()0/4\*:\*)))..)(,42(-(,.))()((..4\<\>4)(-.).5))-4)((((((.-)

@M02352:182:000000000-BRDB5:1:1101:9865:2050 2:N:0:169

[fastq\_files\_demux.csv](https://cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/flex002/uploads/qiime21/original/2X/c/c99ceb07bdedd8347f0653334c4d2205d64c9c67.csv) (9.2 KB)

---

<div class="post-metadata">

### Author: ![colinbrislawn](https://forum.qiime2.org/user_avatar/forum.qiime2.org/colinbrislawn/32/6221_2.png) [@colinbrislawn](https://forum.qiime2.org/u/colinbrislawn)
#### Post date: [October 12, 2018, 3:58pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/2 "2018-10-12T15:58:29Z")

</div>

Good morning,

Could you post the full text of the error you are getting about your csv file? It's hard to say more without seeing the error itself, but I wonder if using the absolute-filepath in the absolute-filepath column would work better... For example, you could try using  
`/Users/stephanieorch/Desktop/disaster/WhitesonTest/16S/50064_S181_L001_R1_001.fastq.gz`  
Which does not start with the `$PDW` bash variable.

Let me know if omitting the `$PWD` helps, or you are able to post the full error so I can continue to look for clues.

Colin

---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 12, 2018, 4:48pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/3 "2018-10-12T16:48:03Z")

</div>

The error I receive is:

```auto
**There was a problem importing fastq_files_demux.csv:**

**fastq_files_demux.csv is not a(n) PairedEndFastqManifestPhred64 file**

```

I tried removing the $PWD and it still didn't work. Any other ideas?

---

<div class="post-metadata">

### Author: ![Yos.Dos](https://forum.qiime2.org/user_avatar/forum.qiime2.org/yos.dos/32/2976_2.png) [@Yos.Dos](https://forum.qiime2.org/u/Yos.Dos)
#### Post date: [October 12, 2018, 4:56pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/4 "2018-10-12T16:56:18Z")

</div>

Hey, What code you are typing to import your files? Cause i dont think that you need a metadata file.  
And as far as i know in qiime the format of the metadata file must be .tsv.

---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 12, 2018, 4:58pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/5 "2018-10-12T16:58:39Z")

</div>

I'm trying to import FASTQ files. Based on the importing tutorial, to import multiple FASTQ files into one .qza you have to create a .csv file that points to the FASTQ files.

```auto
qiime tools import \
--type 'SampleData[PairedEndSequencesWithQuality]' \
--input-path fastq_files_demux.csv \
--output-path demux_seqs.qza \
--input-format PairedEndFastqManifestPhred64

```

---

<div class="post-metadata">

### Author: ![ebolyen](https://forum.qiime2.org/user_avatar/forum.qiime2.org/ebolyen/32/11_2.png) [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [October 12, 2018, 5:21pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/6 "2018-10-12T17:21:42Z")

</div>



---

<div class="post-metadata">

### Author: ![Mehrbod\_Estaki](https://forum.qiime2.org/user_avatar/forum.qiime2.org/mehrbod_estaki/32/4001_2.png) [@Mehrbod\_Estaki](https://forum.qiime2.org/u/Mehrbod_Estaki)
#### Post date: [October 12, 2018, 5:22pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/7 "2018-10-12T17:22:01Z")

</div>

Hi @Stephanieorch,  
Assuming you're using your 2018.8 qiime version your commands all look ok to me. And the manifest file is indeed suppose to be a .csv file so that's not the issue either. And I don't have an answer for you either, but while one of the devs gives this a more thorough look I just wanted to ask if you're sure your phred score is actually 64 and not 33. I don't think this is the source of your problem since I expect you would get a different error for that but I looked at the quality scores you provided and based on the [33 vs 64 phred scores](https://www.drive5.com/usearch/manual/quality_score.html), **I think** yours makes more sense as type 33. I could be wrong but worth double checking. Again, I doubt this is going to solve your pbolem, but a potential problem later:P

---

<div class="post-metadata">

### Author: ![ebolyen](https://forum.qiime2.org/user_avatar/forum.qiime2.org/ebolyen/32/11_2.png) [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [October 12, 2018, 5:22pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/8 "2018-10-12T17:22:15Z")

</div>



---

<div class="post-metadata">

### Author: ![ebolyen](https://forum.qiime2.org/user_avatar/forum.qiime2.org/ebolyen/32/11_2.png) [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [October 12, 2018, 5:22pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/9 "2018-10-12T17:22:27Z")

</div>



---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 12, 2018, 5:24pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/11 "2018-10-12T17:24:39Z")

</div>

Yes, I thought that may be the problem to so I tried both 33 and 64 and still received an error.

---

<div class="post-metadata">

### Author: ![colinbrislawn](https://forum.qiime2.org/user_avatar/forum.qiime2.org/colinbrislawn/32/6221_2.png) [@colinbrislawn](https://forum.qiime2.org/u/colinbrislawn)
#### Post date: [October 12, 2018, 5:45pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/13 "2018-10-12T17:45:58Z")

</div>

Hello again Stephanie,

I'm glad lots of folks are Qiime-ing in this morning. To help out.

I think I may have found a clue.

 ![42%20AM](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/2X/e/e0f6392b602809ec02725836ec436e0d3bc6769d.png)

What program did you use to make the .csv file? I'm not sure if using Windows lie ending causes a problem, but I know some programs expect Unix line endings instead.

I've uploaded a version of this document using Unix line endings, which might be worth trying.  
[fastq\_files\_demux.csv](https://cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/flex002/uploads/qiime21/original/2X/b/b3765e075a3acab0958b8fc3474439ab1e4d7781.csv) (8.8 KB)

Colin

---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 12, 2018, 5:59pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/14 "2018-10-12T17:59:05Z")

</div>

I originally used excel to make the document and then used the command line to edit.

I tried the document you sent me and got the error again. I used Atom to try adding $PWD and see if that would work and it didnt. I also tried both Phred33 and Phred64 for all of these.

Any other ideas? Thanks!

---

<div class="post-metadata">

### Author: ![colinbrislawn](https://forum.qiime2.org/user_avatar/forum.qiime2.org/colinbrislawn/32/6221_2.png) [@colinbrislawn](https://forum.qiime2.org/u/colinbrislawn)
#### Post date: [October 12, 2018, 8:48pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/15 "2018-10-12T20:48:58Z")

</div>

I think you are doing everything right... I'm not sure what's going wrong.

As as a guess... what happens if you comment out or remove the sequences that start with `O.` like `O.50064`. Capitol Os and .periods. should not cause problems, but I'm just looking for any potential issues.

Qiime devs, is there a script (or internal function), we can use to validate this file? Looks like Keemei does not validate FastqManifest files, but maybe it could!

Colin

---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 12, 2018, 11:45pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/16 "2018-10-12T23:45:33Z")

</div>

I tried importing the files after removing the FASTQ files starting with Os from the .csv file and it still gave me the same errors.

---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 15, 2018, 4:10pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/17 "2018-10-15T16:10:02Z")

</div>

I solved the issue! Turns out .csv files DO NOT work. You MUST use TestEdit.

---

<div class="post-metadata">

### Author: ![colinbrislawn](https://forum.qiime2.org/user_avatar/forum.qiime2.org/colinbrislawn/32/6221_2.png) [@colinbrislawn](https://forum.qiime2.org/u/colinbrislawn)
#### Post date: [October 17, 2018, 8:06pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/18 "2018-10-17T20:06:29Z")

</div>

So saving as a `.csv` file using Excel doesn't work, but saving as a `.txt` file using TextEdit does? That could be really helpful for future users.

Colin

---

<div class="post-metadata">

### Author: ![Nicholas\_Bokulich](https://forum.qiime2.org/user_avatar/forum.qiime2.org/nicholas_bokulich/32/19937_2.png) [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [October 17, 2018, 8:42pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/19 "2018-10-17T20:42:18Z")

</div>

> [@Stephanieorch](#):
>
> I solved the issue! Turns out .csv files DO NOT work. You MUST use TestEdit.

> [@colinbrislawn](#):
>
> So saving as a `.csv` file using Excel doesn’t work, but saving as a `.txt` file using TextEdit does?

If you use Excel to make your CSV, it will insert CRLF line breaks (whether or not you are using windows, I believe). So you need to either just open in textedit and use find/replace to fix the line breaks, or use `mac2unix` or a program like that to convert the carriage returns.

So Excel is okay — just make sure to convert the line endings.

This will be an issue with making sample metadata files, too, not only manifest files.

---

<div class="post-metadata">

### Author: ![Nicholas\_Bokulich](https://forum.qiime2.org/user_avatar/forum.qiime2.org/nicholas_bokulich/32/19937_2.png) [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [October 17, 2018, 8:42pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/20 "2018-10-17T20:42:19Z")

</div>



---

<div class="post-metadata">

### Author: ![Stephanieorch](https://forum.qiime2.org/user_avatar/forum.qiime2.org/stephanieorch/32/4269_2.png) [@Stephanieorch](https://forum.qiime2.org/u/Stephanieorch)
#### Post date: [October 17, 2018, 9:57pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/21 "2018-10-17T21:57:20Z")

</div>

> [@colinbrislawn](#):
>
> Unix line endings

I believe @colinbrislawn tried to alleviate the line break issue when he gave me a document to try that he created using Unix line endings. However, this document did not work either.

---

<div class="post-metadata">

### Author: ![Nicholas\_Bokulich](https://forum.qiime2.org/user_avatar/forum.qiime2.org/nicholas_bokulich/32/19937_2.png) [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [October 18, 2018, 12:34pm UTC](https://forum.qiime2.org/t/fastq-files-importing-error/6412/22 "2018-10-18T12:34:01Z")

</div>

The issue could be other special characters that Excel is importing. In general, your advice to use textedit is valid: Excel creates headaches downstream!

[Next page](https://forum.qiime2.org/t/fastq-files-importing-error/6412.md?page=2)
