Problems with importing data after some manipulations

[Yos.Dos (Yosef Avrahami)]

Hello Qiimers!

I have sequences in single reads:

Barpear_NP_11-Endo-AT_S18_L001_R1_001.fastq.gz Barpear_NP_7-Exo-AT_S13_L001_R1_001.fastq.gz Barpear_NP_NTC2_S22_L001_R1_001.fastq.gz Barpear_NP_11-Exo-AT_S17_L001_R1_001.fastq.gz Barpear_NP_9-Endo-AT_S16_L001_R1_001.fastq.gz Barpear_NP_NTC_S19_L001_R1_001.fastq.gz Barpear_NP_1-Endo-AT_S8_L001_R1_001.fastq.gz Barpear_NP_9-Exo-AT_S15_L001_R1_001.fastq.gz Barpear_NP_S10-AT_S5_L001_R1_001.fastq.gz Barpear_NP_1-Exo-AT_S7_L001_R1_001.fastq.gz Barpear_NP_core7_10S-Ashraf_S1_L001_R1_001.fastq.gz Barpear_NP_S12-AT_S6_L001_R1_001.fastq.gz Barpear_NP_3-Endo-AT_S10_L001_R1_001.fastq.gz Barpear_NP_core7_12S-Ashraf_S2_L001_R1_001.fastq.gz Barpear_NP_S2-AT_S1_L001_R1_001.fastq.gz Barpear_NP_3-Exo-AT_S9_L001_R1_001.fastq.gz Barpear_NP_core7_2S-Ashraf_S3_L001_R1_001.fastq.gz Barpear_NP_S4-AT_S2_L001_R1_001.fastq.gz Barpear_NP_5-Endo-AT_S12_L001_R1_001.fastq.gz Barpear_NP_core7_4S-Ashraf_S4_L001_R1_001.fastq.gz Barpear_NP_S6-AT_S3_L001_R1_001.fastq.gz Barpear_NP_5-Exo-AT_S11_L001_R1_001.fastq.gz Barpear_NP_core7_6S-Ashraf_S5_L001_R1_001.fastq.gz Barpear_NP_S8-AT_S4_L001_R1_001.fastq.gz Barpear_NP_7-Endo-AT_S14_L001_R1_001.fastq.gz Barpear_NP_core7_8S-Ashraf_S6_L001_R1_001.fastq.gz

I tried with:

   qiime tools import \
          --type 'SampleData[SequencesWithQuality]' \
          --input-path new \
          --input-format CasavaOneEightSingleLanePerSampleDirFmt \
          --output-path demux-single-end.qza

and i got this error:

There was a problem importing new:

  new/Barpear_NP_3-Endo-AT_S10_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

  Header on line 13 is not FASTQ, records may be misaligned

NOTE: 1. When i tried to import particular files it worked, i don’t understand why.
2. Those file were paired-end reads and i did some manipulation on the files before i tried to import them (PEAR, Bowtie and some other name changing)

thank you all

[ebolyen (Evan Bolyen)]

Hi @Yos.Dos,

If you were to decompress Barpear_NP_3-Endo-AT_S10_L001_R1_001.fastq.gz (with something like gunzip or whatever utility your OS has baked in), what is in the first 15 lines?

[Yos.Dos (Yosef Avrahami)]

Hey,
Thanks for you response.
This are the first lines of this sample… (head):

@3-Endo-AT_S10.1101.17161.1634 1.N.0.AGCCTATC
GTGTCAGCAGCCGCGGTAAGACAGAGGATGCAAGCGTTATCCGGAATGATTGGGCGTAAAGCGTCTGTAGGTGGCTTTTTAAGTCCGCCGTCAAATCCCAGGGCTCAACCCTGGACAGGCGGTGGAAACTTCCAATCTGGTGTACGGTAGGGGAAGAGGGAAGTTCCGGTGGAGCGGTGAAATGCGGAGAGATAGGAAAGAACACCAACGGCGAAAGCACTCTGCTGGGCCGACACTGACACTGAGAGACGAAAGCTAGGGGAGCGAATGGGATTAGAAACCCGCGTAGTA
+
1>1AAFFFFFFAGGGCFA0DGGC000C0B1B111//BE/B2D////B11DD10/0/>///DB/BEE/G221F1/0BFGHF/11F@1/>?>EE//1F<EC11//<FC11<B?G0/001//?//>//<11<>11>111<>111/HIIII<II==>1GCGGDF??/////<@1CCB</BB2FGC?//<B1F2GBB21BHHGFFE?/?0/E>//E/HHGBB1B/0000//>?>//2D2D1AEF2D21FABA/B/FHFDA0/EEEEFEA2ABB11EFB22FGEF>01111B1>>11
@3-Endo-AT_S10.1101.13462.1635 1.N.0.AGCCTATC
GTGTCAGCAGCCGCGGTAAGACGGGGGGGGCTAGTGTTCTTCGGATTTACTGGGCGTAAAGGGCACGTAGGCGGTGTATCGGGTTGTATGTGAAATTCGCCAACTCCTCCCGGAATGCTCTCTTAACCCTTTCACTTTGGTGAGACAGAGGAGAGTGGAATTTCGAGTGTAGGGGTGAAATCCGCAAATCTACGAAGGAAGACCAAAAGCGAAGGCAGCTCACTGGGACCCTACCGACGCTGGGGTGCGAAAGCATGGGGAGCGAACAGGATTAGAAACCCGAGTAGAA
+
1>1>AFFFDFFAEEGGGEA1BF?0A///--9---99BF/BBF----;//9;//---;---//---9-;---/--;--/;/99-;A--///;//////;--9---//99-/;----9//9;9//////.--;BB/BF///II=FH1?IIIIIGHGF1DF??/0//@1@1F/F>0DF2DGB>/</>1F2B//0F/F2GEFFB1//1G>F>?/GHGB@1B@111111>?B0////A//EB/0//A/AA/A/EHGDA0A0AAAABA03FF1GBBB33F1EF>11113B1>>11
--

[ebolyen (Evan Bolyen)]

Thanks @Yos.Dos,

Would you be able to run head with something like -n 15? It looks like line 13 is specifically suspect.

And then you can paste in your response with a “code fence” which will avoid treating any of the contents like markup:

```text
The triple backtick (~` key) will treat everything within it as un-formatted text.
Adding "text" or a language like "bash" at the end of the starting fence will set the
syntax highlighting.
```

[Yos.Dos (Yosef Avrahami)]

Thank you

here there are:

@3-Endo-AT_S10.1101.17161.1634 1.N.0.AGCCTATC
GTGTCAGCAGCCGCGGTAAGACAGAGGATGCAAGCGTTATCCGGAATGATTGGGCGTAAAGCGTCTGTAGGTGGCTTTTTAAGTCCGCCGTCAAATCCCAGGGCTCAACCCTGGACAGGCGGTGGAAACTTCCAATCTGGTGTACGGTAGGGGAAGAGGGAAGTTCCGGTGGAGCGGTGAAATGCGGAGAGATAGGAAAGAACACCAACGGCGAAAGCACTCTGCTGGGCCGACACTGACACTGAGAGACGAAAGCTAGGGGAGCGAATGGGATTAGAAACCCGCGTAGTA
+
1>1AAFFFFFFAGGGCFA0DGGC000C0B1B111//BE/B2D////B11DD10/0/>///DB/BEE/G221F1/0BFGHF/11F@1/>?>EE//1F<EC11//<FC11<B?G0/001//?//>//<11<>11>111<>111/HIIII<II==>1GCGGDF??/////<@1CCB</BB2FGC?//<B1F2GBB21BHHGFFE?/?0/E>//E/HHGBB1B/0000//>?>//2D2D1AEF2D21FABA/B/FHFDA0/EEEEFEA2ABB11EFB22FGEF>01111B1>>11
@3-Endo-AT_S10.1101.13462.1635 1.N.0.AGCCTATC
GTGTCAGCAGCCGCGGTAAGACGGGGGGGGCTAGTGTTCTTCGGATTTACTGGGCGTAAAGGGCACGTAGGCGGTGTATCGGGTTGTATGTGAAATTCGCCAACTCCTCCCGGAATGCTCTCTTAACCCTTTCACTTTGGTGAGACAGAGGAGAGTGGAATTTCGAGTGTAGGGGTGAAATCCGCAAATCTACGAAGGAAGACCAAAAGCGAAGGCAGCTCACTGGGACCCTACCGACGCTGGGGTGCGAAAGCATGGGGAGCGAACAGGATTAGAAACCCGAGTAGAA
+
1>1>AFFFDFFAEEGGGEA1BF?0A///--9---99BF/BBF----;//9;//---;---//---9-;---/--;--/;/99-;A--///;//////;--9---//99-/;----9//9;9//////.--;BB/BF///II=FH1?IIIIIGHGF1DF??/0//@1@1F/F>0DF2DGB>/</>1F2B//0F/F2GEFFB1//1G>F>?/GHGB@1B@111111>?B0////A//EB/0//A/AA/A/EHGDA0A0AAAABA03FF1GBBB33F1EF>11113B1>>11
--
@3-Endo-AT_S10.1101.17575.1666 1.N.0.AGCCTATC
GTGTCAGCAGCCGCGGTAAGACAGAGGATGCAAGCGTTATCCGGAATGATTGGGCGTAAAGCGTCTGTAGGTGGCTTTTTAAGTCCGCCGTCAAATCCCAGGGCTCAACCCTGGACAGGCGGTGGAAACTACCAAGCTGGAGTACGGTAGGGGCAGAGGGAATTTCCGGGGGAGCGGTGAAATGCGTAGAGATCGGAAAGAACACCAACGGCGAAAGCACTCTGCTGGGCCGACACTGACACTGAGAGACGAAAGCTAGGGGAGCGAATGGGATTAGAAACCCCCGTAGTA
+
3A?ABFFFFFFBEGGCEAC35DGC33E2BAC35AA2BE2FFG1010B33DD5B1B0>0115@1BAE1G443B3??GFGHH133FEFE?E/EED/BFDFG3B//?GC11BGDHBA0@1/<F@CC@DH11<F1><1<0<<1/..IIHGHIIIIGD0HHHGFHG<----?A?/ECA//FF2GFE?/</B2HFGB?//HHHGGFE?/?0/E>/EE?HHGGG@B/000B//@?>//BD2F@2DD2D21GFCGEF/FHGDA0/HFEEEEA2BFB1FFAB2AGDEE>01111B1>>11
--
@3-Endo-AT_S10.1101.17465.1684 1.N.0.AGCCTATC

[Yos.Dos (Yosef Avrahami)]

OK, thank you @ebolyen
i solved my problem.
the two dashes that somehow (i guess with grep), entered my sequences did all the mess.
so i just removed those line away.
thank you again!