Problems with importing data

import

(pooja) #1

Hello, I am relatively new to qiime. I am using the version QIIME2 (2018.8) and I am facing problems with importing the data.

The data is paired end sequencing data generated from HiSeq-PE 250 for 30 samples. I have two separate file one for forward and one for reverse in fastq.gz format.

I have tried all the tutorials but somehow I am not able to solve my issue. It would be great if someone can help me.

Thank you in advance.


(Nicholas Bokulich) #2

Where are your barcodes? If they are part of the sequences, you should follow this tutorial. If they are in separate barcode files, you should use the EMP paired end format described in the importing tutorial.

Good luck!


(pooja) #3

I don’t have a barcode.fastq.gz file but the are in a separate metadata file along with primer sequence and I have raw sequences with barcodes


(Nicholas Bokulich) #4

So there you have it — use the tutorial I linked to above (cutadapt). Good luck!


(pooja) #5

qiime tools import \ --type MultiplexedSingleEndBarcodeInSequence \ --input-path /home/ice.mpg.de/pmehta/ITS_DATA/analysis/raw_pair_end_reads_demultiplexed^C \ --output-path multiplexed-seqs.qza
Usage: qiime tools import [OPTIONS]

Error: Missing option “–type”.

this is the error that is showing up, can you please help me with this


(pooja) #6

I have 30 samples, each with a file which has raw tags after reads merging. I combined all these fastq files into one fastq file and then converted into merged.fastq.gz format.

At this point I have one merged.fastq.gz and a metadata file.


(Chris Keefe) #9

(Chris Keefe) #10

Hi @pmehta!
Is the ^C in your input-path argument part of the actual file path, or is that a typo/artifact of some kind?


(Nicholas Bokulich) #11

(pooja) #12

Hii,

No that is some error.

I have used following command:

qiime tools import
–type MultiplexedPairedEndBarcodeInSequence
–input-path qiime2
–output-path multiplexed-seqs.qza

here qiime2 is name of directory where I have stored two files: forward.fastq.gz and reverse.fastq.gz

and when I run it in terminal, I get following error:

There was a problem importing qiime2:

Missing one or more files for MultiplexedPairedEndBarcodeInSequenceDirFmt: 'reverse.fastq.gz’


(pooja) #13

My files were originally names as:
DMP06592_L1_AF2.raw_1.fq.gz
DMP06592_L1_AF2.raw_2.fq.gz …

so I also created another directory named raw_pair_end_reads_demultiplexed with all forwards and reverse reads named as above and I used the following command:

qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path raw_pair_end_reads_demultiplexed
–input-format CasavaOneEightSingleLanePerSampleDirFmt
–output-path demux-paired-end.qza

and it resulted in following error:

There was a problem importing raw_pair_end_reads_demultiplexed:

Missing one or more files for CasavaOneEightSingleLanePerSampleDirFmt: '.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz’


(Nicholas Bokulich) #14

(Chris Keefe) #15

@pmehta, thank you for providing such a detailed description of your process. I’m pretty new here myself, and it helps so much! :hugs:

These errors both seem to point at incorrectly-formatted filenames. Is there any chance your reverse.fastq.gz is misspelled, named reverse.fq.gz, or there’s some similar typo?

Background Information:
qiime tools import is looking through your --input-path directories for files that meet specific naming conventions. The error messages you’re receiving describe the exact pattern of characters the command is looking for. If the names of your files don’t match the expected format for the type of Artifact you’re trying to create, import won’t be able to find them.

Hope this helps,
Chris


(Chris Keefe) #16

#17

[email protected]!
Have you looked at the tutorial for importing files using a manifest.tsv file. I guess this should work in importing your two .gz files into qiime2 for downstream analysis.


(pooja) #18

hello everyone,

Thank you for your help.
I was finally able to import data using the fastq manifest data import tutorial.


#19

hello, I also met the same questions, could you show me your solution or import code


(Justine) #20

Hi @whitewind,

Are you also having issues with casava imports in particular, or with importing in general?

Best,
Justine


(pooja) #21

yes sure,

My data was not supported by emp or casava formats. Therefore I generated a fastq manifest file (.csv or .txt).

Please check the “fastq manifest format section” in the following tutorial and generate a manifest file first:
https://docs.qiime2.org/2018.11/tutorials/importing/#sequence-data-with-sequence-quality-information-i-e-fastq

After I generated a manifest file as instructed in tutorial, I used the following code

qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path its_manifest
–output-path paired-end-demux.qza
–input-format PairedEndFastqManifestPhred33

It will take a minute or two to process and generate a qiime artifact.


(pooja) #22

My data was not supporting the casava or EMP formats. That was the issue.