Problems with importing my fastq data

User Support
1

Hello! I have multiple shotgun metagenomic data that I want to import into qiime2. Before the import, I processed my reads with Kraken2 to remove sequences that map to human genome. I do not have the barcodes file for my fastq data presented in the importing data tutorial so I decided to make a manifest file. This is how my file looks like:

sample-id forward-absolute-filepath reverse-absolute-filepath
10x /scratch/general/nfs1/Keri/codes/unclassified_reads/forward_unclassified/10x_1_nohum.fastq_forward_unclassified.fastq /scratch/general/nfs1/Keri/codes/
unclassified_reads/reverse_unclassified/10x_2_nohum.fastq_reverse_unclassified.fastq
11x /scratch/general/nfs1/Keri/codes/unclassified_reads/forward_unclassified/11x_1_nohum.fastq_forward_unclassified.fastq /scratch/general/nfs1/Keri/codes/
unclassified_reads/reverse_unclassified/11x_2_nohum.fastq_reverse_unclassified.fastq
12x /scratch/general/nfs1/Keri/codes/unclassified_reads/forward_unclassified/12x_1_nohum.fastq_forward_unclassified.fastq /scratch/general/nfs1/Keri/codes/
unclassified_reads/reverse_unclassified/12x_2_nohum.fastq_reverse_unclassified.fastq
13x /scratch/general/nfs1/Keri/codes/unclassified_reads/forward_unclassified/13x_1_nohum.fastq_forward_unclassified.fastq /scratch/general/nfs1/Keri/codes/
unclassified_reads/reverse_unclassified/13x_2_nohum.fastq_reverse_unclassified.fastq

and this is my code:

qiime tools import
--type 'SampleData[SequencesWithQuality]'
--input-path /scratch/general/nfs1/Keri/codes/manifest.txt
--output-path single-end-demux.qza
--input-format SingleEndFastqManifestPhred64V2

and this is the error I keep on getting:

/scratch/general/nfs1/Keri/codes/manifest.txt is not a(n) SingleEndFastqManifestPhred64V2 file:

Found unrecognized ID column name 'sample-id forward-absolute-filepath
reverse-absolute-filepath' while searching for header. The first column name in the header defines the ID column, and must be one of these values:

Case-insensitive: 'feature id', 'feature-id', 'featureid', 'id', 'sample id', 'sample-id', 'sampleid'

Case-sensitive: '#OTU ID', '#OTUID', '#Sample ID', '#SampleID', 'sample_name'

NOTE: Metadata files must contain tab-separated values.

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: https://keemei.qii
me2.org

Find details on QIIME 2 metadata requirements here: Metadata in QIIME 2 — QIIME 2 2023.5.1 documentation

2

Hello @Kerianalee_Rivera,

From the error message:

Found unrecognized ID column name 'sample-id forward-absolute-filepath
reverse-absolute-filepath' while searching for header.
(...)
NOTE: Metadata files must contain tab-separated values.

It's probably reading your columns as one column because your columns aren't tab- but space-separated.

3

Hello! I am trying to import some forward and reverse reads into qiime2 but I am not able to. First, I do not have the barcodes so I created a manifest.txt file and validated it with the keemi add on in google sheets, but I keep on getting this error:

There was a problem importing /scratch/general/nfs1/Keri/codes/docs_manifest.txt - Sheet1.tsv:

/scratch/general/nfs1/Keri/codes/docs_manifest.txt - Sheet1.tsv is not a(n) PairedEndFastqManifestPhred33V2 file:

Filepath on line 1 and column "forward-absolute-filepath" could not be found (/scratch/general/nfs1/Keri/codes/unclassified_reads/forward_unclassified/10x_
1_nohum.fastq_forward_unclassified.fastq) for sample "10x".

and this is the code that I am trying to run:

SCRATCH=/scratch/general/nfs1/Keri

module load anaconda3/2023.03
module load qiime2/2023.5

qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path /scratch/general/nfs1/Keri/codes/'docs_manifest.txt - Sheet1.tsv'
--output-path paired-end-demux.qza
--input-format PairedEndFastqManifestPhred33V2

What can I do about it?

4

Hello @Kerianalee_Rivera,

This simply means that that file path does not exist. Either the file exists in a different directory, or there is a typo somewhere in the path.

I would strongly recommend against having spaces in your filenames, that will only complicate things.

5

Hello! I fixed my manifest file with your recommendations but now I am getting a different error, this is the error:

There was a problem importing /scratch/general/nfs1/Keri/codes/manifest_direction.txt:

/scratch/general/nfs1/Keri/codes/manifest_direction.txt is not a(n) PairedEndFastqManif
estPhred33V2 file:

'forward-absolute-filepath' is not a column in the metadata. Available columns: 'absolu
te-filepath', 'direction'

I first tried to do my manifest file following this template:

sample-id forward-absolute-filepath reverse-absolute-filepath

but i got the same error, so someone in my lab recommended for me to use this instead:

sample-id absolute-filepath direction

is there anything I can do?

6

Hello @Kerianalee_Rivera,

It seems like you changed two things at once. What happened after only correcting the paths? Everything should have been fine after that.

The column structure sample-id | absolute-file path | direction is not supported by qiime. You have to use the sample-id | forward-absolute-filepath | reverse-absolute-filepath structure for paired-end reads.

7

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.