Create manifest file from fastq

[Tahmine_Aldaghi]

Hi. I have a question. I use this command:

ls *_1.fastq | sed 's/_1.fastq//' | while read s; do \
echo -e "$s\t$PWD/${s}_1.fastq\tforward"; \
echo -e "$s\t$PWD/${s}_2.fastq\treverse"; \
done > manifest.csv to create manifest file.
I chekced the output with keemei and downloded the new file and run the command:
qiime tools import \
--type 'SampleData[PairedEndSequencesWithQuality]' \
--input-path manifest.csv \
--output-path demux.qza \
--input-format PairedEndFastqManifestPhred33V2
It shows an error:
There was a problem importing manifest.csv:

manifest.csv is not a(n) PairedEndFastqManifestPhred33V2 file:

Found unrecognized ID column name 'sample-id,absolute-filepath,direction' while searching for header. The first column name in the header defines the ID column, and must be one of these values:

Case-insensitive: 'feature id', 'feature-id', 'featureid', 'id', 'sample id', 'sample-id', 'sampleid'

Case-sensitive: '#OTU ID', '#OTUID', '#Sample ID', '#SampleID', 'sample_name'

NOTE: Metadata files must contain tab-separated values.

There may be more errors present in the metadata file. To get a full report, sample/feature metadata files can be validated with Keemei: [https://keemei.qiime2.org](javascript:void0

What should I do? Thanks in advance for your help.

[lizgehret]

Hey @Tahmine_Aldaghi,

Welcome to the forum

Here's where the problem lies:

@Tahmine_Aldaghi

Found unrecognized ID column name 'sample-id,absolute-filepath,direction' while searching for header.

The important bit to notice here is that all of your columns were identified as a single column name from the error message. Just below this we see:

@Tahmine_Aldaghi

The first column name in the header defines the ID column, and must be one of these values:

Case-insensitive: 'feature id', 'feature-id', 'featureid', 'id', 'sample id', 'sample-id', 'sampleid'

Case-sensitive: '#OTU ID', '#OTUID', '#Sample ID', '#SampleID', 'sample_name'

NOTE: Metadata files must contain tab-separated values.

Without looking at your manifest file directly, it appears as though you are missing tabs between each column name, which is why the collection of your columns is being idenfitied as a single column name.

You'll need to either modify or re-create your manifest file in tab separated format (rather than comma separated format). This change should resolve the error that you're currently seeing.

Cheers

[Tahmine_Aldaghi]

Thanks for your answer.
I did it. I did the below command:
printf "sample-id\tabsolute-filepath\tdirection\n" > manifest.tsv
for f in *_1.fastq; do
s=${f%_1.fastq}
printf "%s\t%s/%s_1.fastq\tforward\n" "$s" "$PWD" "$s" >> manifest.tsv
printf "%s\t%s/%s_2.fastq\treverse\n" "$s" "$PWD" "$s" >> manifest.tsv
done
and then:
qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path manifest.tsv
--output-path demux.qza
--input-format PairedEndFastqManifestPhred33V2
But again, another error:
There was a problem importing manifest.tsv:
manifest.tsv is not a(n) PairedEndFastqManifestPhred33V2 file:
There was an issue with loading the metadata file:
Metadaza IDs must be unique. The following IDs are duplicated: 'SRR14293234', 'SRR14293237', 'SRR14293238', 'SRR14293239', 'SRR14293240', 'SRR14293241', 'SRR14293244', 'SRR14293246', 'SRR14293247', 'SRR14293249', 'SRR14293250', 'SRR14293251', 'SRR14293253', 'SRR14293255', 'SRR14293256', 'SRR14293258', ...
But it is not a metadata file.
I dont know what should I do??

[lizgehret]

Hey @Tahmine_Aldaghi,

Take a close look at the error this time around:

@Tahmine_Aldaghi

There was an issue with loading the metadata file:
Metadata IDs must be unique. The following IDs are duplicated: 'SRR14293234', 'SRR14293237', 'SRR14293238', 'SRR14293239', 'SRR14293240', 'SRR14293241', 'SRR14293244', 'SRR14293246', 'SRR14293247', 'SRR14293249', 'SRR14293250', 'SRR14293251', 'SRR14293253', 'SRR14293255', 'SRR14293256', 'SRR14293258', ...

It looks like you have duplicate IDs in this manifest file - which is a separate issue from before. I'd recommend examining your manifest file to confirm if you expect these IDs to show up more than once, or if this is due to a formatting issue.

Cheers

[Tahmine_Aldaghi]

I have downloaded srr from ncbi through:
while read srr; do prefetch $srr; done < srr_list.txt
Transformed to FASTQ:
while read srr; do fasterq-dump $srr --split-files --threads 4; done < srr_list.txt
And now I want to create that manifest to import to QIIME.

I tried two ways, but both had errors. One mentioned that the file should be tab-separated, but when I did so, I got another error because the sample names are not unique. I even used keemei to validate my manifest file. But again, an error. Is there any other way?

[lizgehret]

Hey @Tahmine_Aldaghi,

It looks like you've fixed the tab separation problem, the main issue now is that you’re creating the older 3-column manifest layout and then trying to import it as PairedEndFastqManifestPhred33V2.

For PairedEndFastqManifestPhred33V2, the format should be one row per sample, with separate forward and reverse filepath columns, like this:

sample-id	 forward-absolute-filepath	      reverse-absolute-filepath
SRR14293234	 /full/path/SRR14293234_1.fastq	  /full/path/SRR14293234_2.fastq
SRR14293237	 /full/path/SRR14293237_1.fastq	  /full/path/SRR14293237_2.fastq

This formatting issue actually may be where the duplicated IDs are coming from - so fixing the format of your manifest file may resolve this. But with that being said, since these are SRA accessions, you may find it easier to use q2-fondue to pull the data directly from SRA into QIIME 2 artifacts, which can avoid the manifest-building step entirely. This plugin is available in the QIIME 2 2026.1 amplicon distribution.

Cheers