Importing the demultiplexed merged data

Hi!

I have used Qiime many times in my previous projects, but this is my first time using Qiime2 and I am having trouble starting the analysis.
I got the demultiplexed and merged data (R1 and R2 joined) from the sequencing facility and the quality is also good.
The way I am planning to do the analysis is:
-Import the demultiplexed merged data into qiime artifact.
-Denoise the data using deblur
-Downstream analysis.
I would like to know if my approach is correct?
Also, when I am trying to import the data into qiime artifact but I am getting an error.
The command I am using:

qiime tools import --type SampleData[JoinedSequencesWithQuality] --input-path pe-joined-manifest.csv --output-path paired-end-2samples

There was a problem importing pe-joined-manifest.csv:

Importing 'SingleLanePerSampleSingleEndFastqDirFmt' requires a directory, not pe-joined-manifest.csv

My manifest file looks like:

# joined paired-end PHRED 33 fastq manifest file for forward reads
Pool-1,$pth/Pool-1_1.fastq,forward
Pool-2,$pth/Pool-2_1.fastq,forward

I would appreciate any kind of help.

Thanks.

EDIT:
I was able to import the data by moving my manifest file in the same directory as my samples fastq files.
But please let me know if my strategy is correct.

Thanks!

1 Like

Hi bioingo_C,

I am facing the same issue and cannot get any help. Your post is the closest one. Could you please tell me exactly how you managed to import your data?
When I moved manifest file into the same directory where my fastq files are located, I am still getting the error message:

Missing one or more files for SingleLanePerSamplePairedEndFastqDirFmt: '.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz'

P.S. I am working with paired-end data, but don't think it matters here at this point.

Hi adoloman,

I think moving the manifest file to the sample directory is not that important as later I checked without moving and was able to import the data.
My issue was not well formatted manifest file.
Make sure there are no hidden characters and extra line in the manifest file, also the header should be exactly like this:
sample-id,absolute-filepath,direction

Hope this helps!

1 Like

Still cannot figure it out. Manifest file looks to be without additional spaces.
What confuses me most is in the Error it refers to the missing file which is not my fastq file. Looks like it's something from the samples listed in the format from qiime manual. Are we really supposed to use

SingleLanePerSamplePairedEndFastqDirFmt

without any modifications?

Hi adoloman,

Sorry to hear that you are still having problems importing the data.
I did get such error while trying to import the data.
If possible can you post your commands and manifest file.
I still believe that manifest file is the culprit.

Thanks!

Thanks for trying to help!
Here is the command I use:

qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path for-qiime2
--source-format CasavaOneEightSingleLanePerSampleDirFmt
--output-path demux-paired-end.qza

And here is the manifest filemanifestGranules.csv (601 Bytes)
manifestGranules.csv (601 Bytes)

Thanks!

qiime tools import –type ‘SampleData[PairedEndSequencesWithQuality]’–input-path for-qiime2–source-format CasavaOneEightSingleLanePerSampleDirFmt –output-path demux-paired-end.qza

You didn’t give the manifest file name at input path.

Got it! Gave the manifest file at the --input-path and did it inside the directory where all my sequences and the manifest file were located.
Looks like it didn’t like that I was not inside that directory. Specifying the path doesn’t work for some reason.
So strange!!

Thanks a lot for helping!

Hey there @adoloman!

If the manifest is in a different directory, you will need to specify the new path to that file, make sense? Nothing is happening automagically here - it all comes down to computers being pretty uninformed - your job is to tell the computer everything it might need to know (e.g. the path to the file it should be looking at).

It isn't clear to me, did you get this working?

Thermokast,
Don’t treat me like a computer noob. I have been using Linux/Mac systems for 6 years now, and most of my work is in Terminal.
Meanwhile, looks like it wasn’t clear what happened here. In my command I specified absolute path to manifest file, not relative to currently used directory. What happened next: qiime was able to read content of that file, but was not able to process files specified there until I copied that file into a working directory and used a relative path. And this makes zero sense.

Also, the manual on importing the files does not specify that everything should be located in the same directory;

This is VERY illogical and thus, should be specified on the manual webpage.

Ah, sorry to hear that! Is there something specific that doesn't make sense here?

I think there might be a misunderstanding here --- no such requirement exists here! This actually ties into my post above --- the import process depends on you telling QIIME 2 where to find files. This happens in two place. First, you tell QIIME 2 where your manifest file is (--input-path), then, inside the manifest file you tell QIIME 2 where to find the individual files necessary for import (there are the entries under the column absolute-filepath). Again, you have to be explicit here, and this really is just the nature of computing (that is, it isn't a problem specific to or unique to QIIME 2).

I took a quick read through your commands above:

you are specifying the wrong source format for working with a manfiest file --- this should be --source-format PairedEndFastqManifestPhred33, CasavaOneEightSingleLanePerSampleDirFmt is an entirely different format.

It sounds like you maybe haven't had a chance to review the Importing data tutorial yet - I highly recommend it.

Hope that helps! :qiime2: :t_rex:

2 Likes

I'm sorry, that wasn't my intent at all! I was trying to highlight at a high-level what is going on here, keeping in mind that many other users (with a wide variety of backgrounds) might stumble across this thread in their own troubleshooting. So my apologies, that was not my aim, and I did not mean to offend you - I'm sorry!

I hope my reply above casts a little more light on the problem overall - if not, you know where to find us!

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.