Import error message

[Kim_Lam_Chiok]

Hello Qiime2:

I am super new to Qiime and went through the tutorials, Please be patient with me. I am trying to look at 16S amplicon sequencing data obtained from 10 different samples. These sequences were obtained by illumina platform, demultiplexed and primers trimmed (.gz format) by the core. Now, I created a manifest file containing the path for each sequence related to its sample and saved as .csv in a folder in my desktop. Here is a bit of the manifest:

|sample-id|absolute-filepath|direction|
|EHDP_1|$PWD/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz|forward|

This is what happens when I attempt to import these sequences:

"(qiime2-2018.6) Kims-iMac:emp-paired-end-sequences wetzelmirage$ qiime tools import \

--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path /Users/wetzelmirage/Desktop/Elk\ Hoof\ Metagenomics/pe-64-manifest.csv
--output-path paired-end-demux.qza
--source-format PairedEndFastqManifestPhred64
/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_types/per_sample_sequences/_transformer.py:344: UserWarning: Importing of PHRED 64 data is slow as it is converted internally to PHRED 33. Working with the imported data will not be slower than working with PHRED 33 data.
warnings.warn(_phred64_warning)
Traceback (most recent call last):
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2cli/tools.py", line 116, in import_data
view_type=source_format)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py", line 219, in import_data
return cls.from_view(type, view, view_type, provenance_capture)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/sdk/result.py", line 244, in _from_view
result = transformation(view)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/qiime2/core/transform.py", line 70, in transformation
new_view = transformer(view)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_types/per_sample_sequences/_transformer.py", line 346, in _9
single_end=False)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_types/per_sample_sequences/_transformer.py", line 268, in _fastq_manifest_helper
absolute=True)
File "/Users/wetzelmirage/miniconda3/envs/qiime2-2018.6/lib/python3.5/site-packages/q2_types/per_sample_sequences/_transformer.py", line 153, in _parse_and_validate_manifest
'%s' % path)
FileNotFoundError: A path specified in the manifest does not exist or is not accessible: /Users/wetzelmirage/qiime2-EHDP-2018/emp-paired-end-sequences/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz

An unexpected error has occurred:

A path specified in the manifest does not exist or is not accessible: /Users/wetzelmirage/qiime2-EHDP-2018/emp-paired-end-sequences/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz

See above for debug info."

I have a few questions:
Should I try to import the manifest to the directory? HOW? Is there a way to test manifest validity? Note that I created a directory and simply dragged the manifest to obtain its path. Is this an issue too? Do I need to unzip the fastq.gz and use the fastq versions instead?

Thank you for your support, it seems that importing data is one of the most painful tasks.

Kim

[thermokarst]

Hey there @Kim_Lam_Chiok!

@Kim_Lam_Chiok

I am super new to Qiime and went through the tutorials, Please be patient with me

Welcome, we are glad you are here!

@Kim_Lam_Chiok

Should I try to import the manifest to the directory? HOW? Is there a way to test manifest validity? Note that I created a directory and simply dragged the manifest to obtain its path. Is this an issue too? Do I need to unzip the fastq.gz and use the fastq versions instead?

These are all good questions! I will try and answer in one pass. The error message you saw is actually related to manifest validation:

@Kim_Lam_Chiok

A path specified in the manifest does not exist or is not accessible: /Users/wetzelmirage/qiime2-EHDP-2018/emp-paired-end-sequences/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz

Take a closer look at that, in particular, this part:

/Users/wetzelmirage/qiime2-EHDP-2018/emp-paired-end-sequences/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz

That path looks pretty crazy, and has big duplicate sections! The reason is because of this part of your manifest file:

$PWD/Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz

The manifest needs absolute filepaths --- /Users/wetzelmirage/Desktop/Elk\ Hook\ Metagenomics/2Read_primersTrimmed_splitBySample/EHDP_1_R1.fastq.gz is an absolute filepath on its own - you don't need the $PWD prefix! The $PWD env var expands to whatever your current working directory is - based on the error message, this looks like /Users/wetzelmirage/qiime2-EHDP-2018/emp-paired-end-sequences/.

Moving forward, just drop the $PWD and you should be cookin'!

Keep us posted!

[Kim_Lam_Chiok]

Hello,

It worked and I am whole today!