Importing demultiplexed fastq.gz files

import
(Sewunet Abera) #1

Dear Friends
I am newbie to qiime2. Now I am trying to import paired-end and already demultiplexed fastq.gz files from a sequencing company. However, I am having difficulty to import them into qiime2:
this is the command I run
(qiime2-2019.1) [email protected]:~/MyMicrobiome$ qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path manifest.csv
–output-path paired-end-demux.qza
–input-format PairedEndFastqManifestPhred64
And I’m getting this error
/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py:344: UserWarning: Importing of PHRED 64 data is slow as it is converted internally to PHRED 33. Working with the imported data will not be slower than working with PHRED 33 data.
warnings.warn(_phred64_warning)
Traceback (most recent call last):
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/tools.py”, line 146, in import_data
view_type=input_format)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 240, in import_data
return cls.from_view(type, view, view_type, provenance_capture)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 265, in _from_view
result = transformation(view)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/core/transform.py”, line 70, in transformation
new_view = transformer(view)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 346, in _9
single_end=False)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 268, in _fastq_manifest_helper
absolute=True)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 158, in _parse_and_validate_manifest
_validate_paired_end_fastq_manifest_directions(manifest)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 211, in _validate_paired_end_fastq_manifest_directions
‘“reverse”, but observed: %s’ % direction)
ValueError: Directions can only be “forward” or “reverse”, but observed: forward

An unexpected error has occurred:

Directions can only be “forward” or “reverse”, but observed: forward

See above for debug info.

Thanks

0 Likes

(Matthew Ryan Dillon) #2

Hi there @sewuneta! If you are trying to import paired-end data, there should be a forward and reverse file pairing for each sample — are there entries for each file in each sample present in your manifest? Also, unrelated, but do you really have Phred 64 offset data? That is super rare these days…

0 Likes

(Sewunet Abera) #3

Thanks a lot. I have paired-end data, 233 forward and 233 reverse sequences for each sample.
Here I have attached my manifest file for the record.manifest.csv (30.6 KB)

0 Likes

(Nicholas Bokulich) assigned thermokarst #4
0 Likes

(Sewunet Abera) #5

Dear Matthew, any suggestions?

0 Likes

(Matthew Ryan Dillon) #6

I don’t see anything wrong with this manifest - can you try again?

0 Likes

(Matthew Ryan Dillon) unassigned thermokarst #7
0 Likes

(Sewunet Abera) #8

Yes I did Matthew, yet now I’m getting a different error
(qiime2-2019.1) [email protected]:~/MyMicrobiome$ qiime tools import --type ‘SampleData[SequencesWithQuality]’ --input-path /home/sewunet/MyMicrobiome/manifest.csv --output-path /home/sewunet/MyMicrobiome/demux.qza --input-format PairedEndFastqManifestPhred33
Traceback (most recent call last):
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/tools.py”, line 146, in import_data
view_type=input_format)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 240, in import_data
return cls.from_view(type, view, view_type, provenance_capture)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 264, in _from_view
recorder=recorder)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/core/transform.py”, line 59, in make_transformation
(self._view_type, other._view_type))
Exception: No transformation from <class ‘q2_types.per_sample_sequences._format.PairedEndFastqManifestPhred33’> to <class ‘q2_types.per_sample_sequences._format.SingleLanePerSampleSingleEndFastqDirFmt’>

An unexpected error has occurred:

No transformation from <class ‘q2_types.per_sample_sequences._format.PairedEndFastqManifestPhred33’> to <class ‘q2_types.per_sample_sequences._format.SingleLanePerSampleSingleEndFastqDirFmt’>

See above for debug info.

0 Likes

(Nicholas Bokulich) assigned thermokarst #9
0 Likes

(Matthew Ryan Dillon) #10

Hi there @sewuneta!

The error is because you changed your --type:

It should be

Update the command and give it a shot! :qiime2:

1 Like

(Matthew Ryan Dillon) unassigned thermokarst #11
0 Likes

(Sewunet Abera) #12

Thanks in advance Matthew.
I did update the command but now I’m getting the earlier error.
I know I’m bothering you with all the nonsense but hope you won’t give up on me.
I spent months working on the tutorials but now I got my own data and couldn’t even import it.
(qiime2-2019.1) [email protected]:~/MyMicrobiome$ qiime tools import --type SampleData[PairedEndSequencesWithQuality] --input-path /home/sewunet/MyMicrobiome/manifest.csv --output-path /home/sewunet/MyMicrobiome/demux.qza --input-format PairedEndFastqManifestPhred33
Traceback (most recent call last):
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/tools.py”, line 146, in import_data
view_type=input_format)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 240, in import_data
return cls.from_view(type, view, view_type, provenance_capture)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/result.py”, line 265, in _from_view
result = transformation(view)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/core/transform.py”, line 70, in transformation
new_view = transformer(view)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 338, in _8
single_end=False)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 268, in _fastq_manifest_helper
absolute=True)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 158, in _parse_and_validate_manifest
_validate_paired_end_fastq_manifest_directions(manifest)
File “/home/sewunet/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py”, line 211, in _validate_paired_end_fastq_manifest_directions
‘“reverse”, but observed: %s’ % direction)
ValueError: Directions can only be “forward” or “reverse”, but observed: reverse

An unexpected error has occurred:

Directions can only be “forward” or “reverse”, but observed: reverse

See above for debug info.

0 Likes

(Matthew Ryan Dillon) #13

Hey there @sewuneta! I think I found the problem:

41%20PM

You have a tab after the word “reverse” on line 328 (reverse read for sample s94). Delete that and re-run and you should hopefully be good to go! :crossed_fingers:

2 Likes

(Sewunet Abera) #14

Yes Yes Mattew, it worked. It finally worked.
I already imported and checked the quality score.
It is perfect and no words to thank you!!

2 Likes

(system) closed #15

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

0 Likes