No transformation from <class import data manifest

Rodrigo_Andres_Echev · July 31, 2023, 7:07pm

Hello, I have a problem when loading my data from 16s that are already demultiplexed to qiime2 using the manifest 64 or 33.

the format of my manifest is as follows:

sample-id,absolute-filepath,direction
1,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/1_1.fastq,forward
1,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/1_2.fastq,reverse
2,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/2_1.fastq,forward
2,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/2_2.fastq,reverse
3,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/3_1.fastq,forward
3,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/3_2.fastq,reverse
4,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/4_1.fastq,forward
4,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/4_2.fastq,reverse
5,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/5_1.fastq,forward
5,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/5_2.fastq,reverse
6,/mnt/c/Users/Proyecto 2018208/Desktop/BGI_Result1/BGI_Result/CleanData/6_1.fastq,forward

the command i am using is:

qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path ../../manifest.csv
--output-path Paired-end-demux.qza
--input-format SingleEndFastqManifestPhred33

The error I get is:

Traceback (most recent call last):
File "/home/gallego/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/builtin/tools.py", line 158, in import_data
view_type=input_format)
File "/home/gallego/miniconda3/envs/qiime2/lib/python3.6/site-packages/qiime2/sdk/result.py", line 241, in import_data
validate_level='max')
File "/home/gallego/miniconda3/envs/qiime2/lib/python3.6/site-packages/qiime2/sdk/result.py", line 266, in _from_view
recorder=recorder)
File "/home/gallego/miniconda3/envs/qiime2/lib/python3.6/site-packages/qiime2/core/transform.py", line 59, in make_transformation
(self._view_type, other._view_type))
Exception: No transformation from <class 'q2_types.per_sample_sequences._format.SingleEndFastqManifestPhred33V2'> to <class 'q2_types.per_sample_sequences._format.SingleLanePerSamplePairedEndFastqDirFmt'>

An unexpected error has occurred:

No transformation from <class 'q2_types.per_sample_sequences._format.SingleEndFastqManifestPhred33V2'> to <class 'q2_types.per_sample_sequences._format.SingleLanePerSamplePairedEndFastqDirFmt'>

See above for debug info.

It doesn't even work for me with the guide files

--source-format command doesn't work, I used --input-format

Help me please

lizgehret · July 31, 2023, 7:19pm

Hi @Rodrigo_Andres_Echev,

Welcome to the :qiime2: forum!

The problem lies in the --input-format that you've specified. Based on all of the details you've provided, it looks like you're working with paired end sequences - but you've set the input format to SingleEndFastqManifestPhred33. You'll need to update that to either PairedEndFastqManifestPhred33V2 or PairedEndFastqManifestPhred64V2 (depending on what your PHRED offset was).

More details on these formats (and general importing instructions) can be found here, for your reference:

https://docs.qiime2.org/2023.5/tutorials/importing/#pairedendfastqmanifestphred33v2

Hope this helps! Cheers

Rodrigo_Andres_Echev · August 1, 2023, 5:34pm

Thank you very much, it worked, formatting the manifest to 642v2, however now it throws me the following error. Sorry I'm a bit new to this

/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_types/per_sample_sequences/_transformer.py:244: UserWarning: Importing of PHRED 64 data is slow as it is converted internally to PHRED 33. Working with the imported data will not be slower than working with PHRED 33 data.
warnings.warn(_phred64_warning)
Traceback (most recent call last):
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 266, in import_data
artifact = qiime2.sdk.Artifact.import_data(type, input_path,
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/result.py", line 327, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/result.py", line 355, in _from_view
result = transformation(view, validate_level)
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/core/transform.py", line 70, in transformation
new_view = transformer(view)
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_types/per_sample_sequences/_transformer.py", line 246, in _26
return _fastq_manifest_helper_partial(old_fmt, _write_phred64_to_phred33,
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_types/per_sample_sequences/_util.py", line 269, in _fastq_manifest_helper
fastq_copy_fn(input_fastq_fp, str(output_fastq_fp))
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_types/per_sample_sequences/_util.py", line 293, in _write_phred64_to_phred33
for seq in skbio.io.read(phred64_fh, format='fastq',
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/registry.py", line 1160, in read
return io_registry.read(file, format=format, into=into, verify=verify,
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/registry.py", line 506, in read
return (x for x in itertools.chain([next(gen)], gen))
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/registry.py", line 531, in _read_gen
yield from reader(file, **kwargs)
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/registry.py", line 1008, in wrapped_reader
yield from reader_function(fhs[-1], **kwargs)
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/format/fastq.py", line 351, in _fastq_to_generator
phred_scores, seq_header = _parse_quality_scores(fh, len(seq),
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/format/fastq.py", line 522, in _parse_quality_scores
_decode_qual_to_phred(chunk, variant=variant,
File "/home/gallegobq/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/io/format/_base.py", line 34, in _decode_qual_to_phred
raise ValueError("Decoded Phred score is out of range [%d, %d]."
ValueError: Decoded Phred score is out of range [0, 62].

An unexpected error has occurred:

Decoded Phred score is out of range [0, 62].

See above for debug info.

lizgehret · August 1, 2023, 5:57pm

Hi @Rodrigo_Andres_Echev,

Thanks for sharing this traceback!

This means your data is most likely Phred33 instead of Phred64 - I'd try re-importing using the following format:

PairedEndFastqManifestPhred33V2

Cheers

Rodrigo_Andres_Echev · August 2, 2023, 12:33am

Thank you very much and I was able to continue with the denoising, very grateful

system · September 2, 2023, 6:34am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.