data won't import

Hello,

I'm trying to import 102 standard, paired end fastq.gz files into the latest version in a MacOS environment. I've created my manifest file and I'm using the code below:

qiime tools import \

--type 'SampleData[SequencesWithQuality]’ \

--input-path manifest.tsv \

--output-path paired-end-demux.qza \

--input-format PairedEndFastqManifestPhred33V2

I get no error messages, everything appears to be running smoothly, but it never finishes. I've stopped and restarted is several times, let it run over night but nothing. When I check the activity monitor on my computer it appears that terminal is not using any CPUs either, regardless of if I just started the process or it's been several hours.

I re-ran some steps on an already analyzed dataset to make sure other parts of qiime2 were working and that ran perfectly fine.

I'm at a complete loss here.

1 Like

Hello Jennifer,

That's strange! :thinking:

Spooky! :ghost:

I'm not sure what's causing this either, but here's some things to try:

qiime tools import \
--type 'SampleData[SequencesWithQuality]’ \
    Hey, take a look a this curly quote ^
    Try replacing ’ with ' 

Do you have any empty fastq files in manifest.tsv?

What happens if you try to import a single pair of fastq files using a new manifest.tsv file?

It's a mystery! Let us know what you find! :female_detective:
Colin

1 Like

Thanks for your help Colin. I replaced the curly quote and got an error message, which is progress, I guess?

None of the fastq files are empty and my manifest file is in the correct format.

Here is the new error message:

Traceback (most recent call last):
File "/Users/jenniferbell/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 156, in import_data
artifact = qiime2.sdk.Artifact.import_data(type, input_path,
File "/Users/jenniferbell/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/result.py", line 277, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/Users/jenniferbell/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/result.py", line 303, in _from_view
transformation = from_type.make_transformation(to_type,
File "/Users/jenniferbell/miniconda3/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/core/transform.py", line 58, in make_transformation
raise Exception("No transformation from %r to %r" %
Exception: No transformation from <class 'q2_types.per_sample_sequences._format.PairedEndFastqManifestPhred33V2'> to <class 'q2_types.per_sample_sequences._format.SingleLanePerSampleSingleEndFastqDirFmt'>

An unexpected error has occurred:

No transformation from <class 'q2_types.per_sample_sequences._format.PairedEndFastqManifestPhred33V2'> to <class 'q2_types.per_sample_sequences._format.SingleLanePerSampleSingleEndFastqDirFmt'>

See above for debug info.

1 Like

I'm glad we made some progress!

EDIT: Looks like it's a mismatch between type and input-format. Whoops!

Looks like the new error is related to transformers. Have you added any to your environment?

Could you try running this in a freshly installed Qiime2 conda environment? Maybe that would help. If not, we could call in thermokarst for advice.

I updated my conda environment and installed the newest version of qiime2 three days ago. Do you think I should uninstall and re-install? There were no installation errors.

Thanks!

1 Like

Oh I was looking at the wrong part of the error!

Here's the issue:
--type 'SampleData[SequencesWithQuality]' is for single-end reads, however you are listing the input format as PairedEndFastqManifestPhred33V2 which is for paired-end reads.

Check out the examples in the importing docs, and try something like

qiime tools import \
  --type 'SampleData[PairedEndSequencesWithQuality]' \
  --input-format PairedEndFastqManifestPhred33V2 \
  ...

so the type and input-format match.

Sorry for any confusion!

Oh good lord, I knew it was going to be something stupidly simple :woman_facepalming:

That, of course, worked and everything is happily imported.

2 Likes

:man_technologist: :woman_technologist:

:no_entry_sign: :ghost:

2 Likes