importing FASTQ file using a server

Technical Support
1

Guys,

I'm accessing a server from my computer using login and password, where Qiime2 was installed and my samples are.

So, I would like to import the samples using the "Casava 1.8 single-end demultiplexed fastq", because the sequences were generated using Ion Torrent platform, and have barcodes in sequence.

The initial command was:
(qiime2-2021.11) gecele@dourado:~/projects/Microbioma_solo_bromelias/drive3/Chip1$ qiime tools import --type 'SampleData[SequencesWithQuality]' --input-path ~/projects/Microbioma_solo_bromelias/drive3/Chip1/ --output-path brotest1.qza

However we had this error:
There was a problem importing /home/gecele/projects/Microbioma_solo_bromelias/drive3/Chip1/:
** Missing one or more files for SingleLanePerSampleSingleEndFastqDirFmt: '.+_.+_L[0-9][0-9][0-9]_R[12]_001\.fastq\.gz'**

First, can you help us understand what went wrong?

In this directory "/projects/Microbioma_solo_bromelias/drive3/Chip1/" we have 77 samples, but I wold like to start with 12, please could you guide us on how to do that?

Also, in the same directory there is a tsv metada file, is that can be the problem for data importing?

Thanks a lot!!
Gecele

2

Hi @Gecele_Matos_Paggi, this error:

As outlined here, the file names do not match the expected schema. As a fall-back, you can use the Manifest format to import your data.

You can use the Manifest format to only include the samples you'd like.

Ideally, only the sequence files should be present within the directory. This could explain the file naming error you are observing.

3

Thanks a lot @SoilRotifer!
I think I'm moving forward...
So, I decide to use the manifest format to import the data, using SingleEndFastqManifestPhred33V2.

Here is my manifest.tsv file:

single-end PHRED 33 fastq manifest file for forward reads

sample-id absolute-filepath
sample-BB12S_69 gecele/sample-BB12S_69_R1.fastq.gz
sample-BB20S_53 gecele/sample-BB20S_53_R1.fastq.gz
sample-BB24S_55 gecele/sample-BB24S_55_R1.fastq.gz
sample-BB4S_65 gecele/sample-BB4S_65_R1.fastq.gz
sample-BB8S_47 gecele/sample-BB8S_47_R1.fastq.gz
sample-BB16S_38 gecele/sample-BB16S_38_R1.fastq.gz
sample-DE7S_32 gecele/sample-DE7S_32_R1.fastq.gz
sample-DM11S_36 gecele/sample-DM11S_36_R1.fastq.gz
sample-DM15S_49 gecele/sample-DM15S_49_R1.fastq.gz
sample-DM19S_52 gecele/sample-DM19S_52_R1.fastq.gz
sample-DM23S_40 gecele/sample-DM23S_40_R1.fastq.gz
sample-DM3S_42 gecele/sample-DM3S_42_R1.fastq.gz

The command I used was:
(qiime2-2021.11) gecele@dourado:~/projects/Microbioma/drive3$ qiime tools import \

--type 'SampleData[SequencesWithQuality]'
--input-path manifestbro1.tsv
--output-path solodemux.qza
--input-format SingleEndFastqManifestPhred33V2

And I got this...
Traceback (most recent call last):
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 157, in import_data
artifact = qiime2.sdk.Artifact.import_data(type, input_path,
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/qiime2/sdk/result.py", line 277, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/qiime2/sdk/result.py", line 305, in _from_view
result = transformation(view, validate_level)
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/qiime2/core/transform.py", line 70, in transformation
new_view = transformer(view)
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2_types/per_sample_sequences/_transformer.py", line 219, in _23
return _fastq_manifest_helper_partial(old_fmt, _copy_with_compression,
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2_types/per_sample_sequences/_util.py", line 233, in _fastq_manifest_helper
input_manifest = _parse_and_validate_manifest(
File "/home/bio/software/miniconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2_types/per_sample_sequences/_util.py", line 108, in _parse_and_validate_manifest
raise ValueError('All paths provided in manifest must be '
ValueError: All paths provided in manifest must be absolute but found relative path: gecele/sample-BB12S_69_R1.fastq.gz

An unexpected error has occurred:
All paths provided in manifest must be absolute but found relative path: gecele/sample-BB12S_69_R1.fastq.gz
See above for debug info.

So, apparently, I have to adjust the filepath in my manifest file...
But how???

Thanks

4

Just go to that line in your manifest file and replace that relative file path with the full absolute path. One way to get the full directory path is to go inside the relevant directory and type pwd. This will print the full working directory. Then you can append this path to the file name within your manifest file.

For example, let's say that the data is in the directory: gecele/

I can change to this directory: cd gecele/.
Then I can type pwd, which will return the full absolute path:
/home/soilrotifer/gecele/

So in my manifest file I can add:
/home/soilrotifer/gecele/sample-BB12S_69_R1.fastq.gz

1 Like
5

@SoilRotifer !!!

It worked!!

Thanks a lot!

Gecele

2 Likes
6

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.