Trying to use my own provided data

Good day all,
Please I need help, I have done many tutorials especially moving pictures usage and Atacama soil Microbiome project.
Now iv been trying to use my own provided data I keep getting a errors.
I attached a picture of my data. Please how do I stop this errors.

Hi @Louis :wave:

Are these still multiplexed? Looks to me like you have two samples in your directory with a forward and reverse read each.

What errors are you getting from Qiime2? If you can copy and paste the errors you are recieving it will help everyone on the forum to diagnose the issue.

all the best,

Vic

4 Likes

Thank you for your reply.
Yes, I had two samples but I have moved one sample to a different directory now Am left with the forwarded and reverse read for only one sample (1Les_S154_R1_001.fastq & 1Les_S154_R2_001.fastq).
Am only able to import them with :
Qiime2 tool import
--type MultiplexedPairedEndBarcodeISequence
--input-path ezediuno
--output-path multiplexed-seq.qza

Note input-path muxed-pe-barcode-in-seq\ was replaced with Ezediuno which is my directory.

And I got the zip file multiplexed-seqs.qza.

After this I can't go further such as demultiplexing the sequence reads using:

qiime demux MultiplexedPairedEndBarcodeISequence
--m-barcodes-file sample-metadata.tsv
--m-barcodes-column barcode-sequence
--p-rev-comp-mapping-barcodes
--i-seqs MultiplexedPairedEndBarcodeISequence.qza
--o-per-sample-sequences demux-full.qza
--o-error-correction-details demux-details.qza

Iv also tried to skip demultiplexing but .

Please how do I continue.

1 Like

Hi again @Louis , :wave:

You likely already know this but I just want to double check :slightly_smiling_face:. If those fastq you have (the forward R1 and reverse R2) belong to ONE sample, then they do not need demultiplexing.
Multiplexed data is when your fastq files contain many samples need separating out using barcodes that are used to tag each sample during library prep :label:

The two errors you have look like you've gotten are becasue you have written "qiime multiplexed-seq.qza" or "qiime ezediuno" niether of these are qiime 2 commands. One is your artifact from the import step and one is a user name. So I think these are just typos. The demultiplex command is "qiime demux emp-paired ".

I would have a good read of the descriptions throughout the tutorials, as they do a really good job of explaining why you do each step and what you might need to consider about your own data.

best,

Vic :smiley:

4 Likes

Thank you so much for assisting me. will try again... Until I finally understand how to use Qiime2.

Good evening jphagen,
Please forgive my intrusion, but am desperately in need of help.
I have gone through the tutorial on moving picture and Atacama soil Microbiome project and was able to complete every step now I have been trying to used my own samples but constantly fail. The sequence I have doesn't come with a separate barcode zip file nor did it come with a metadata document. It's a metagenomics sample (DNA sequences from poultry droppings) (most say am new to all of this, so I may be a little confused at times).
My challenge is that I can go to the point of Qiime tool import to get a qza file after that i get stocked. Now from the tutorial the nxt step for a multiplex sample should be demultiplexing but I keep getting an error message (I attached a screenshot of the error message). I've tried jumping the step to trim, but I still get error. I can't do other analysis because I don't have the files needed for them, please help as I really need Qiime for my project work.
Thank you sir.

Hi again @Louis,

I know it's difficult when you are struggling, however, it doesn't appear you have tried anything different from the last suggestions. The issue you are having is you have typed "qiime multiplexed-seq.qza" or "qiime ezediuno" niether of these are qiime 2 commands.

You need to type qiime demux emp-paired i.e.:

qiime demux emp-paired \
  --m-barcodes-file sample-metadata.tsv \
  --m-barcodes-column barcode-sequence \
  --p-rev-comp-mapping-barcodes \
  --i-seqs emp-paired-end-sequences.qza \
  --o-per-sample-sequences demux-full.qza \
  --o-error-correction-details demux-details.qza

The tutorial is here

6 Likes

Thank you for trying to help me, am very grateful.
I am still trying, this time not changing any line of command.

1 Like

Good morning house,
It's me again; Still struggling.
Maybe I have been presenting my problem wrongly.
The thing is that I think I have a MULTIPLEXED PAIRED-END BARCODES IN SEQUENCE READ because I can only import my reads using the import command:
qiime tools import
--type MultiplexedPairedEndBarcodeISequence
--input-path muxed-pe-barcode-in-seq
--output-path multiplexed-seq.qza
With that command I get the file: multiplexed-seqs.qza.
NOW, HOW DO I CARRY OUT DEMULTIPLEXING AS NON OF THE COMMAND (FROM ATACAMA, OR MOVING PICTURES Tutorial) worked.
Thank you... And sorry for all the troubles.
Thank you.

Hi again @Louis :wave: :smiley:

Now you have imported your data - what errors are you now getting when you type the above recommened command? As your errors before seemed to be the result of typos.

I've popped some comments to make it more explicit:

qiime demux emp-paired
--m-barcodes-file sample-metadata.tsv \ # this is the file with your barcodes in
--m-barcodes-column barcode-sequence \ #this tells the command what column to look at
--p-rev-comp-mapping-barcodes
--i-seqs multiplexed-seqs.qza \ #your file input you created in the last step
--o-per-sample-sequences demux-full.qza \ #outfile
--o-error-correction-details demux-details.qza #outfile

Another option is to try the cutadapt plug in, instruction for this are here:

qiime cutadapt demux-paired
--i-seqs multiplexed-seqs.qza
--m-forward-barcodes-file md.tsv
--m-forward-barcodes-column barcode-sequence
--o-per-sample-sequences per-sample-sequences.qza
--o-untrimmed-sequences untrimmed-sequences.qza

best,

Vic

2 Likes

Thanks for trying to help me.
Iv tried everything as much as I can understand and it's still giving error message.
I give up!, I think I should just look for another program to do my analysis as am running out of time.
Thank you again.

Hi @Louis,

I am so sorry that this has been a frustrating experience! debugging errors while analysis is definitely a difficult part of data analysis

Can you post the error that you got from running @buzic Commands?

We not really help you unless you provide the errors so that we can debug.

Please provide a screenshot of the command and the errors so that we can get a better idea of whats going on?

Good morning Buzic,
Thank you for giving me this opportunity again to be assisted.

From Qiime2 importing data tutorial there are many ways to import data into qiime2, this include:

Sequence data with sequence quality information.

-"EMP protocol" multiplexed single-end fastq
Import data created is:
emp-single-end-sequences.qza.

-"EMP protocol" multiplexed paired-end fastq
Import data created is:
emp-paired-end-sequences.qza

  • Multiplexed single-end FASTQ with barcode in sequence
    Import data created is:
    Multiplexed-seqs.qza

  • Multiplexed paired-end FASTQ with barcode in sequence
    Import data created is:
    Multiplexed-seq.qza

-Casava 1.8 single-end demultiplexed fastq
Import data created is:
demux-single-end.qza

-Casava 1.8 paired-end demultiplexed fastq
Import data created is:
demux-paired-end.qza

And others.
My data fit into:

  • Multiplexed paired-end FASTQ with barcode in sequence
    Import data created is:
    Multiplexed-seq.qza

My challenge is how do I proceed with the file multiplexed-seq.qza.

From the tutorial on moving pictures and Atacama soil Microbiome
Which contain the rest of the downstream analysis such as demultiplexing, denoising and other statistical and bioinformatics analysis, demultiplexing required EMP files:

qiime demux emp-single
--i-seq emp-single-end-sequences.qza
--m-barcodes-file sample-metadata.tsv
--m-barcodes-column barcode-sequence
--o-per-sample-sequences demux-full.qza
--o-error-correction-details demux-details.qza

qiime demux emp-paired
--m-barcodes-file sample-metadata.tsv
--m-barcodes-column barcode-sequence
--p-rev-com-mapping-barcodes
--i-seq emp-paired-end-sequences.qza
--o-per-sample-sequences demux-full.qza
--o-error-correction-details demux-details.qza

None of these commands works for the file
multiplexed-seqs.qza
Or for the file
demux-single-end.qza
demux-paired-end.qza.

How do I proceed with the file "multiplexed-seq.qza"?

Thank you for your patience with me, am very grateful.

Hi again @Louis,

If you are having on going issues it's best to just post in the forum thread you created rather than private message seperate issues. This way a group of people can figure out what the issue is rather than just one. So I've posted it here.

When you say this:

What errors are you getting? Post a screen shot or the code fo the error else it is very difficult for us to know what the issue is or how to fix it.

best,

Vic

1 Like


This is the error am getting.
This is because the imported file I can generate is the "multiplexed-seqs.qza because the data is a multiplexed paired-end FASTQ with barcode in sequence.

How do I proceed to carry other analysis like demultiplexing and others after generating a multiplexed-seq.qza file from a multiplexed paired-end FASTQ with barcodes in sequence.

Hi @Louis

Thats much more helpful, thank you! :slightly_smiling_face:

The error is telling you that the file you use for the input --m-barcodes-file called sample-metadata.tsv does not exsist or qiime2 cannot find it.

The solution is to make sure the file sample-metadata.tsv is in the directory/folder you are running the command from, and if it's not make sure you tell qiime2 where it is.

best,

Vic

Hi again @Louis,

I think there might be a further issue here. You imported your data and saved it as multiplexed-seqs.qza but then try the next step using the tutorial name of paired-end-sequences.qza. That won't work. Specifically, your data is not called paired-end-sequences.qza, your data is called multiplexed-seqs.qza. The tutorials are a guide for you to see how data is processed, but try it's vital to pay attention the name differences between the tutorial and your own data.

I also just want to check that you are creating your own metadata files and barcode metadata files for your own data. You must make a file called sample-metadata.tsv, it must contain the per-sample barcodes used in your study.

There was an example command that I placed up here (anything after the # sign on each line is a description):

The comments say "your barcodes" and "your file input you created".

I hope that helps.

all the best,

Vic

2 Likes

The example i am using only have a forward and reverse sequence data, there is no barcode file or metadata file as shown on the picture. This is the same kind of data I got from where I carried out sequencing .
The example on Qiime2 which I am trying to use doesn't come with a metadata nor a barcode file, just the forward and reverse file. With that I can generate a multiplexed-seqs.qza file.
Am guessing I can't continue unless I provide the metadata file or/and barcode files which of cause the example multiplexed paired-end FASTQ with barcodes in sequence doesn't provide any.
Thank you.

1 Like

@Louis

I think that there might be a reason for this. If you have a forward and reverse read for exach sample I think the reason you don't have barcodes for your samples is that your data might not be multiplexed, it might be demultiplexed.

I asked this earlier:

Do you have an R1 and an R2 per sample? Is there a reason you think you have mutiplexed data?

Yes you absolutley can continue, you import in a different way, but first we need to figure out what type of data you have.

Your own data wouldn't necessarily come with metadata files, you have to make this. You should know what barcodes were used with your own data, especially if receiving the data in multiplexed format.

4 Likes

Hello, good morning,
I want to use this opportunity to really appreciate all of you that have tried to help me, you are awesome and you have such wonderful nature, i don't know if i myself can be that patient with me and all my questions.
Thank you and am very grateful.
I have a tiny suggestion and am hoping you will consider it, especially for greenhorn like myself who is very very new in molecular biology and a novice in DNA analysis.
My suggestion is, could it be possible you hold a kind of online workshop so that we could learn directly from you guys... So that we can ask questions directly and get answers directly. I feel this would really help us.
Please consider my suggestion, I and am sure many others will be very grateful if this happens.
Thank you so much again for all you have done for me and thank you in anticipation of responding positively to my humble suggestion.

1 Like