It has been some time but I've finally had the chance to look at the project I'd been planning on doing, which you previously advised me on.
I now have the raw multiplexed and mapping files (see below), and I've tried to run the data through deblur by following the steps outline in the Moving Pictures Tutorial—however, I've come across a few issues.
First, when I try to import, this is the issue I am getting:
(qiime2) taylorrayne@Taylors-Air qiime2-moving-pictures-tutorial % qiime tools import \
--type EMPSingleEndSequences \
--input-path trail1stock \
--output-path trail1stock-seq.qza
There was a problem importing trail1stock:
Missing one or more files for EMPSingleEndDirFmt: 'sequences.fastq.gz'
Do you know how I can instruct EMP to accept my data file?
Also, when I am to demultiplex my data with demux , how do I have this command read my files properly. My sequences are in .fastq format and my mapping file is just a .txt and looks like this:
I made this public partially because I've been super busy recently and not on the forum as much; I apologize.
It looks like the error is that your sequences need to be named sequences.fastq.gz and they might be called sequences.fastq. Yes, it's a weird little nuaunce. And yes, I think we've all done it 1000 times.
Unfortunately, our general tutorials can't each provide information about importing all kinds of data. The EMP data imports is really just a "convenience method" that makes bringing a particular kind of data(that is, data that has been collected, sequenced, and processed so far in accordance with the Earth Microbiome Protocol) into QIIME 2. However if your data in anyway does not fulfill the expectations baked into the EMP import, using this method will not end up being more convenient
All convenience importing methods are just pre-parameterized calls to a "manifest import", which you can do directly yourself! See the documentation on how to do so here! The manifest imports let you tell QIIME 2 exactly where your files are and which file represents which piece of data.
If your data was sequenced even remotely recently, the --input-format you will want to import as is very likely to be PairedEndFastqManifestPhred33V2. Note that you need to be very precise in specifying file locations in your manifest document that you prepare, though you can use environment variables such as $HOME to indicate where your files are.
The good news is that at least in regards to using the QIIME 2 tools, importing your data is often the most difficult part of the process!
Do either of you know what might be the issue here and how to resolve it?
I understand that my folder se-33-manifest is evidently not a file; but I will need to somehow specify an input path that accepts all my .fastq files.
You're on the right track! But the error message you are getting is exactly correct (as you mentioned) - se-33-manifest is not a file, it's a directory. You'll want to create an actual file that contains the filepath information for all of your .fastq files - something that looks like this:
I am not sure what the issue might be...it seems like I have the correct file path.. when I look at the file directly it shows that this is correct /Users/taylorrayne/qiime2-moving-pictures-tutorial/demultiplexed_data
$PWD will give the path to the location from which you are calling for this path, try using $HOME instead, if you are going to give an absolute path like that, if you wanted to keep it relative you could instead do something like $PWD/demultiplexed_data/blah...blah.fastq, if you were calling your import from inside the directory containing your demultiplexed folder and your manifest file.
Hello @Keegan-Evans ! Thanks for following up with this suggestion. I went ahead and tried running the command with $HOME instead but still got the same issue. I'll include details below; let me know if you see anything immediately wrong or not what you were suggesting in what I have done!
The path you're using is redundant. $HOME is the same location as /Users/taylorrpaye/. So, you either need to use $HOME or /Users/taylorrpaye/ at the begining, not both. On my mac, I find better performance if I dont use local variables, like $HOME and instead stick with the full path (In your case /Users/taylorrpaye/...
As a PS, do you know that if you drag a file into the command line on a mac, it will show you the path? So, if you're not sure how the computer knows the address, open up a terminal, and drop it in! It's saved me a few times when I was lost.
Hello @jwdebelius - thank you for this suggestion with the drop-and-drag trick - super helpful!
I have now resolved the path issue by using /Users ... instead.
However, I am now finding an issue with the files themselves, and I am wondering if I can get your insights on .fastq files.
And here is a view of my files. As far as I can see, they conform to the .fastq formatting, so it is not clear what the issue might be...let me know if anything is obvious to you!
Hello! I am back again!
...and with more errors - however, I was able to resolve my above issue with the file alignment ... turns out that the > in my header was throwing things off.
Anyhow, I have now made it to the last step with the deblur processing! My error now is this:
I have been searching online as to how to solve a max() arg error; however, nothing seems to be specific to deblur and I am not sure what I can change to resolve this...it seems like something coming from functions more internal to deblur itself.
Let me know if you can help!
p.s also, let me know if you have expirience interpreting the deblur log file:
@Taylorr,
Could you DM me that log file? To be honest, I am not sure exactly where that error is coming from, but hopefully if I can take a look at the full log file I will have a better idea of the direction to point you in.
