# Importing fastq files after demultiplexing
**URL:** https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394
**Category:** User Support
**Created:** [April 26, 2019, 12:30pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394 "2019-04-26T12:30:15Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 26, 2019, 12:30pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/1 "2019-04-26T12:30:15Z")
Hi,
I have got fastq.gz samples from my sequencing provider after spilt libraries and demultuplexing.
In this run, I had 6 samples and he sent me 6 forward file and 6 reverse files separately.
I used to work with Qiime 1.... a lot and now I wish to try Qiime 2.
I have several Q:
1. Do I need to combine all the 6 forward files into one combined file and all the 6 reverse files into another file in order to start the analysis?
2. I tried to understand from the tutorial how should I upload the files but I couldn't, should I use the : “Fastq manifest” formats" " step?
Thanks a lot,
Nir
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [April 26, 2019, 1:52pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/2 "2019-04-26T13:52:29Z")
> [@Nir\_Friedman](#):
>
> - Do I need to combine all the 6 forward files into one combined file and all the 6 reverse files into another file in order to start the analysis?
No! If your reads are already demultiplexed you can use the manifest format to import these.
However, you mention "split libraries"... if your sequencing provider is running the qiime 1 split libraries script or performing any other kind of quality control on the sequences, you should ask for the raw data.
Good luck!
---
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 26, 2019, 2:30pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/3 "2019-04-26T14:30:35Z")
Thanks Nicholas.
My seq provider did not use qiime for sure, he uses Qiagen's CLC.
According to him the used split libraries ( maybe he mistakenly used this term, when he actually meant for demultiplexing the samples).
The output that I have got is 12 files, 6 forward (R1) , 6 revers (R2) fastq.gz files.
Now, should I use the manifest format?
And if I do, just to be sure: I copy to the same directory both the fastq files and the manifest file and I am using this script:
qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path pe-64-manifest
--output-path paired-end-demux.qza
--input-format PairedEndFastqManifestPhred64
Thanks again,
Nir
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [April 26, 2019, 2:32pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/4 "2019-04-26T14:32:57Z")
> [@Nir\_Friedman](#):
>
> My seq provider did not use qiime for sure, he uses Qiagen’s CLC.
> According to him the used split libraries ( maybe he mistakenly used this term, when he actually meant for demultiplexing the samples).
Thanks for clarifying! That's good — some sequence providers use qiime1 scripts or other commands that demultiplex + QC. We recommend not running any QC, since this can conflict with later steps in QIIME 2 (e.g., running dada2).
> [@Nir\_Friedman](#):
>
> Now, should I use the manifest format?
Yes, unless if your filenames fit the Casava format described in the tutorial. If in doubt, always use the manifest format.
> [@Nir\_Friedman](#):
>
> And if I do, just to be sure: I copy to the same directory both the fastq files and the manifest file and I am using this script:
For manifest format the files can be anywhere. You specify the file path in the manifest. But that command looks correct.
Good luck!
---
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 26, 2019, 2:41pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/5 "2019-04-26T14:41:42Z")
Excellent.
Thanks a lot.
Will try it and get back to you if needed.
Nir
---
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 29, 2019, 11:57am UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/6 "2019-04-29T11:57:43Z")
Hi Nicholas,
Your answer was very helpful and i manged to upload the files.
I then ran this command to clean non biological sequences.
qiime dada2 denoise-paired
--i-demultiplexed-seqs /home/qiime2/Desktop/Netanya\_6/paired-end-demux.qza
--p-trim-left-f 0
--p-trim-left-r 0
--p-trunc-len-f 240
--p-trunc-len-r 240
--p-n-threads 0
--o-representative-sequences 6\_electra\_rep-seqs.qza
--o-table 6\_electra\_table.qza
--o-denoising-stats 6\_electra\_dada2.qza
--verbose
I got this error:
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run\_dada\_paired.R /tmp/tmpv2rg0v6\_/forward /tmp/tmpv2rg0v6\_/reverse /tmp/tmpv2rg0v6\_/output.tsv.biom /tmp/tmpv2rg0v6\_/track.tsv /tmp/tmpv2rg0v6\_/filt\_f /tmp/tmpv2rg0v6\_/filt\_r 240 240 0 0 2.0 2 consensus 1.0 0 1000000
R version 3.4.1 (2017-06-30)
Loading required package: Rcpp
DADA2 R package version: 1.6.0
1. Filtering ......
2. Learning Error Rates
2a) Forward Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 125909 reads in 48728 unique sequences.
Sample 2 - 124740 reads in 45881 unique sequences.
Sample 3 - 112445 reads in 48095 unique sequences.
Sample 4 - 134674 reads in 55631 unique sequences.
Sample 5 - 113621 reads in 43446 unique sequences.
Sample 6 - 135461 reads in 57261 unique sequences.
selfConsist step 2
Error in dada\_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), :
Memory allocation failed.
Calls: dada -\> dada\_uniques -\> .Call
Execution halted
Warning message:
system call failed: Cannot allocate memory
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2\_dada2/_denoise.py", line 231, in denoise\_paired
run\_commands([cmd])
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2\_dada2/denoise.py", line 36, in run\_commands
subprocess.run(cmd, check=True)
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run\_dada\_paired.R', '/tmp/tmpv2rg0v6/forward', '/tmp/tmpv2rg0v6_/reverse', '/tmp/tmpv2rg0v6\_/output.tsv.biom', '/tmp/tmpv2rg0v6\_/track.tsv', '/tmp/tmpv2rg0v6\_/filt\_f', '/tmp/tmpv2rg0v6\_/filt\_r', '240', '240', '0', '0', '2.0', '2', 'consensus', '1.0', '0', '1000000']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/commands.py", line 274, in **call**
results = action(\*\*arguments)
File "\", line 2, in denoise\_paired
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound\_callable
output\_types, provenance)
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py", line 365, in _callable\_executor_
output\_views = self.\_callable(\*\*view\_args)
File "/home/qiime2/miniconda/envs/qiime2-2019.1/lib/python3.6/site-packages/q2\_dada2/\_denoise.py", line 246, in denoise\_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
See above for debug info.
What does it means and what should I do to fix it?
Thanks a lot,
Nir
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [April 29, 2019, 12:01pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/7 "2019-04-29T12:01:37Z")
Hi @Nir_Friedman,
Here is the key line:
> [@Nir\_Friedman](#):
>
> Memory allocation failed.
You do not have enough memory to run this command! You will probably need to find a more powerful machine to run this (usually 8 GB RAM should suffice for a dada2 with a single sequencing run, but the more the merrier). See these forum posts for more troubleshooting advice: [Search results for 'dada2 'Memory allocation failed'' - QIIME 2 Forum](https://forum.qiime2.org/search?q=dada2%20%27Memory%20allocation%20failed%27)
---
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 29, 2019, 12:07pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/8 "2019-04-29T12:07:40Z")
Hi Nicholas,
I have designated 4 GB RAM and 3 CPUs ( my laptop has 32 GB RAM and 8 CPUs)
Should I increase only the RAM or also the CPUs?
Was this the only problem? ( just to rerun the command?)
Tx
Nir
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [April 29, 2019, 12:13pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/9 "2019-04-29T12:13:33Z")
> [@Nir\_Friedman](#):
>
> I have designated 4 GB RAM and 3 CPUs ( my laptop has 32 GB RAM and 8 CPUs)
> Should I increase only the RAM or also the CPUs?
Increase RAM. No need to increase CPUs unless if you want to.
> [@Nir\_Friedman](#):
>
> Was this the only problem? ( just to rerun the command?)
It's the only problem reported in that error message 😄
Good luck!
---
### Author:  [@Nir\_Friedman](https://forum.qiime2.org/u/Nir_Friedman)
#### Post date: [April 29, 2019, 1:15pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/10 "2019-04-29T13:15:35Z")
Great...
Thank a lot,
Nir
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [April 30, 2019, 1:44pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/11 "2019-04-30T13:44:18Z")
An off-topic reply has been split into a new topic: [how do I import reference sequences for feature classification?](https://forum.qiime2.org/t/how-do-i-import-reference-sequences-for-feature-classification/9443)
Please keep replies on-topic in the future.
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [May 31, 2019, 7:48pm UTC](https://forum.qiime2.org/t/importing-fastq-files-after-demultiplexing/9394/12 "2019-05-31T19:48:59Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.