Plugin error from deblur:max() arg is an empty sequence

Sowmiya_Kathir · May 29, 2024, 12:29pm

Hello everyone,
I am facing some issues with qiime deblur denoise-16. My data is sequenced using NextSeq5000 with read length 300 bp. I am using qiime2023.7 version. I have shared the command, error and input data. Please help me with it.

Command:
"qiime deblur denoise-16S --i-demultiplexed-seqs paired-end-demux.qza --p-trim-length 220 p-sample-stats --output-dir deblurresults"

error:
INFO(140094201935680)2024-05-29 09:27:48,801:*************************
INFO(140094201935680)2024-05-29 09:27:48,801:deblurring started
WARNING(140094201935680)2024-05-29 09:27:48,801:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm
WARNING(140094201935680)2024-05-29 09:27:48,802:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(140094201935680)2024-05-29 09:27:48,802:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140094201935680)2024-05-29 09:27:48,802:deblur main program started
INFO(140094201935680)2024-05-29 09:27:48,802:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm
INFO(140094201935680)2024-05-29 09:27:48,803:building negative db sortmerna index files
INFO(140094201935680)2024-05-29 09:27:48,803:build_index_sortmerna files ['/home/oneomics2/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140094201935680)2024-05-29 09:27:48,939:building positive db sortmerna index files
INFO(140094201935680)2024-05-29 09:27:48,939:build_index_sortmerna files ['/home/oneomics2/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140094201935680)2024-05-29 09:28:45,600:parallel processing per sample fasta files
INFO(140094201935680)2024-05-29 09:28:45,601:parallel deblur started for 2 inputs
INFO(140315021563712)2024-05-29 09:28:46,906:*************************
INFO(140315021563712)2024-05-29 09:28:46,906:deblurring started
WARNING(140315021563712)2024-05-29 09:28:46,907:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
WARNING(140315021563712)2024-05-29 09:28:46,907:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140315021563712)2024-05-29 09:28:46,907:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140315021563712)2024-05-29 09:28:46,907:deblur worker thread started
INFO(140315021563712)2024-05-29 09:28:46,907:processing per sample fasta files
INFO(140315021563712)2024-05-29 09:28:46,907:--------------------------------------------------------
INFO(140315021563712)2024-05-29 09:28:46,907:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:28:47,017:*************************
INFO(140018238961472)2024-05-29 09:28:47,017:deblurring started
WARNING(140018238961472)2024-05-29 09:28:47,018:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
WARNING(140018238961472)2024-05-29 09:28:47,018:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140315021563712)2024-05-29 09:28:46,907:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140315021563712)2024-05-29 09:28:46,907:deblur worker thread started
INFO(140315021563712)2024-05-29 09:28:46,907:processing per sample fasta files
INFO(140315021563712)2024-05-29 09:28:46,907:--------------------------------------------------------
INFO(140315021563712)2024-05-29 09:28:46,907:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:28:47,017:*************************
INFO(140018238961472)2024-05-29 09:28:47,017:deblurring started
WARNING(140018238961472)2024-05-29 09:28:47,018:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
WARNING(140018238961472)2024-05-29 09:28:47,018:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log',
'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140018238961472)2024-05-29 09:28:47,018:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140018238961472)2024-05-29 09:28:47,018:deblur worker thread started
INFO(140018238961472)2024-05-29 09:28:47,018:processing per sample fasta files
INFO(140018238961472)2024-05-29 09:28:47,018:--------------------------------------------------------
INFO(140018238961472)2024-05-29 09:28:47,018:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:29:14,148:dereplicate seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(140018238961472)2024-05-29 09:29:14,348:remove_artifacts_seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(140018238961472)2024-05-29 09:29:15,280:total sequences 3450, passing sequences 3450, failing sequences 0
INFO(140018238961472)2024-05-29 09:29:15,280:multiple_sequence_alignment seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140315021563712)2024-05-29 09:29:27,640:dereplicate seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(140315021563712)2024-05-29 09:29:27,884:remove_artifacts_seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(140315021563712)2024-05-29 09:29:28,975:total sequences 4592, passing sequences 4592, failing sequences 0
INFO(140315021563712)2024-05-29 09:29:28,975:multiple_sequence_alignment seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140018238961472)2024-05-29 09:29:46,671:deblurring 3450 sequences
INFO(140315021563712)2024-05-29 09:30:20,166:deblurring 4592 sequences
INFO(140018238961472)2024-05-29 09:30:21,023:2985 unique sequences left following deblurring
INFO(140018238961472)2024-05-29 09:30:21,139:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140018238961472)2024-05-29 09:30:23,119:finished processing file
INFO(140018238961472)2024-05-29 09:30:23,131:finished processing per sample fasta files
INFO(140018238961472)2024-05-29 09:30:23,131:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz finished
INFO(140315021563712)2024-05-29 09:31:11,098:3684 unique sequences left following deblurring
INFO(140315021563712)2024-05-29 09:31:11,235:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140315021563712)2024-05-29 09:31:13,743:finished processing file
INFO(140315021563712)2024-05-29 09:31:13,759:finished processing per sample fasta files
INFO(140315021563712)2024-05-29 09:31:13,760:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz finished
INFO(140094201935680)2024-05-29 09:31:14,103:create_otu_table for 2 samples, into output table /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:14,602:for output biom table loaded 2 samples, 6089 unique sequences
INFO(140094201935680)2024-05-29 09:31:14,612:keeping 254 (out of 6089 sequences) with >=10 reads
INFO(140094201935680)2024-05-29 09:31:14,670:saved to biom file /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:14,671:saved sequence fasta file to /tmp/tmpeayx__ef/all.seqs.fa
INFO(140094201935680)2024-05-29 09:31:14,672:getting 16s sequences from the biom table
INFO(140094201935680)2024-05-29 09:31:14,672:remove_artifacts_seqs file /tmp/tmpeayx__ef/all.seqs.fa
INFO(140094201935680)2024-05-29 09:31:17,705:total sequences 254, passing sequences 0, failing sequences 254
WARNING(140094201935680)2024-05-29 09:31:17,706:input file /tmp/tmpeayx__ef/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(140094201935680)2024-05-29 09:31:17,742:wrote artifact only filtered biom table to /tmp/tmpeayx__ef/reference-non-hit.biom
INFO(140094201935680)2024-05-29 09:31:17,744:saved biom table sequences to fasta file /tmp/tmpeayx__ef/reference-non-hit.seqs.fa
WARNING(140094201935680)2024-05-29 09:31:17,744:removed 2 samples with reads per sample<1
INFO(140094201935680)2024-05-29 09:31:17,752:wrote 16s filtered biom table to /tmp/tmpeayx__ef/reference-hit.biom
INFO(140094201935680)2024-05-29 09:31:17,753:saved biom table sequences to fasta file /tmp/tmpeayx__ef/reference-hit.seqs.fa
INFO(140094201935680)2024-05-29 09:31:17,753:Cleaning up temp files
INFO(140094201935680)2024-05-29 09:31:17,823:deblur workflow finished
INFO(140094201935680)2024-05-29 09:31:17,823:output saved to /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:17,823:------------------

File link: .qza file

Thank you.

colinbrislawn · June 2, 2024, 1:56pm

Hello @Sowmiya_Kathir,

Welcome to the forums. I appreciate your patience while we work on this.

Here are the important lines from the log:

INFO(140018238961472)2024-05-29 09:29:15,280:total sequences 3450, passing sequences 3450, failing sequences 0
INFO(140315021563712)2024-05-29 09:29:28,975:total sequences 4592, passing sequences 4592, failing sequences 0
INFO(140094201935680)2024-05-29 09:31:17,705:total sequences 254, passing sequences 0, failing sequences 254

Two things I would like to check. Are you using more than 3 samples, or are we only seeing the log file from the first three samples?

It looks like most reads are passing the filter, which is a good sign!
It does say output saved to ... so does it make any output in this folder?

Sowmiya_Kathir · June 3, 2024, 4:57am

Hi @colinbrislawn ,

Thanks for your reply. I am using only 2 samples, and also it doesn't create any output.

colinbrislawn · June 3, 2024, 12:36pm

Okay. Good to know.

Something is missing here... because I can count three samples in the output log posted above.

Perhaps you can try rerunning this analysis in a new location and seeing what happens there? Gaining an extra sample for unknown reasons is spooky!

Sowmiya_Kathir · June 6, 2024, 4:24am

As you suggested I have rerun the the analysis in new location. And still I got this issue.

deblur.log file output

INFO(139755635787584)2024-06-04 09:17:45,047:*************************
INFO(139755635787584)2024-06-04 09:17:45,047:deblurring started
WARNING(139755635787584)2024-06-04 09:17:45,048:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid
WARNING(139755635787584)2024-06-04 09:17:45,048:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid', 'output_dir': '/tmp/tmp61j_3_tv', 'pos_ref_fp': (), 'pos_r
ef_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], '
indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/
home/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(139755635787584)2024-06-04 09:17:45,048:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139755635787584)2024-06-04 09:17:45,048:deblur main program started
INFO(139755635787584)2024-06-04 09:17:45,048:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid
INFO(139755635787584)2024-06-04 09:17:45,049:building negative db sortmerna index files
INFO(139755635787584)2024-06-04 09:17:45,049:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to di
r /tmp/tmp61j_3_tv/deblur_working_dir
INFO(139755635787584)2024-06-04 09:17:45,111:building positive db sortmerna index files
INFO(139755635787584)2024-06-04 09:17:45,111:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to d
ir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(139755635787584)2024-06-04 09:18:35,351:parallel processing per sample fasta files
INFO(139755635787584)2024-06-04 09:18:35,351:parallel deblur started for 2 inputs
INFO(140277559166784)2024-06-04 09:18:36,605:*************************
INFO(140560198866752)2024-06-04 09:18:36,605:*************************
INFO(140277559166784)2024-06-04 09:18:36,605:deblurring started
INFO(140560198866752)2024-06-04 09:18:36,605:deblurring started
WARNING(140560198866752)2024-06-04 09:18:36,605:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz
WARNING(140277559166784)2024-06-04 09:18:36,605:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz
WARNING(140560198866752)2024-06-04 09:18:36,605:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmp61j_3_tv'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_
start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
WARNING(140277559166784)2024-06-04 09:18:36,605:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmp61j_3_tv'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_
start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140560198866752)2024-06-04 09:18:36,605:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140277559166784)2024-06-04 09:18:36,605:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140560198866752)2024-06-04 09:18:36,606:deblur worker thread started
INFO(140277559166784)2024-06-04 09:18:36,606:deblur worker thread started
INFO(140560198866752)2024-06-04 09:18:36,606:processing per sample fasta files
INFO(140277559166784)2024-06-04 09:18:36,606:processing per sample fasta files
INFO(140560198866752)2024-06-04 09:18:36,606:--------------------------------------------------------
INFO(140277559166784)2024-06-04 09:18:36,606:--------------------------------------------------------
INFO(140560198866752)2024-06-04 09:18:36,606:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz
INFO(140277559166784)2024-06-04 09:18:36,606:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz
INFO(140560198866752)2024-06-04 09:19:02,943:dereplicate seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(140560198866752)2024-06-04 09:19:03,178:remove_artifacts_seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(140560198866752)2024-06-04 09:19:03,926:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(140560198866752)2024-06-04 09:19:03,927:multiple_sequence_alignment seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140277559166784)2024-06-04 09:19:13,262:dereplicate seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(140277559166784)2024-06-04 09:19:13,549:remove_artifacts_seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(140277559166784)2024-06-04 09:19:14,787:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(140277559166784)2024-06-04 09:19:14,787:multiple_sequence_alignment seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140560198866752)2024-06-04 09:20:50,998:deblurring 7749 sequences
INFO(140277559166784)2024-06-04 09:22:28,627:deblurring 10695 sequences
INFO(140560198866752)2024-06-04 09:23:19,715:6465 unique sequences left following deblurring
INFO(140560198866752)2024-06-04 09:23:19,987:remove_chimeras_denovo_from_seqs seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(140560198866752)2024-06-04 09:23:23,489:finished processing file
INFO(140560198866752)2024-06-04 09:23:23,518:finished processing per sample fasta files
INFO(140560198866752)2024-06-04 09:23:23,518:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz finished
INFO(140277559166784)2024-06-04 09:27:07,851:8755 unique sequences left following deblurring
INFO(140277559166784)2024-06-04 09:27:08,278:remove_chimeras_denovo_from_seqs seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(140277559166784)2024-06-04 09:27:12,026:finished processing file
INFO(140277559166784)2024-06-04 09:27:12,065:finished processing per sample fasta files
INFO(140277559166784)2024-06-04 09:27:12,065:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz finished
INFO(139755635787584)2024-06-04 09:27:12,287:create_otu_table for 2 samples, into output table /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:13,145:for output biom table loaded 2 samples, 13673 unique sequences
INFO(139755635787584)2024-06-04 09:27:13,155:keeping 596 (out of 13673 sequences) with >=10 reads
INFO(139755635787584)2024-06-04 09:27:13,182:saved to biom file /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:13,183:saved sequence fasta file to /tmp/tmp61j_3_tv/all.seqs.fa
INFO(139755635787584)2024-06-04 09:27:13,184:getting 16s sequences from the biom table
INFO(139755635787584)2024-06-04 09:27:13,184:remove_artifacts_seqs file /tmp/tmp61j_3_tv/all.seqs.fa
INFO(139755635787584)2024-06-04 09:27:16,146:total sequences 596, passing sequences 0, failing sequences 596
WARNING(139755635787584)2024-06-04 09:27:16,147:input file /tmp/tmp61j_3_tv/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(139755635787584)2024-06-04 09:27:16,180:wrote artifact only filtered biom table to /tmp/tmp61j_3_tv/reference-non-hit.biom
INFO(139755635787584)2024-06-04 09:27:16,182:saved biom table sequences to fasta file /tmp/tmp61j_3_tv/reference-non-hit.seqs.fa
WARNING(139755635787584)2024-06-04 09:27:16,182:removed 2 samples with reads per sample<1
INFO(139755635787584)2024-06-04 09:27:16,188:wrote 16s filtered biom table to /tmp/tmp61j_3_tv/reference-hit.biom
INFO(139755635787584)2024-06-04 09:27:16,188:saved biom table sequences to fasta file /tmp/tmp61j_3_tv/reference-hit.seqs.fa
INFO(139755635787584)2024-06-04 09:27:16,188:Cleaning up temp files
INFO(139755635787584)2024-06-04 09:27:16,249:deblur workflow finished
INFO(139755635787584)2024-06-04 09:27:16,249:output saved to /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:16,249:------------------
INFO(140171525900096)2024-06-05 09:29:20,510:*************************
INFO(140171525900096)2024-06-05 09:29:20,519:deblurring started
WARNING(140171525900096)2024-06-05 09:29:20,519:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp
WARNING(140171525900096)2024-06-05 09:29:20,519:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(140171525900096)2024-06-05 09:29:20,519:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140171525900096)2024-06-05 09:29:20,519:deblur main program started
INFO(140171525900096)2024-06-05 09:29:20,520:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp
INFO(140171525900096)2024-06-05 09:29:20,521:building negative db sortmerna index files
INFO(140171525900096)2024-06-05 09:29:20,521:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(140171525900096)2024-06-05 09:29:20,584:building positive db sortmerna index files
INFO(140171525900096)2024-06-05 09:29:20,585:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(140171525900096)2024-06-05 09:30:10,565:parallel processing per sample fasta files
INFO(140171525900096)2024-06-05 09:30:10,565:parallel deblur started for 2 inputs
INFO(139704787404608)2024-06-05 09:30:11,785:*************************
INFO(139704787404608)2024-06-05 09:30:11,785:deblurring started
WARNING(139704787404608)2024-06-05 09:30:11,786:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz
WARNING(139704787404608)2024-06-05 09:30:11,786:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(139704787404608)2024-06-05 09:30:11,786:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139704787404608)2024-06-05 09:30:11,786:deblur worker thread started
INFO(139704787404608)2024-06-05 09:30:11,786:processing per sample fasta files
INFO(139704787404608)2024-06-05 09:30:11,786:--------------------------------------------------------
INFO(139704787404608)2024-06-05 09:30:11,786:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz
INFO(139757023082304)2024-06-05 09:30:11,845:*************************
INFO(139757023082304)2024-06-05 09:30:11,845:deblurring started
WARNING(139757023082304)2024-06-05 09:30:11,846:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz
WARNING(139757023082304)2024-06-05 09:30:11,846:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(139757023082304)2024-06-05 09:30:11,846:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139757023082304)2024-06-05 09:30:11,846:deblur worker thread started
INFO(139757023082304)2024-06-05 09:30:11,846:processing per sample fasta files
INFO(139757023082304)2024-06-05 09:30:11,846:--------------------------------------------------------
INFO(139757023082304)2024-06-05 09:30:11,846:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz
INFO(139704787404608)2024-06-05 09:30:38,120:dereplicate seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(139704787404608)2024-06-05 09:30:38,443:remove_artifacts_seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(139704787404608)2024-06-05 09:30:39,411:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(139704787404608)2024-06-05 09:30:39,411:multiple_sequence_alignment seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(139757023082304)2024-06-05 09:30:51,440:dereplicate seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(139757023082304)2024-06-05 09:30:51,768:remove_artifacts_seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(139757023082304)2024-06-05 09:30:52,666:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(139757023082304)2024-06-05 09:30:52,666:multiple_sequence_alignment seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(139704787404608)2024-06-05 09:32:30,137:deblurring 7749 sequences
INFO(139757023082304)2024-06-05 09:34:17,208:deblurring 10695 sequences
INFO(139704787404608)2024-06-05 09:34:56,485:6465 unique sequences left following deblurring
INFO(139704787404608)2024-06-05 09:34:56,749:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(139704787404608)2024-06-05 09:35:00,282:finished processing file
INFO(139704787404608)2024-06-05 09:35:00,311:finished processing per sample fasta files
INFO(139704787404608)2024-06-05 09:35:00,311:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz finished
INFO(139757023082304)2024-06-05 09:39:04,798:8755 unique sequences left following deblurring
INFO(139757023082304)2024-06-05 09:39:05,229:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(139757023082304)2024-06-05 09:39:10,384:finished processing file
INFO(139757023082304)2024-06-05 09:39:10,422:finished processing per sample fasta files
INFO(139757023082304)2024-06-05 09:39:10,422:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz finished
INFO(140171525900096)2024-06-05 09:39:10,645:create_otu_table for 2 samples, into output table /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:11,429:for output biom table loaded 2 samples, 13673 unique sequences
INFO(140171525900096)2024-06-05 09:39:11,439:keeping 596 (out of 13673 sequences) with >=10 reads
INFO(140171525900096)2024-06-05 09:39:11,470:saved to biom file /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:11,471:saved sequence fasta file to /tmp/tmpwnk3zped/all.seqs.fa
INFO(140171525900096)2024-06-05 09:39:11,472:getting 16s sequences from the biom table
INFO(140171525900096)2024-06-05 09:39:11,472:remove_artifacts_seqs file /tmp/tmpwnk3zped/all.seqs.fa
INFO(140171525900096)2024-06-05 09:39:14,476:total sequences 596, passing sequences 0, failing sequences 596
WARNING(140171525900096)2024-06-05 09:39:14,477:input file /tmp/tmpwnk3zped/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(140171525900096)2024-06-05 09:39:14,514:wrote artifact only filtered biom table to /tmp/tmpwnk3zped/reference-non-hit.biom
INFO(140171525900096)2024-06-05 09:39:14,517:saved biom table sequences to fasta file /tmp/tmpwnk3zped/reference-non-hit.seqs.fa
WARNING(140171525900096)2024-06-05 09:39:14,518:removed 2 samples with reads per sample<1
INFO(140171525900096)2024-06-05 09:39:14,530:wrote 16s filtered biom table to /tmp/tmpwnk3zped/reference-hit.biom
INFO(140171525900096)2024-06-05 09:39:14,530:saved biom table sequences to fasta file /tmp/tmpwnk3zped/reference-hit.seqs.fa
INFO(140171525900096)2024-06-05 09:39:14,530:Cleaning up temp files
INFO(140171525900096)2024-06-05 09:39:14,635:deblur workflow finished
INFO(140171525900096)2024-06-05 09:39:14,636:output saved to /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:14,636:------------------

can you check this. I have also attached the .qza file at the 1st post for your reference.

colinbrislawn · June 6, 2024, 2:20pm

It looks like you ran more samples?

Or is only part of the log file included here?

INFO(140560198866752)2024-06-04 09:19:03,926:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(140277559166784)2024-06-04 09:19:14,787:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(139755635787584)2024-06-04 09:27:16,146:total sequences 596, passing sequences 0, failing sequences 596
INFO(139704787404608)2024-06-05 09:30:39,411:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(139757023082304)2024-06-05 09:30:52,666:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(140171525900096)2024-06-05 09:39:14,476:total sequences 596, passing sequences 0, failing sequences 596

Okay. Can you run qiime demux summarize on this and post that output?

Sowmiya_Kathir · June 7, 2024, 10:12am

My apologize, I have run the deblur command multiple times by changing the trim-length. whether, because of that it shows multiple sample?.

I have run qiime demux summarize and i have shared the result.
command:
qiime demux summarize --p-n 10000 --i-data paired-end-demux.qza --o-visualization paired-end-demux.qza

paired-end-demux.qzv (310.4 KB)

colinbrislawn · June 14, 2024, 5:09pm

Hello Sowmiya,

Thank you for your patience. I'm sorry to keep you waiting.

I'm not very familear with the deblur program, but I think the issue is here:

INFO(140171525900096)2024-06-05 09:39:14,476:
total sequences 596, passing sequences 0, failing sequences 596

So all sequences look like artifacts are removed during the remove_artifacts_seqs step! That's not good!

Then later,

WARNING(140171525900096)2024-06-05 09:39:14,518:removed 2 samples with reads per sample<1

Both of your samples are lost here, but the true problem is earlier in the 'remove_artifacts_seqs` step.

Very strange! I'm not sure what is causing this issue with your 16S data. What primers were used to amplify this target rRNA?

system · July 15, 2024, 11:09pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.