Plugin error from deblur:max() arg is an empty sequence

Hello everyone,
I am facing some issues with qiime deblur denoise-16. My data is sequenced using NextSeq5000 with read length 300 bp. I am using qiime2023.7 version. I have shared the command, error and input data. Please help me with it.

Command:
"qiime deblur denoise-16S --i-demultiplexed-seqs paired-end-demux.qza --p-trim-length 220 p-sample-stats --output-dir deblurresults"

error:
INFO(140094201935680)2024-05-29 09:27:48,801:*************************
INFO(140094201935680)2024-05-29 09:27:48,801:deblurring started
WARNING(140094201935680)2024-05-29 09:27:48,801:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm
WARNING(140094201935680)2024-05-29 09:27:48,802:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(140094201935680)2024-05-29 09:27:48,802:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140094201935680)2024-05-29 09:27:48,802:deblur main program started
INFO(140094201935680)2024-05-29 09:27:48,802:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm
INFO(140094201935680)2024-05-29 09:27:48,803:building negative db sortmerna index files
INFO(140094201935680)2024-05-29 09:27:48,803:build_index_sortmerna files ['/home/oneomics2/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140094201935680)2024-05-29 09:27:48,939:building positive db sortmerna index files
INFO(140094201935680)2024-05-29 09:27:48,939:build_index_sortmerna files ['/home/oneomics2/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140094201935680)2024-05-29 09:28:45,600:parallel processing per sample fasta files
INFO(140094201935680)2024-05-29 09:28:45,601:parallel deblur started for 2 inputs
INFO(140315021563712)2024-05-29 09:28:46,906:*************************
INFO(140315021563712)2024-05-29 09:28:46,906:deblurring started
WARNING(140315021563712)2024-05-29 09:28:46,907:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
WARNING(140315021563712)2024-05-29 09:28:46,907:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140315021563712)2024-05-29 09:28:46,907:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140315021563712)2024-05-29 09:28:46,907:deblur worker thread started
INFO(140315021563712)2024-05-29 09:28:46,907:processing per sample fasta files
INFO(140315021563712)2024-05-29 09:28:46,907:--------------------------------------------------------
INFO(140315021563712)2024-05-29 09:28:46,907:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:28:47,017:*************************
INFO(140018238961472)2024-05-29 09:28:47,017:deblurring started
WARNING(140018238961472)2024-05-29 09:28:47,018:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
WARNING(140018238961472)2024-05-29 09:28:47,018:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140315021563712)2024-05-29 09:28:46,907:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140315021563712)2024-05-29 09:28:46,907:deblur worker thread started
INFO(140315021563712)2024-05-29 09:28:46,907:processing per sample fasta files
INFO(140315021563712)2024-05-29 09:28:46,907:--------------------------------------------------------
INFO(140315021563712)2024-05-29 09:28:46,907:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:28:47,017:*************************
INFO(140018238961472)2024-05-29 09:28:47,017:deblurring started
WARNING(140018238961472)2024-05-29 09:28:47,018:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
WARNING(140018238961472)2024-05-29 09:28:47,018:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpeayx__ef'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpeayx__ef/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 220, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/oneomics2/data/BDU/Achiraman/16s/Analysis_trim_silva/deblur.log',
'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140018238961472)2024-05-29 09:28:47,018:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140018238961472)2024-05-29 09:28:47,018:deblur worker thread started
INFO(140018238961472)2024-05-29 09:28:47,018:processing per sample fasta files
INFO(140018238961472)2024-05-29 09:28:47,018:--------------------------------------------------------
INFO(140018238961472)2024-05-29 09:28:47,018:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz
INFO(140018238961472)2024-05-29 09:29:14,148:dereplicate seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(140018238961472)2024-05-29 09:29:14,348:remove_artifacts_seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(140018238961472)2024-05-29 09:29:15,280:total sequences 3450, passing sequences 3450, failing sequences 0
INFO(140018238961472)2024-05-29 09:29:15,280:multiple_sequence_alignment seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140315021563712)2024-05-29 09:29:27,640:dereplicate seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(140315021563712)2024-05-29 09:29:27,884:remove_artifacts_seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(140315021563712)2024-05-29 09:29:28,975:total sequences 4592, passing sequences 4592, failing sequences 0
INFO(140315021563712)2024-05-29 09:29:28,975:multiple_sequence_alignment seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140018238961472)2024-05-29 09:29:46,671:deblurring 3450 sequences
INFO(140315021563712)2024-05-29 09:30:20,166:deblurring 4592 sequences
INFO(140018238961472)2024-05-29 09:30:21,023:2985 unique sequences left following deblurring
INFO(140018238961472)2024-05-29 09:30:21,139:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpeayx__ef/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140018238961472)2024-05-29 09:30:23,119:finished processing file
INFO(140018238961472)2024-05-29 09:30:23,131:finished processing per sample fasta files
INFO(140018238961472)2024-05-29 09:30:23,131:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G1_0_L001_R1_001.fastq.gz finished
INFO(140315021563712)2024-05-29 09:31:11,098:3684 unique sequences left following deblurring
INFO(140315021563712)2024-05-29 09:31:11,235:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpeayx__ef/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpeayx__ef/deblur_working_dir
INFO(140315021563712)2024-05-29 09:31:13,743:finished processing file
INFO(140315021563712)2024-05-29 09:31:13,759:finished processing per sample fasta files
INFO(140315021563712)2024-05-29 09:31:13,760:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hnz_d9sm/G4_1_L001_R1_001.fastq.gz finished
INFO(140094201935680)2024-05-29 09:31:14,103:create_otu_table for 2 samples, into output table /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:14,602:for output biom table loaded 2 samples, 6089 unique sequences
INFO(140094201935680)2024-05-29 09:31:14,612:keeping 254 (out of 6089 sequences) with >=10 reads
INFO(140094201935680)2024-05-29 09:31:14,670:saved to biom file /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:14,671:saved sequence fasta file to /tmp/tmpeayx__ef/all.seqs.fa
INFO(140094201935680)2024-05-29 09:31:14,672:getting 16s sequences from the biom table
INFO(140094201935680)2024-05-29 09:31:14,672:remove_artifacts_seqs file /tmp/tmpeayx__ef/all.seqs.fa
INFO(140094201935680)2024-05-29 09:31:17,705:total sequences 254, passing sequences 0, failing sequences 254
WARNING(140094201935680)2024-05-29 09:31:17,706:input file /tmp/tmpeayx__ef/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(140094201935680)2024-05-29 09:31:17,742:wrote artifact only filtered biom table to /tmp/tmpeayx__ef/reference-non-hit.biom
INFO(140094201935680)2024-05-29 09:31:17,744:saved biom table sequences to fasta file /tmp/tmpeayx__ef/reference-non-hit.seqs.fa
WARNING(140094201935680)2024-05-29 09:31:17,744:removed 2 samples with reads per sample<1
INFO(140094201935680)2024-05-29 09:31:17,752:wrote 16s filtered biom table to /tmp/tmpeayx__ef/reference-hit.biom
INFO(140094201935680)2024-05-29 09:31:17,753:saved biom table sequences to fasta file /tmp/tmpeayx__ef/reference-hit.seqs.fa
INFO(140094201935680)2024-05-29 09:31:17,753:Cleaning up temp files
INFO(140094201935680)2024-05-29 09:31:17,823:deblur workflow finished
INFO(140094201935680)2024-05-29 09:31:17,823:output saved to /tmp/tmpeayx__ef/all.biom
INFO(140094201935680)2024-05-29 09:31:17,823:------------------

File link: .qza file

Thank you.

Hello @Sowmiya_Kathir,

Welcome to the forums. I appreciate your patience while we work on this.

Here are the important lines from the log:

INFO(140018238961472)2024-05-29 09:29:15,280:total sequences 3450, passing sequences 3450, failing sequences 0
INFO(140315021563712)2024-05-29 09:29:28,975:total sequences 4592, passing sequences 4592, failing sequences 0
INFO(140094201935680)2024-05-29 09:31:17,705:total sequences 254, passing sequences 0, failing sequences 254

Two things I would like to check. Are you using more than 3 samples, or are we only seeing the log file from the first three samples?

It looks like most reads are passing the filter, which is a good sign!
It does say output saved to ... so does it make any output in this folder?

Hi @colinbrislawn ,

Thanks for your reply. I am using only 2 samples, and also it doesn't create any output.

Okay. Good to know.

Something is missing here... because I can count three samples in the output log posted above.

Perhaps you can try rerunning this analysis in a new location and seeing what happens there? Gaining an extra sample for unknown reasons is spooky! :ghost:

As you suggested I have rerun the the analysis in new location. And still I got this issue.

deblur.log file output

INFO(139755635787584)2024-06-04 09:17:45,047:*************************
INFO(139755635787584)2024-06-04 09:17:45,047:deblurring started
WARNING(139755635787584)2024-06-04 09:17:45,048:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid
WARNING(139755635787584)2024-06-04 09:17:45,048:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid', 'output_dir': '/tmp/tmp61j_3_tv', 'pos_ref_fp': (), 'pos_r
ef_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], '
indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/
home/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(139755635787584)2024-06-04 09:17:45,048:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139755635787584)2024-06-04 09:17:45,048:deblur main program started
INFO(139755635787584)2024-06-04 09:17:45,048:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid
INFO(139755635787584)2024-06-04 09:17:45,049:building negative db sortmerna index files
INFO(139755635787584)2024-06-04 09:17:45,049:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to di
r /tmp/tmp61j_3_tv/deblur_working_dir
INFO(139755635787584)2024-06-04 09:17:45,111:building positive db sortmerna index files
INFO(139755635787584)2024-06-04 09:17:45,111:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to d
ir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(139755635787584)2024-06-04 09:18:35,351:parallel processing per sample fasta files
INFO(139755635787584)2024-06-04 09:18:35,351:parallel deblur started for 2 inputs
INFO(140277559166784)2024-06-04 09:18:36,605:*************************
INFO(140560198866752)2024-06-04 09:18:36,605:*************************
INFO(140277559166784)2024-06-04 09:18:36,605:deblurring started
INFO(140560198866752)2024-06-04 09:18:36,605:deblurring started
WARNING(140560198866752)2024-06-04 09:18:36,605:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz
WARNING(140277559166784)2024-06-04 09:18:36,605:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz
WARNING(140560198866752)2024-06-04 09:18:36,605:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmp61j_3_tv'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_
start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
WARNING(140277559166784)2024-06-04 09:18:36,605:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmp61j_3_tv'
, 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmp61j_3_tv/deblur_working_dir/artifacts',), 'overwrite': Tru
e, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_leng
th': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_
start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(140560198866752)2024-06-04 09:18:36,605:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140277559166784)2024-06-04 09:18:36,605:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140560198866752)2024-06-04 09:18:36,606:deblur worker thread started
INFO(140277559166784)2024-06-04 09:18:36,606:deblur worker thread started
INFO(140560198866752)2024-06-04 09:18:36,606:processing per sample fasta files
INFO(140277559166784)2024-06-04 09:18:36,606:processing per sample fasta files
INFO(140560198866752)2024-06-04 09:18:36,606:--------------------------------------------------------
INFO(140277559166784)2024-06-04 09:18:36,606:--------------------------------------------------------
INFO(140560198866752)2024-06-04 09:18:36,606:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz
INFO(140277559166784)2024-06-04 09:18:36,606:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz
INFO(140560198866752)2024-06-04 09:19:02,943:dereplicate seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(140560198866752)2024-06-04 09:19:03,178:remove_artifacts_seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(140560198866752)2024-06-04 09:19:03,926:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(140560198866752)2024-06-04 09:19:03,927:multiple_sequence_alignment seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140277559166784)2024-06-04 09:19:13,262:dereplicate seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(140277559166784)2024-06-04 09:19:13,549:remove_artifacts_seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(140277559166784)2024-06-04 09:19:14,787:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(140277559166784)2024-06-04 09:19:14,787:multiple_sequence_alignment seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(140560198866752)2024-06-04 09:20:50,998:deblurring 7749 sequences
INFO(140277559166784)2024-06-04 09:22:28,627:deblurring 10695 sequences
INFO(140560198866752)2024-06-04 09:23:19,715:6465 unique sequences left following deblurring
INFO(140560198866752)2024-06-04 09:23:19,987:remove_chimeras_denovo_from_seqs seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(140560198866752)2024-06-04 09:23:23,489:finished processing file
INFO(140560198866752)2024-06-04 09:23:23,518:finished processing per sample fasta files
INFO(140560198866752)2024-06-04 09:23:23,518:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G1_0_L001_R1_001.fastq.gz finished
INFO(140277559166784)2024-06-04 09:27:07,851:8755 unique sequences left following deblurring
INFO(140277559166784)2024-06-04 09:27:08,278:remove_chimeras_denovo_from_seqs seqs file /tmp/tmp61j_3_tv/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmp61j_3_tv/deblur_working_dir
INFO(140277559166784)2024-06-04 09:27:12,026:finished processing file
INFO(140277559166784)2024-06-04 09:27:12,065:finished processing per sample fasta files
INFO(140277559166784)2024-06-04 09:27:12,065:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-miquuyid/G4_1_L001_R1_001.fastq.gz finished
INFO(139755635787584)2024-06-04 09:27:12,287:create_otu_table for 2 samples, into output table /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:13,145:for output biom table loaded 2 samples, 13673 unique sequences
INFO(139755635787584)2024-06-04 09:27:13,155:keeping 596 (out of 13673 sequences) with >=10 reads
INFO(139755635787584)2024-06-04 09:27:13,182:saved to biom file /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:13,183:saved sequence fasta file to /tmp/tmp61j_3_tv/all.seqs.fa
INFO(139755635787584)2024-06-04 09:27:13,184:getting 16s sequences from the biom table
INFO(139755635787584)2024-06-04 09:27:13,184:remove_artifacts_seqs file /tmp/tmp61j_3_tv/all.seqs.fa
INFO(139755635787584)2024-06-04 09:27:16,146:total sequences 596, passing sequences 0, failing sequences 596
WARNING(139755635787584)2024-06-04 09:27:16,147:input file /tmp/tmp61j_3_tv/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(139755635787584)2024-06-04 09:27:16,180:wrote artifact only filtered biom table to /tmp/tmp61j_3_tv/reference-non-hit.biom
INFO(139755635787584)2024-06-04 09:27:16,182:saved biom table sequences to fasta file /tmp/tmp61j_3_tv/reference-non-hit.seqs.fa
WARNING(139755635787584)2024-06-04 09:27:16,182:removed 2 samples with reads per sample<1
INFO(139755635787584)2024-06-04 09:27:16,188:wrote 16s filtered biom table to /tmp/tmp61j_3_tv/reference-hit.biom
INFO(139755635787584)2024-06-04 09:27:16,188:saved biom table sequences to fasta file /tmp/tmp61j_3_tv/reference-hit.seqs.fa
INFO(139755635787584)2024-06-04 09:27:16,188:Cleaning up temp files
INFO(139755635787584)2024-06-04 09:27:16,249:deblur workflow finished
INFO(139755635787584)2024-06-04 09:27:16,249:output saved to /tmp/tmp61j_3_tv/all.biom
INFO(139755635787584)2024-06-04 09:27:16,249:------------------
INFO(140171525900096)2024-06-05 09:29:20,510:*************************
INFO(140171525900096)2024-06-05 09:29:20,519:deblurring started
WARNING(140171525900096)2024-06-05 09:29:20,519:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp
WARNING(140171525900096)2024-06-05 09:29:20,519:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': (), 'neg_ref_fp': (), 'neg_ref_db_fp': (), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': False, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 6, 'is_worker_thread': False, 'logger': <Logger main (INFO)>}
INFO(140171525900096)2024-06-05 09:29:20,519:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(140171525900096)2024-06-05 09:29:20,519:deblur main program started
INFO(140171525900096)2024-06-05 09:29:20,520:processing directory /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp
INFO(140171525900096)2024-06-05 09:29:20,521:building negative db sortmerna index files
INFO(140171525900096)2024-06-05 09:29:20,521:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/artifacts.fa'] to dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(140171525900096)2024-06-05 09:29:20,584:building positive db sortmerna index files
INFO(140171525900096)2024-06-05 09:29:20,585:build_index_sortmerna files ['home/miniconda3/envs/qiime2-2023.7/lib/python3.8/site-packages/deblur/support_files/88_otus.fasta'] to dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(140171525900096)2024-06-05 09:30:10,565:parallel processing per sample fasta files
INFO(140171525900096)2024-06-05 09:30:10,565:parallel deblur started for 2 inputs
INFO(139704787404608)2024-06-05 09:30:11,785:*************************
INFO(139704787404608)2024-06-05 09:30:11,785:deblurring started
WARNING(139704787404608)2024-06-05 09:30:11,786:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz
WARNING(139704787404608)2024-06-05 09:30:11,786:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(139704787404608)2024-06-05 09:30:11,786:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139704787404608)2024-06-05 09:30:11,786:deblur worker thread started
INFO(139704787404608)2024-06-05 09:30:11,786:processing per sample fasta files
INFO(139704787404608)2024-06-05 09:30:11,786:--------------------------------------------------------
INFO(139704787404608)2024-06-05 09:30:11,786:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz
INFO(139757023082304)2024-06-05 09:30:11,845:*************************
INFO(139757023082304)2024-06-05 09:30:11,845:deblurring started
WARNING(139757023082304)2024-06-05 09:30:11,846:deblur version 1.1.1 workflow started on /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz
WARNING(139757023082304)2024-06-05 09:30:11,846:parameters: {'seqs_fp': '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz', 'output_dir': '/tmp/tmpwnk3zped', 'pos_ref_fp': (), 'pos_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/88_otus',), 'neg_ref_fp': (), 'neg_ref_db_fp': ('/tmp/tmpwnk3zped/deblur_working_dir/artifacts',), 'overwrite': True, 'mean_error': 0.005, 'error_dist': [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'indel_prob': 0.01, 'indel_max': 3, 'trim_length': 180, 'left_trim_length': 0, 'min_reads': 10, 'min_size': 2, 'threads_per_sample': 1, 'keep_tmp_files': True, 'log_level': 2, 'log_file': '/home/deblur.log', 'jobs_to_start': 1, 'is_worker_thread': True, 'logger': <Logger main (INFO)>}
INFO(139757023082304)2024-06-05 09:30:11,846:error_dist is : [1.0, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(139757023082304)2024-06-05 09:30:11,846:deblur worker thread started
INFO(139757023082304)2024-06-05 09:30:11,846:processing per sample fasta files
INFO(139757023082304)2024-06-05 09:30:11,846:--------------------------------------------------------
INFO(139757023082304)2024-06-05 09:30:11,846:launch_workflow for file /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz
INFO(139704787404608)2024-06-05 09:30:38,120:dereplicate seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim
INFO(139704787404608)2024-06-05 09:30:38,443:remove_artifacts_seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep
INFO(139704787404608)2024-06-05 09:30:39,411:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(139704787404608)2024-06-05 09:30:39,411:multiple_sequence_alignment seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(139757023082304)2024-06-05 09:30:51,440:dereplicate seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim
INFO(139757023082304)2024-06-05 09:30:51,768:remove_artifacts_seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep
INFO(139757023082304)2024-06-05 09:30:52,666:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(139757023082304)2024-06-05 09:30:52,666:multiple_sequence_alignment seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(139704787404608)2024-06-05 09:32:30,137:deblurring 7749 sequences
INFO(139757023082304)2024-06-05 09:34:17,208:deblurring 10695 sequences
INFO(139704787404608)2024-06-05 09:34:56,485:6465 unique sequences left following deblurring
INFO(139704787404608)2024-06-05 09:34:56,749:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpwnk3zped/deblur_working_dir/G1_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(139704787404608)2024-06-05 09:35:00,282:finished processing file
INFO(139704787404608)2024-06-05 09:35:00,311:finished processing per sample fasta files
INFO(139704787404608)2024-06-05 09:35:00,311:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G1_0_L001_R1_001.fastq.gz finished
INFO(139757023082304)2024-06-05 09:39:04,798:8755 unique sequences left following deblurring
INFO(139757023082304)2024-06-05 09:39:05,229:remove_chimeras_denovo_from_seqs seqs file /tmp/tmpwnk3zped/deblur_working_dir/G4_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /tmp/tmpwnk3zped/deblur_working_dir
INFO(139757023082304)2024-06-05 09:39:10,384:finished processing file
INFO(139757023082304)2024-06-05 09:39:10,422:finished processing per sample fasta files
INFO(139757023082304)2024-06-05 09:39:10,422:worker thread for /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-1qr78udp/G4_1_L001_R1_001.fastq.gz finished
INFO(140171525900096)2024-06-05 09:39:10,645:create_otu_table for 2 samples, into output table /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:11,429:for output biom table loaded 2 samples, 13673 unique sequences
INFO(140171525900096)2024-06-05 09:39:11,439:keeping 596 (out of 13673 sequences) with >=10 reads
INFO(140171525900096)2024-06-05 09:39:11,470:saved to biom file /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:11,471:saved sequence fasta file to /tmp/tmpwnk3zped/all.seqs.fa
INFO(140171525900096)2024-06-05 09:39:11,472:getting 16s sequences from the biom table
INFO(140171525900096)2024-06-05 09:39:11,472:remove_artifacts_seqs file /tmp/tmpwnk3zped/all.seqs.fa
INFO(140171525900096)2024-06-05 09:39:14,476:total sequences 596, passing sequences 0, failing sequences 596
WARNING(140171525900096)2024-06-05 09:39:14,477:input file /tmp/tmpwnk3zped/all.seqs.fa.no_artifacts does not appear to be FASTA or FASTQ
INFO(140171525900096)2024-06-05 09:39:14,514:wrote artifact only filtered biom table to /tmp/tmpwnk3zped/reference-non-hit.biom
INFO(140171525900096)2024-06-05 09:39:14,517:saved biom table sequences to fasta file /tmp/tmpwnk3zped/reference-non-hit.seqs.fa
WARNING(140171525900096)2024-06-05 09:39:14,518:removed 2 samples with reads per sample<1
INFO(140171525900096)2024-06-05 09:39:14,530:wrote 16s filtered biom table to /tmp/tmpwnk3zped/reference-hit.biom
INFO(140171525900096)2024-06-05 09:39:14,530:saved biom table sequences to fasta file /tmp/tmpwnk3zped/reference-hit.seqs.fa
INFO(140171525900096)2024-06-05 09:39:14,530:Cleaning up temp files
INFO(140171525900096)2024-06-05 09:39:14,635:deblur workflow finished
INFO(140171525900096)2024-06-05 09:39:14,636:output saved to /tmp/tmpwnk3zped/all.biom
INFO(140171525900096)2024-06-05 09:39:14,636:------------------

can you check this. I have also attached the .qza file at the 1st post for your reference.

It looks like you ran more samples?

Or is only part of the log file included here?

INFO(140560198866752)2024-06-04 09:19:03,926:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(140277559166784)2024-06-04 09:19:14,787:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(139755635787584)2024-06-04 09:27:16,146:total sequences 596, passing sequences 0, failing sequences 596
INFO(139704787404608)2024-06-05 09:30:39,411:total sequences 7749, passing sequences 7749, failing sequences 0
INFO(139757023082304)2024-06-05 09:30:52,666:total sequences 10695, passing sequences 10695, failing sequences 0
INFO(140171525900096)2024-06-05 09:39:14,476:total sequences 596, passing sequences 0, failing sequences 596

Okay. Can you run qiime demux summarize on this and post that output?

My apologize, I have run the deblur command multiple times by changing the trim-length. whether, because of that it shows multiple sample?.

I have run qiime demux summarize and i have shared the result.
command:
qiime demux summarize --p-n 10000 --i-data paired-end-demux.qza --o-visualization paired-end-demux.qza

paired-end-demux.qzv (310.4 KB)

Hello Sowmiya,

Thank you for your patience. I'm sorry to keep you waiting.

I'm not very familear with the deblur program, but I think the issue is here:

INFO(140171525900096)2024-06-05 09:39:14,476:
total sequences 596, passing sequences 0, failing sequences 596

So all sequences look like artifacts are removed during the remove_artifacts_seqs step! That's not good!

Then later,

WARNING(140171525900096)2024-06-05 09:39:14,518:removed 2 samples with reads per sample<1

Both of your samples are lost here, but the true problem is earlier in the 'remove_artifacts_seqs` step.

Very strange! I'm not sure what is causing this issue with your 16S data. What primers were used to amplify this target rRNA?