Deblur not completing

[Cistron (Michael Baron)]

Hi all,

I’ve run into an issue with Deblur on one specific dataset. For similarly sized data, from the same sequencing run, deblur finishes within 10-15 minutes. However, here it never comes to conclusion, well, at least not within the 4 hours I had booked on the node.

The data has gone through the basic pipeline leading up tho this: import with manifest, end-joining, q-score filtering.

qiime deblur denoise-16S \
    --i-demultiplexed-seqs filtered/filtered_sequences.qza \
    --p-trim-length 253 \
    --output-dir deblur \
    --p-sample-stats \
    --p-jobs-to-start 24 \
    --verbose

There is no output in std or stderr. I’ve scanned the deblur.log file, but nothing stands out (to me). It seems that deblur is doing its job - to a point.

INFO(47663048769152)2019-10-23 13:14:17,491:*************************
INFO(47663048769152)2019-10-23 13:14:17,492:deblurring started
WARNING(47663048769152)2019-10-23 13:14:17,492:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data
WARNING(47663048769152)2019-10-23 13:14:17,492:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': None, 'jobs_to_start': 24, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': (), 'neg_ref_fp': (), 'pos_ref_db_fp': (), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data'}
INFO(47663048769152)2019-10-23 13:14:17,492:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47663048769152)2019-10-23 13:14:17,492:deblur main program started
INFO(47663048769152)2019-10-23 13:14:17,495:processing directory /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data
INFO(47663048769152)2019-10-23 13:14:17,499:building negative db sortmerna index files
INFO(47663048769152)2019-10-23 13:14:17,499:build_index_sortmerna files ['/home/mbaron/.conda/envs/qiime2-2019.7/lib/python3.6/site-packages/deblur/support_files/artifacts.fa'] to dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47663048769152)2019-10-23 13:14:17,642:building positive db sortmerna index files
INFO(47663048769152)2019-10-23 13:14:17,642:build_index_sortmerna files ['/home/mbaron/.conda/envs/qiime2-2019.7/lib/python3.6/site-packages/deblur/support_files/88_otus.fasta'] to dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47663048769152)2019-10-23 13:14:55,303:parallel processing per sample fasta files
INFO(47663048769152)2019-10-23 13:14:55,303:parallel deblur started for 26 inputs
INFO(47226886340224)2019-10-23 13:14:59,667:*************************
INFO(47226886340224)2019-10-23 13:14:59,668:deblurring started
INFO(46943431437952)2019-10-23 13:14:59,668:*************************
INFO(47328209841792)2019-10-23 13:14:59,668:*************************
INFO(47658780970624)2019-10-23 13:14:59,668:*************************
WARNING(47226886340224)2019-10-23 13:14:59,668:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc106_5_L001_R1_001.fastq.gz
INFO(47380914761344)2019-10-23 13:14:59,669:*************************
INFO(47328209841792)2019-10-23 13:14:59,669:deblurring started
INFO(46943431437952)2019-10-23 13:14:59,669:deblurring started
INFO(47074912816768)2019-10-23 13:14:59,669:*************************
INFO(47658780970624)2019-10-23 13:14:59,670:deblurring started
INFO(46927533465216)2019-10-23 13:14:59,670:*************************
WARNING(47226886340224)2019-10-23 13:14:59,671:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc106_5_L001_R1_001.fastq.gz'}
INFO(47175497367168)2019-10-23 13:14:59,671:*************************
INFO(47437922061952)2019-10-23 13:14:59,671:*************************
INFO(47151820611200)2019-10-23 13:14:59,671:*************************
INFO(47697176854144)2019-10-23 13:14:59,671:*************************
INFO(47545043615360)2019-10-23 13:14:59,671:*************************
INFO(47552626577024)2019-10-23 13:14:59,671:*************************
INFO(47518425677440)2019-10-23 13:14:59,671:*************************
INFO(47226886340224)2019-10-23 13:14:59,673:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(46927533465216)2019-10-23 13:14:59,673:deblurring started
INFO(47175497367168)2019-10-23 13:14:59,673:deblurring started
INFO(47362578534016)2019-10-23 13:14:59,671:*************************
INFO(47436923805312)2019-10-23 13:14:59,671:*************************
INFO(47027941146240)2019-10-23 13:14:59,671:*************************
INFO(47151820611200)2019-10-23 13:14:59,673:deblurring started
INFO(47268581682816)2019-10-23 13:14:59,671:*************************
INFO(47437922061952)2019-10-23 13:14:59,673:deblurring started
INFO(47697176854144)2019-10-23 13:14:59,673:deblurring started
INFO(47545043615360)2019-10-23 13:14:59,673:deblurring started
INFO(47226886340224)2019-10-23 13:14:59,673:deblur worker thread started
INFO(47380914761344)2019-10-23 13:14:59,671:deblurring started
WARNING(47658780970624)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc100_0_L001_R1_001.fastq.gz
INFO(47552626577024)2019-10-23 13:14:59,673:deblurring started
INFO(47518425677440)2019-10-23 13:14:59,673:deblurring started
INFO(47698108919424)2019-10-23 13:14:59,672:*************************
INFO(47575769874048)2019-10-23 13:14:59,672:*************************
INFO(47362578534016)2019-10-23 13:14:59,673:deblurring started
INFO(47436923805312)2019-10-23 13:14:59,673:deblurring started
INFO(47027941146240)2019-10-23 13:14:59,673:deblurring started
INFO(47226886340224)2019-10-23 13:14:59,673:processing per sample fasta files
INFO(47699738279552)2019-10-23 13:14:59,672:*************************
WARNING(47175497367168)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc92_18_L001_R1_001.fastq.gz
INFO(47074912816768)2019-10-23 13:14:59,672:deblurring started
WARNING(46943431437952)2019-10-23 13:14:59,672:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc81_7_L001_R1_001.fastq.gz
WARNING(47328209841792)2019-10-23 13:14:59,672:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc83_9_L001_R1_001.fastq.gz
WARNING(46927533465216)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc85_11_L001_R1_001.fastq.gz
WARNING(47151820611200)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc91_17_L001_R1_001.fastq.gz
INFO(47268581682816)2019-10-23 13:14:59,673:deblurring started
INFO(47698108919424)2019-10-23 13:14:59,673:deblurring started
WARNING(47697176854144)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc90_16_L001_R1_001.fastq.gz
WARNING(47437922061952)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc94_20_L001_R1_001.fastq.gz
WARNING(47074912816768)2019-10-23 13:14:59,674:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc86_12_L001_R1_001.fastq.gz
INFO(47575769874048)2019-10-23 13:14:59,673:deblurring started
WARNING(47545043615360)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc82_8_L001_R1_001.fastq.gz
WARNING(47552626577024)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc88_14_L001_R1_001.fastq.gz
WARNING(47380914761344)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc95_21_L001_R1_001.fastq.gz
INFO(47699738279552)2019-10-23 13:14:59,673:deblurring started
WARNING(47518425677440)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc96_22_L001_R1_001.fastq.gz
WARNING(47658780970624)2019-10-23 13:14:59,673:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc100_0_L001_R1_001.fastq.gz'}
WARNING(46943431437952)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc81_7_L001_R1_001.fastq.gz'}
WARNING(47151820611200)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc91_17_L001_R1_001.fastq.gz'}
WARNING(46927533465216)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc85_11_L001_R1_001.fastq.gz'}
WARNING(47362578534016)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc87_13_L001_R1_001.fastq.gz
WARNING(47027941146240)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc105_4_L001_R1_001.fastq.gz
WARNING(47698108919424)2019-10-23 13:14:59,674:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc104_3_L001_R1_001.fastq.gz
WARNING(47328209841792)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc83_9_L001_R1_001.fastq.gz'}
INFO(47226886340224)2019-10-23 13:14:59,673:--------------------------------------------------------
WARNING(47697176854144)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc90_16_L001_R1_001.fastq.gz'}
INFO(47658780970624)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47268581682816)2019-10-23 13:14:59,674:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc93_19_L001_R1_001.fastq.gz
WARNING(47575769874048)2019-10-23 13:14:59,674:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc84_10_L001_R1_001.fastq.gz
WARNING(47074912816768)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc86_12_L001_R1_001.fastq.gz'}
INFO(46943431437952)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47436923805312)2019-10-23 13:14:59,673:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc89_15_L001_R1_001.fastq.gz
INFO(46927533465216)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47151820611200)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47175497367168)2019-10-23 13:14:59,673:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc92_18_L001_R1_001.fastq.gz'}
INFO(47334999735936)2019-10-23 13:14:59,673:*************************
WARNING(47699738279552)2019-10-23 13:14:59,674:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc97_23_L001_R1_001.fastq.gz
WARNING(47552626577024)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc88_14_L001_R1_001.fastq.gz'}
INFO(47328209841792)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47380914761344)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc95_21_L001_R1_001.fastq.gz'}
WARNING(47437922061952)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc94_20_L001_R1_001.fastq.gz'}
INFO(47175497367168)2019-10-23 13:14:59,675:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47658780970624)2019-10-23 13:14:59,674:deblur worker thread started
INFO(47697176854144)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47074912816768)2019-10-23 13:14:59,674:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47518425677440)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc96_22_L001_R1_001.fastq.gz'}
WARNING(47699738279552)2019-10-23 13:14:59,675:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc97_23_L001_R1_001.fastq.gz'}
INFO(46943431437952)2019-10-23 13:14:59,674:deblur worker thread started
INFO(47151820611200)2019-10-23 13:14:59,674:deblur worker thread started
INFO(47175497367168)2019-10-23 13:14:59,675:deblur worker thread started
INFO(46927533465216)2019-10-23 13:14:59,674:deblur worker thread started
INFO(47697176854144)2019-10-23 13:14:59,675:deblur worker thread started
WARNING(47545043615360)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc82_8_L001_R1_001.fastq.gz'}
INFO(47074912816768)2019-10-23 13:14:59,675:deblur worker thread started
INFO(47437922061952)2019-10-23 13:14:59,675:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47226886340224)2019-10-23 13:14:59,674:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc106_5_L001_R1_001.fastq.gz
INFO(47552626577024)2019-10-23 13:14:59,675:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47027941146240)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc105_4_L001_R1_001.fastq.gz'}
INFO(47328209841792)2019-10-23 13:14:59,675:deblur worker thread started
INFO(47699738279552)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47334999735936)2019-10-23 13:14:59,675:deblurring started
INFO(46943431437952)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47380914761344)2019-10-23 13:14:59,675:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47151820611200)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47175497367168)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47697176854144)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(46927533465216)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47658780970624)2019-10-23 13:14:59,675:processing per sample fasta files
INFO(47074912816768)2019-10-23 13:14:59,676:processing per sample fasta files
WARNING(47698108919424)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc104_3_L001_R1_001.fastq.gz'}
INFO(47437922061952)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47699738279552)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47545043615360)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47518425677440)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47437922061952)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47552626577024)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47328209841792)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47699738279552)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(46943431437952)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47151820611200)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47175497367168)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(46927533465216)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47545043615360)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47074912816768)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47027941146240)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47697176854144)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47380914761344)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47698108919424)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47151820611200)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc91_17_L001_R1_001.fastq.gz
INFO(46943431437952)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc81_7_L001_R1_001.fastq.gz
INFO(47175497367168)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc92_18_L001_R1_001.fastq.gz
INFO(46927533465216)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc85_11_L001_R1_001.fastq.gz
INFO(47074912816768)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc86_12_L001_R1_001.fastq.gz
INFO(47545043615360)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47697176854144)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc90_16_L001_R1_001.fastq.gz
INFO(47518425677440)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47699738279552)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47552626577024)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47437922061952)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47698108919424)2019-10-23 13:14:59,676:deblur worker thread started
WARNING(47334999735936)2019-10-23 13:14:59,676:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc102_1_L001_R1_001.fastq.gz
INFO(47658780970624)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47699738279552)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc97_23_L001_R1_001.fastq.gz
INFO(47380914761344)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47027941146240)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47437922061952)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc94_20_L001_R1_001.fastq.gz
INFO(47698108919424)2019-10-23 13:14:59,676:processing per sample fasta files
WARNING(47575769874048)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc84_10_L001_R1_001.fastq.gz'}
INFO(47545043615360)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47518425677440)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47057873309312)2019-10-23 13:14:59,676:*************************
WARNING(47268581682816)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc93_19_L001_R1_001.fastq.gz'}
INFO(47328209841792)2019-10-23 13:14:59,676:--------------------------------------------------------
WARNING(47362578534016)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc87_13_L001_R1_001.fastq.gz'}
INFO(47658780970624)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc100_0_L001_R1_001.fastq.gz
WARNING(47436923805312)2019-10-23 13:14:59,674:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc89_15_L001_R1_001.fastq.gz'}
INFO(47027941146240)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47545043615360)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc82_8_L001_R1_001.fastq.gz
INFO(47552626577024)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47328209841792)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc83_9_L001_R1_001.fastq.gz
INFO(47057873309312)2019-10-23 13:14:59,676:deblurring started
INFO(47268581682816)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
WARNING(47334999735936)2019-10-23 13:14:59,676:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc102_1_L001_R1_001.fastq.gz'}
INFO(47698108919424)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47380914761344)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47575769874048)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47362578534016)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47552626577024)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc88_14_L001_R1_001.fastq.gz
INFO(47436923805312)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47518425677440)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47268581682816)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47698108919424)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc104_3_L001_R1_001.fastq.gz
INFO(47027941146240)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47380914761344)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc95_21_L001_R1_001.fastq.gz
INFO(47334999735936)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47518425677440)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc96_22_L001_R1_001.fastq.gz
INFO(47362578534016)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47575769874048)2019-10-23 13:14:59,676:deblur worker thread started
WARNING(47057873309312)2019-10-23 13:14:59,676:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc107_6_L001_R1_001.fastq.gz
INFO(47436923805312)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47268581682816)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47027941146240)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc105_4_L001_R1_001.fastq.gz
INFO(47334999735936)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47362578534016)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47575769874048)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47436923805312)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47334999735936)2019-10-23 13:14:59,676:processing per sample fasta files
WARNING(47057873309312)2019-10-23 13:14:59,676:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc107_6_L001_R1_001.fastq.gz'}
INFO(47268581682816)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47362578534016)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47575769874048)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47436923805312)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47268581682816)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc93_19_L001_R1_001.fastq.gz
INFO(47334999735936)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47057873309312)2019-10-23 13:14:59,676:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47362578534016)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc87_13_L001_R1_001.fastq.gz
INFO(47575769874048)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc84_10_L001_R1_001.fastq.gz
INFO(47436923805312)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc89_15_L001_R1_001.fastq.gz
INFO(47334999735936)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc102_1_L001_R1_001.fastq.gz
INFO(47057873309312)2019-10-23 13:14:59,676:deblur worker thread started
INFO(47057873309312)2019-10-23 13:14:59,676:processing per sample fasta files
INFO(47057873309312)2019-10-23 13:14:59,676:--------------------------------------------------------
INFO(47057873309312)2019-10-23 13:14:59,676:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc107_6_L001_R1_001.fastq.gz
INFO(47117067812480)2019-10-23 13:14:59,723:*************************
INFO(47117067812480)2019-10-23 13:14:59,723:deblurring started
WARNING(47117067812480)2019-10-23 13:14:59,723:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc98_24_L001_R1_001.fastq.gz
WARNING(47117067812480)2019-10-23 13:14:59,723:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc98_24_L001_R1_001.fastq.gz'}
INFO(47117067812480)2019-10-23 13:14:59,723:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47117067812480)2019-10-23 13:14:59,723:deblur worker thread started
INFO(47117067812480)2019-10-23 13:14:59,723:processing per sample fasta files
INFO(47117067812480)2019-10-23 13:14:59,723:--------------------------------------------------------
INFO(47117067812480)2019-10-23 13:14:59,723:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc98_24_L001_R1_001.fastq.gz
INFO(47658780970624)2019-10-23 13:15:03,152:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc100_0_L001_R1_001.fastq.gz.trim
INFO(47658780970624)2019-10-23 13:15:03,221:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc100_0_L001_R1_001.fastq.gz.trim.derep
INFO(47658780970624)2019-10-23 13:15:03,842:total sequences 774, passing sequences 736, failing sequences 38
INFO(47658780970624)2019-10-23 13:15:03,842:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc100_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47658780970624)2019-10-23 13:15:06,254:deblurring 736 sequences
INFO(47658780970624)2019-10-23 13:15:07,215:621 unique sequences left following deblurring
INFO(47658780970624)2019-10-23 13:15:07,222:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc100_0_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47175497367168)2019-10-23 13:15:07,482:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc92_18_L001_R1_001.fastq.gz.trim
INFO(47175497367168)2019-10-23 13:15:07,675:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc92_18_L001_R1_001.fastq.gz.trim.derep
INFO(47658780970624)2019-10-23 13:15:07,774:finished processing file
INFO(47658780970624)2019-10-23 13:15:07,775:finished processing per sample fasta files
INFO(47658780970624)2019-10-23 13:15:07,775:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc100_0_L001_R1_001.fastq.gz finished
INFO(47175497367168)2019-10-23 13:15:08,343:total sequences 1140, passing sequences 1066, failing sequences 74
INFO(47175497367168)2019-10-23 13:15:08,344:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc92_18_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47835567722112)2019-10-23 13:15:09,622:*************************
INFO(47835567722112)2019-10-23 13:15:09,622:deblurring started
WARNING(47835567722112)2019-10-23 13:15:09,622:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc103_2_L001_R1_001.fastq.gz
WARNING(47835567722112)2019-10-23 13:15:09,622:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc103_2_L001_R1_001.fastq.gz'}
INFO(47835567722112)2019-10-23 13:15:09,622:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47835567722112)2019-10-23 13:15:09,622:deblur worker thread started
INFO(47835567722112)2019-10-23 13:15:09,622:processing per sample fasta files
INFO(47835567722112)2019-10-23 13:15:09,622:--------------------------------------------------------
INFO(47835567722112)2019-10-23 13:15:09,622:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc103_2_L001_R1_001.fastq.gz
INFO(47552626577024)2019-10-23 13:15:11,654:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc88_14_L001_R1_001.fastq.gz.trim
INFO(47552626577024)2019-10-23 13:15:11,768:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc88_14_L001_R1_001.fastq.gz.trim.derep
INFO(47175497367168)2019-10-23 13:15:11,787:deblurring 1066 sequences
INFO(47552626577024)2019-10-23 13:15:12,405:total sequences 1099, passing sequences 990, failing sequences 109
INFO(47552626577024)2019-10-23 13:15:12,405:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc88_14_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47328209841792)2019-10-23 13:15:12,994:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc83_9_L001_R1_001.fastq.gz.trim
INFO(47328209841792)2019-10-23 13:15:13,071:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc83_9_L001_R1_001.fastq.gz.trim.derep
INFO(47328209841792)2019-10-23 13:15:13,760:total sequences 3233, passing sequences 3088, failing sequences 145
INFO(47328209841792)2019-10-23 13:15:13,760:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc83_9_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47175497367168)2019-10-23 13:15:13,904:947 unique sequences left following deblurring
INFO(47175497367168)2019-10-23 13:15:13,919:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc92_18_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(46927533465216)2019-10-23 13:15:14,740:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc85_11_L001_R1_001.fastq.gz.trim
INFO(47175497367168)2019-10-23 13:15:14,764:finished processing file
INFO(47175497367168)2019-10-23 13:15:14,766:finished processing per sample fasta files
INFO(47175497367168)2019-10-23 13:15:14,766:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc92_18_L001_R1_001.fastq.gz finished
INFO(47074912816768)2019-10-23 13:15:14,768:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc86_12_L001_R1_001.fastq.gz.trim
INFO(47074912816768)2019-10-23 13:15:14,843:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc86_12_L001_R1_001.fastq.gz.trim.derep
INFO(46927533465216)2019-10-23 13:15:14,844:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc85_11_L001_R1_001.fastq.gz.trim.derep
INFO(47552626577024)2019-10-23 13:15:15,406:deblurring 990 sequences
INFO(47074912816768)2019-10-23 13:15:15,517:total sequences 2171, passing sequences 1975, failing sequences 196
INFO(47074912816768)2019-10-23 13:15:15,518:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc86_12_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(46927533465216)2019-10-23 13:15:15,524:total sequences 2301, passing sequences 2235, failing sequences 66
INFO(46927533465216)2019-10-23 13:15:15,951:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc85_11_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47402277957248)2019-10-23 13:15:17,086:*************************
INFO(47402277957248)2019-10-23 13:15:17,087:deblurring started
WARNING(47402277957248)2019-10-23 13:15:17,087:deblur version 1.1.0 workflow started on /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc99_25_L001_R1_001.fastq.gz
WARNING(47402277957248)2019-10-23 13:15:17,087:parameters: {'logger': <Logger __main__ (INFO)>, 'is_worker_thread': True, 'jobs_to_start': 1, 'log_file': '/nfs/home4/mbaron/datest/bank_notes2/deblur.log', 'log_level': 2, 'keep_tmp_files': True, 'threads_per_sample': 1, 'min_size': 2, 'min_reads': 10, 'left_trim_length': 0, 'trim_length': 253, 'indel_max': 3, 'indel_prob': 0.01, 'error_dist': [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005], 'mean_error': 0.005, 'overwrite': True, 'neg_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/artifacts',), 'neg_ref_fp': (), 'pos_ref_db_fp': ('/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/88_otus',), 'pos_ref_fp': (), 'output_dir': '/scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan', 'seqs_fp': '/scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc99_25_L001_R1_001.fastq.gz'}
INFO(47402277957248)2019-10-23 13:15:17,087:error_dist is : [1, 0.06, 0.02, 0.02, 0.01, 0.005, 0.005, 0.005, 0.001, 0.001, 0.001, 0.0005]
INFO(47402277957248)2019-10-23 13:15:17,087:deblur worker thread started
INFO(47402277957248)2019-10-23 13:15:17,087:processing per sample fasta files
INFO(47402277957248)2019-10-23 13:15:17,087:--------------------------------------------------------
INFO(47402277957248)2019-10-23 13:15:17,087:launch_workflow for file /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc99_25_L001_R1_001.fastq.gz
INFO(46943431437952)2019-10-23 13:15:17,174:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc81_7_L001_R1_001.fastq.gz.trim
INFO(46943431437952)2019-10-23 13:15:17,313:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc81_7_L001_R1_001.fastq.gz.trim.derep
INFO(47552626577024)2019-10-23 13:15:17,381:879 unique sequences left following deblurring
INFO(47552626577024)2019-10-23 13:15:17,395:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc88_14_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(46943431437952)2019-10-23 13:15:18,014:total sequences 3024, passing sequences 2810, failing sequences 214
INFO(46943431437952)2019-10-23 13:15:18,015:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc81_7_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47552626577024)2019-10-23 13:15:18,161:finished processing file
INFO(47552626577024)2019-10-23 13:15:18,162:finished processing per sample fasta files
INFO(47552626577024)2019-10-23 13:15:18,162:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc88_14_L001_R1_001.fastq.gz finished
INFO(47699738279552)2019-10-23 13:15:19,061:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc97_23_L001_R1_001.fastq.gz.trim
INFO(47699738279552)2019-10-23 13:15:19,153:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc97_23_L001_R1_001.fastq.gz.trim.derep
INFO(47699738279552)2019-10-23 13:15:19,871:total sequences 2231, passing sequences 2047, failing sequences 184
INFO(47699738279552)2019-10-23 13:15:19,872:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc97_23_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47151820611200)2019-10-23 13:15:21,770:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc91_17_L001_R1_001.fastq.gz.trim
INFO(47151820611200)2019-10-23 13:15:21,867:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc91_17_L001_R1_001.fastq.gz.trim.derep
INFO(47151820611200)2019-10-23 13:15:22,652:total sequences 5405, passing sequences 5263, failing sequences 142
INFO(47151820611200)2019-10-23 13:15:22,652:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc91_17_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47074912816768)2019-10-23 13:15:23,701:deblurring 1975 sequences
INFO(46927533465216)2019-10-23 13:15:25,048:deblurring 2235 sequences
INFO(47699738279552)2019-10-23 13:15:27,749:deblurring 2047 sequences
INFO(47328209841792)2019-10-23 13:15:29,214:deblurring 3088 sequences
INFO(47057873309312)2019-10-23 13:15:29,651:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc107_6_L001_R1_001.fastq.gz.trim
INFO(47057873309312)2019-10-23 13:15:29,773:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc107_6_L001_R1_001.fastq.gz.trim.derep
INFO(47057873309312)2019-10-23 13:15:30,843:total sequences 5737, passing sequences 3724, failing sequences 2013
INFO(47057873309312)2019-10-23 13:15:30,843:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc107_6_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47074912816768)2019-10-23 13:15:31,368:1730 unique sequences left following deblurring
INFO(47074912816768)2019-10-23 13:15:31,385:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc86_12_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(46943431437952)2019-10-23 13:15:31,560:deblurring 2810 sequences
INFO(47074912816768)2019-10-23 13:15:32,986:finished processing file
INFO(47074912816768)2019-10-23 13:15:32,988:finished processing per sample fasta files
INFO(47074912816768)2019-10-23 13:15:32,989:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc86_12_L001_R1_001.fastq.gz finished
INFO(46927533465216)2019-10-23 13:15:34,070:2040 unique sequences left following deblurring
INFO(46927533465216)2019-10-23 13:15:34,087:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc85_11_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47699738279552)2019-10-23 13:15:34,805:1519 unique sequences left following deblurring
INFO(47699738279552)2019-10-23 13:15:34,821:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc97_23_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(46927533465216)2019-10-23 13:15:35,975:finished processing file
INFO(46927533465216)2019-10-23 13:15:35,978:finished processing per sample fasta files
INFO(46927533465216)2019-10-23 13:15:35,978:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc85_11_L001_R1_001.fastq.gz finished
INFO(47699738279552)2019-10-23 13:15:36,194:finished processing file
INFO(47699738279552)2019-10-23 13:15:36,196:finished processing per sample fasta files
INFO(47699738279552)2019-10-23 13:15:36,197:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc97_23_L001_R1_001.fastq.gz finished
INFO(47545043615360)2019-10-23 13:15:36,287:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc82_8_L001_R1_001.fastq.gz.trim
INFO(47545043615360)2019-10-23 13:15:36,450:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc82_8_L001_R1_001.fastq.gz.trim.derep
INFO(47545043615360)2019-10-23 13:15:37,132:total sequences 3207, passing sequences 3031, failing sequences 176
INFO(47545043615360)2019-10-23 13:15:37,132:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc82_8_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47835567722112)2019-10-23 13:15:38,017:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc103_2_L001_R1_001.fastq.gz.trim
INFO(47835567722112)2019-10-23 13:15:38,130:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc103_2_L001_R1_001.fastq.gz.trim.derep
INFO(47835567722112)2019-10-23 13:15:38,996:total sequences 6821, passing sequences 6723, failing sequences 98
INFO(47835567722112)2019-10-23 13:15:38,997:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc103_2_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47328209841792)2019-10-23 13:15:46,343:2779 unique sequences left following deblurring
INFO(47328209841792)2019-10-23 13:15:46,365:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc83_9_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(46943431437952)2019-10-23 13:15:47,563:2474 unique sequences left following deblurring
INFO(46943431437952)2019-10-23 13:15:47,588:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc81_7_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47328209841792)2019-10-23 13:15:49,037:finished processing file
INFO(47328209841792)2019-10-23 13:15:49,039:finished processing per sample fasta files
INFO(47328209841792)2019-10-23 13:15:49,039:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc83_9_L001_R1_001.fastq.gz finished
INFO(47436923805312)2019-10-23 13:15:49,241:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc89_15_L001_R1_001.fastq.gz.trim
INFO(47268581682816)2019-10-23 13:15:49,405:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc93_19_L001_R1_001.fastq.gz.trim
INFO(47436923805312)2019-10-23 13:15:49,425:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc89_15_L001_R1_001.fastq.gz.trim.derep
INFO(47437922061952)2019-10-23 13:15:49,453:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc94_20_L001_R1_001.fastq.gz.trim
INFO(47268581682816)2019-10-23 13:15:49,543:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc93_19_L001_R1_001.fastq.gz.trim.derep
INFO(47437922061952)2019-10-23 13:15:49,658:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc94_20_L001_R1_001.fastq.gz.trim.derep
INFO(46943431437952)2019-10-23 13:15:49,937:finished processing file
INFO(46943431437952)2019-10-23 13:15:49,940:finished processing per sample fasta files
INFO(46943431437952)2019-10-23 13:15:49,940:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc81_7_L001_R1_001.fastq.gz finished
INFO(47268581682816)2019-10-23 13:15:50,206:total sequences 2427, passing sequences 2122, failing sequences 305
INFO(47268581682816)2019-10-23 13:15:50,207:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc93_19_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47436923805312)2019-10-23 13:15:50,371:total sequences 6931, passing sequences 6161, failing sequences 770
INFO(47436923805312)2019-10-23 13:15:50,442:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc89_15_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47437922061952)2019-10-23 13:15:50,510:total sequences 3742, passing sequences 2518, failing sequences 1224
INFO(47437922061952)2019-10-23 13:15:50,616:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc94_20_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47057873309312)2019-10-23 13:15:51,788:deblurring 3724 sequences
INFO(47545043615360)2019-10-23 13:15:52,289:deblurring 3031 sequences
INFO(47268581682816)2019-10-23 13:15:58,519:deblurring 2122 sequences
INFO(47151820611200)2019-10-23 13:16:01,492:deblurring 5263 sequences
INFO(47437922061952)2019-10-23 13:16:02,626:deblurring 2518 sequences
INFO(47575769874048)2019-10-23 13:16:02,885:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc84_10_L001_R1_001.fastq.gz.trim
INFO(47575769874048)2019-10-23 13:16:03,121:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc84_10_L001_R1_001.fastq.gz.trim.derep
INFO(47575769874048)2019-10-23 13:16:03,934:total sequences 4871, passing sequences 4352, failing sequences 519
INFO(47575769874048)2019-10-23 13:16:03,934:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc84_10_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47268581682816)2019-10-23 13:16:05,049:1365 unique sequences left following deblurring
INFO(47268581682816)2019-10-23 13:16:05,063:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc93_19_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47697176854144)2019-10-23 13:16:05,459:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc90_16_L001_R1_001.fastq.gz.trim
INFO(47697176854144)2019-10-23 13:16:06,037:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc90_16_L001_R1_001.fastq.gz.trim.derep
INFO(47697176854144)2019-10-23 13:16:06,892:total sequences 6079, passing sequences 5622, failing sequences 457
INFO(47697176854144)2019-10-23 13:16:06,892:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc90_16_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47268581682816)2019-10-23 13:16:06,980:finished processing file
INFO(47268581682816)2019-10-23 13:16:06,982:finished processing per sample fasta files
INFO(47268581682816)2019-10-23 13:16:06,982:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc93_19_L001_R1_001.fastq.gz finished
INFO(47362578534016)2019-10-23 13:16:08,115:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc87_13_L001_R1_001.fastq.gz.trim
INFO(47362578534016)2019-10-23 13:16:08,378:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc87_13_L001_R1_001.fastq.gz.trim.derep
INFO(47362578534016)2019-10-23 13:16:09,418:total sequences 10366, passing sequences 10093, failing sequences 273
INFO(47362578534016)2019-10-23 13:16:09,418:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc87_13_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47545043615360)2019-10-23 13:16:09,917:2608 unique sequences left following deblurring
INFO(47545043615360)2019-10-23 13:16:09,945:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc82_8_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47226886340224)2019-10-23 13:16:09,953:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc106_5_L001_R1_001.fastq.gz.trim
INFO(47226886340224)2019-10-23 13:16:10,241:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc106_5_L001_R1_001.fastq.gz.trim.derep
INFO(47226886340224)2019-10-23 13:16:11,734:total sequences 3741, passing sequences 3151, failing sequences 590
INFO(47226886340224)2019-10-23 13:16:11,735:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc106_5_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47545043615360)2019-10-23 13:16:12,467:finished processing file
INFO(47545043615360)2019-10-23 13:16:12,470:finished processing per sample fasta files
INFO(47545043615360)2019-10-23 13:16:12,470:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc82_8_L001_R1_001.fastq.gz finished
INFO(47437922061952)2019-10-23 13:16:14,224:1679 unique sequences left following deblurring
INFO(47437922061952)2019-10-23 13:16:14,245:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc94_20_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47437922061952)2019-10-23 13:16:15,761:finished processing file
INFO(47437922061952)2019-10-23 13:16:15,763:finished processing per sample fasta files
INFO(47437922061952)2019-10-23 13:16:15,764:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc94_20_L001_R1_001.fastq.gz finished
INFO(47057873309312)2019-10-23 13:16:16,191:2864 unique sequences left following deblurring
INFO(47057873309312)2019-10-23 13:16:16,220:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc107_6_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47057873309312)2019-10-23 13:16:18,969:finished processing file
INFO(47057873309312)2019-10-23 13:16:18,973:finished processing per sample fasta files
INFO(47057873309312)2019-10-23 13:16:18,973:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc107_6_L001_R1_001.fastq.gz finished
INFO(47226886340224)2019-10-23 13:16:27,761:deblurring 3151 sequences
INFO(47575769874048)2019-10-23 13:16:32,200:deblurring 4352 sequences
INFO(47835567722112)2019-10-23 13:16:38,547:deblurring 6723 sequences
INFO(47226886340224)2019-10-23 13:16:40,386:1719 unique sequences left following deblurring
INFO(47226886340224)2019-10-23 13:16:40,403:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc106_5_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47380914761344)2019-10-23 13:16:41,143:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc95_21_L001_R1_001.fastq.gz.trim
INFO(47380914761344)2019-10-23 13:16:41,741:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc95_21_L001_R1_001.fastq.gz.trim.derep
INFO(47226886340224)2019-10-23 13:16:41,972:finished processing file
INFO(47226886340224)2019-10-23 13:16:41,974:finished processing per sample fasta files
INFO(47226886340224)2019-10-23 13:16:41,974:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc106_5_L001_R1_001.fastq.gz finished
INFO(47436923805312)2019-10-23 13:16:42,363:deblurring 6161 sequences
INFO(47380914761344)2019-10-23 13:16:42,565:total sequences 4403, passing sequences 3696, failing sequences 707
INFO(47380914761344)2019-10-23 13:16:42,565:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc95_21_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47151820611200)2019-10-23 13:16:44,270:3926 unique sequences left following deblurring
INFO(47151820611200)2019-10-23 13:16:44,305:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc91_17_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47151820611200)2019-10-23 13:16:48,214:finished processing file
INFO(47151820611200)2019-10-23 13:16:48,220:finished processing per sample fasta files
INFO(47151820611200)2019-10-23 13:16:48,220:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc91_17_L001_R1_001.fastq.gz finished
INFO(47697176854144)2019-10-23 13:16:50,098:deblurring 5622 sequences
INFO(47334999735936)2019-10-23 13:16:51,808:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc102_1_L001_R1_001.fastq.gz.trim
INFO(47334999735936)2019-10-23 13:16:52,239:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc102_1_L001_R1_001.fastq.gz.trim.derep
INFO(47334999735936)2019-10-23 13:16:53,422:total sequences 13862, passing sequences 13695, failing sequences 167
INFO(47334999735936)2019-10-23 13:16:53,422:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc102_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47027941146240)2019-10-23 13:17:03,005:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc105_4_L001_R1_001.fastq.gz.trim
INFO(47380914761344)2019-10-23 13:17:03,273:deblurring 3696 sequences
INFO(47027941146240)2019-10-23 13:17:03,424:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc105_4_L001_R1_001.fastq.gz.trim.derep
INFO(47027941146240)2019-10-23 13:17:04,537:total sequences 7475, passing sequences 5758, failing sequences 1717
INFO(47027941146240)2019-10-23 13:17:04,537:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc105_4_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47575769874048)2019-10-23 13:17:08,052:3455 unique sequences left following deblurring
INFO(47575769874048)2019-10-23 13:17:08,089:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc84_10_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47575769874048)2019-10-23 13:17:11,451:finished processing file
INFO(47575769874048)2019-10-23 13:17:11,455:finished processing per sample fasta files
INFO(47575769874048)2019-10-23 13:17:11,456:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc84_10_L001_R1_001.fastq.gz finished
INFO(47518425677440)2019-10-23 13:17:17,080:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc96_22_L001_R1_001.fastq.gz.trim
INFO(47518425677440)2019-10-23 13:17:17,622:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc96_22_L001_R1_001.fastq.gz.trim.derep
INFO(47380914761344)2019-10-23 13:17:18,705:1751 unique sequences left following deblurring
INFO(47380914761344)2019-10-23 13:17:18,724:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc95_21_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47518425677440)2019-10-23 13:17:19,100:total sequences 19339, passing sequences 18920, failing sequences 419
INFO(47518425677440)2019-10-23 13:17:19,101:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc96_22_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47698108919424)2019-10-23 13:17:19,180:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc104_3_L001_R1_001.fastq.gz.trim
INFO(47698108919424)2019-10-23 13:17:19,666:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc104_3_L001_R1_001.fastq.gz.trim.derep
INFO(47380914761344)2019-10-23 13:17:20,379:finished processing file
INFO(47380914761344)2019-10-23 13:17:20,382:finished processing per sample fasta files
INFO(47380914761344)2019-10-23 13:17:20,382:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc95_21_L001_R1_001.fastq.gz finished
INFO(47698108919424)2019-10-23 13:17:20,799:total sequences 12490, passing sequences 12305, failing sequences 185
INFO(47698108919424)2019-10-23 13:17:20,799:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc104_3_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47402277957248)2019-10-23 13:17:37,885:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc99_25_L001_R1_001.fastq.gz.trim
INFO(47402277957248)2019-10-23 13:17:38,454:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc99_25_L001_R1_001.fastq.gz.trim.derep
INFO(47402277957248)2019-10-23 13:17:39,664:total sequences 13529, passing sequences 13198, failing sequences 331
INFO(47402277957248)2019-10-23 13:17:39,664:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc99_25_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47697176854144)2019-10-23 13:17:40,528:3884 unique sequences left following deblurring
INFO(47697176854144)2019-10-23 13:17:40,567:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc90_16_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47697176854144)2019-10-23 13:17:44,402:finished processing file
INFO(47697176854144)2019-10-23 13:17:44,407:finished processing per sample fasta files
INFO(47697176854144)2019-10-23 13:17:44,407:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc90_16_L001_R1_001.fastq.gz finished
INFO(47436923805312)2019-10-23 13:17:48,343:4264 unique sequences left following deblurring
INFO(47436923805312)2019-10-23 13:17:48,390:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc89_15_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47835567722112)2019-10-23 13:17:50,136:4728 unique sequences left following deblurring
INFO(47835567722112)2019-10-23 13:17:50,174:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc103_2_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47027941146240)2019-10-23 13:17:51,023:deblurring 5758 sequences
INFO(47436923805312)2019-10-23 13:17:52,674:finished processing file
INFO(47436923805312)2019-10-23 13:17:52,680:finished processing per sample fasta files
INFO(47436923805312)2019-10-23 13:17:52,681:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc89_15_L001_R1_001.fastq.gz finished
INFO(47835567722112)2019-10-23 13:17:54,934:finished processing file
INFO(47835567722112)2019-10-23 13:17:54,939:finished processing per sample fasta files
INFO(47835567722112)2019-10-23 13:17:54,939:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc103_2_L001_R1_001.fastq.gz finished
INFO(47362578534016)2019-10-23 13:18:19,802:deblurring 10093 sequences
INFO(47027941146240)2019-10-23 13:18:40,487:3745 unique sequences left following deblurring
INFO(47027941146240)2019-10-23 13:18:40,531:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc105_4_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47027941146240)2019-10-23 13:18:44,191:finished processing file
INFO(47027941146240)2019-10-23 13:18:44,196:finished processing per sample fasta files
INFO(47027941146240)2019-10-23 13:18:44,196:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc105_4_L001_R1_001.fastq.gz finished
INFO(47698108919424)2019-10-23 13:20:27,543:deblurring 12305 sequences
INFO(47362578534016)2019-10-23 13:20:48,784:6224 unique sequences left following deblurring
INFO(47362578534016)2019-10-23 13:20:48,851:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc87_13_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47334999735936)2019-10-23 13:20:48,947:deblurring 13695 sequences
INFO(47362578534016)2019-10-23 13:20:55,193:finished processing file
INFO(47362578534016)2019-10-23 13:20:55,203:finished processing per sample fasta files
INFO(47362578534016)2019-10-23 13:20:55,203:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc87_13_L001_R1_001.fastq.gz finished
INFO(47402277957248)2019-10-23 13:21:30,550:deblurring 13198 sequences
INFO(47698108919424)2019-10-23 13:23:32,494:6589 unique sequences left following deblurring
INFO(47698108919424)2019-10-23 13:23:32,571:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc104_3_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47698108919424)2019-10-23 13:23:39,398:finished processing file
INFO(47698108919424)2019-10-23 13:23:39,409:finished processing per sample fasta files
INFO(47698108919424)2019-10-23 13:23:39,409:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc104_3_L001_R1_001.fastq.gz finished
INFO(47402277957248)2019-10-23 13:23:55,861:4675 unique sequences left following deblurring
INFO(47402277957248)2019-10-23 13:23:55,919:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc99_25_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47402277957248)2019-10-23 13:24:00,573:finished processing file
INFO(47402277957248)2019-10-23 13:24:00,583:finished processing per sample fasta files
INFO(47402277957248)2019-10-23 13:24:00,583:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc99_25_L001_R1_001.fastq.gz finished
INFO(47334999735936)2019-10-23 13:24:46,305:6972 unique sequences left following deblurring
INFO(47334999735936)2019-10-23 13:24:46,379:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc102_1_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47518425677440)2019-10-23 13:24:52,772:deblurring 18920 sequences
INFO(47334999735936)2019-10-23 13:24:53,638:finished processing file
INFO(47334999735936)2019-10-23 13:24:53,650:finished processing per sample fasta files
INFO(47334999735936)2019-10-23 13:24:53,650:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc102_1_L001_R1_001.fastq.gz finished
INFO(47117067812480)2019-10-23 13:30:29,442:dereplicate seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim
INFO(47117067812480)2019-10-23 13:30:33,616:remove_artifacts_seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim.derep
INFO(47117067812480)2019-10-23 13:30:40,239:total sequences 126394, passing sequences 125193, failing sequences 1201
INFO(47117067812480)2019-10-23 13:30:40,239:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim.derep.no_artifacts
INFO(47518425677440)2019-10-23 13:31:06,408:8655 unique sequences left following deblurring
INFO(47518425677440)2019-10-23 13:31:06,498:remove_chimeras_denovo_from_seqs seqs file /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir/515rcbc96_22_L001_R1_001.fastq.gz.trim.derep.no_artifacts.msa.deblurto working dir /scratch/nodespecific/tcn702/mbaron.7010488/tmpgd29sdan/deblur_working_dir
INFO(47518425677440)2019-10-23 13:31:15,808:finished processing file
INFO(47518425677440)2019-10-23 13:31:15,818:finished processing per sample fasta files
INFO(47518425677440)2019-10-23 13:31:15,818:worker thread for /scratch/nodespecific/tcn702/mbaron.7010488/qiime2-archive-jisind7n/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc96_22_L001_R1_001.fastq.gz finished

Any ideas what might be going on?

Thanks,
Mike

[wasade (Daniel McDonald)]

Hi @Cistron,

The timestamps in the log file look like they cover just a few minutes. This suggests Deblur wasn’t doing much if the requested reservation was for 4 hours. What type of information is reported by the cluster regarding wall time, CPU time, peak resident memory, and exit status?

Best,
Daniel

[Cistron (Michael Baron)]

Hi Daniel,

Thanks for your response!

By default, the job-scheduler isn’t reporting those (SLURM in this instance). I’ll do another test-job later today and see whether I can get that information.

Though when I ran Deblur interactively through Jupyter, I checked with htop and it used only a few GBs of memory and saturated one of the cores. Yet, htop showed all the worker threads (I suppose so does the log). The filtered_sequences.qza file is only 300MB, so I wondered whether it didn’t need that much memory and spent most time writing to scratch?

Best,
Mike

[wasade (Daniel McDonald)]

Hi Mike,

Thanks!

The log does include reports from all workers.

Deblur’ing doesn’t have high memory requirements, and 300MB to a scratch filesystem hopefully wouldn’t saturate the IO as that’s not much data.

Best,
Daniel

[Cistron (Michael Baron)]

Hmm, I have some more time-resolved information. Deblur takes off strong and saturates the node well.

However, after about 10-15 minutes, there seems to be only one process running, which is /home/mbaron/.conda/envs/qiime2-2019.7/libexec/mafft/splittbfast -U -x -b 62 -f -1.53 -Q 100.0 -h 0 -p 50 -s -1 -X 0.1 -i infile.

I've also done some sleuthing through the log-file and it seems that for each of my sample-ids there is an entry indicating that the worker-thread has finished (INFO(47554051731072)2019-10-25 12:13:43,685:worker thread for /scratch/nodespecific/tcn1113/mbaron.7018144/qiime2-archive-3c12i800/026983af-fd5d-47e4-b064-164e0f56d398/data/515rcbc99_25_L001_R1_001.fastq.gz finished).

Well, for all but one sample (515rbc98). Where the last lines, which (I think) correspond to the sample are as follows:

INFO(47818473729664)2019-10-25 12:19:53,106:dereplicate seqs file /scratch/nodespecific/tcn1113/mbaron.7018144/tmplhbljtuh/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim
INFO(47818473729664)2019-10-25 12:19:57,350:remove_artifacts_seqs file /scratch/nodespecific/tcn1113/mbaron.7018144/tmplhbljtuh/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim.derep
INFO(47818473729664)2019-10-25 12:20:03,836:total sequences 126394, passing sequences 125193, failing sequences 1201
INFO(47818473729664)2019-10-25 12:20:03,836:multiple_sequence_alignment seqs file /scratch/nodespecific/tcn1113/mbaron.7018144/tmplhbljtuh/deblur_working_dir/515rcbc98_24_L001_R1_001.fastq.gz.trim.derep.no_artifacts

Is any of this useful to decipher what's going on?

Thanks!

[Cistron (Michael Baron)]

Hmm, it seems that it eventually a worker thread is dealing with the last sample.

I might just remove the sample from the manifest and give it another try.

[Cistron (Michael Baron)]

Having removed sample 98, deblur completes within less than 20 minutes. I’m happy to share the removed sequences, if anyone wants to have a closer look.

[wasade (Daniel McDonald)]

Thanks, @Cistron! This investigative work is great, and very helpful for understanding a bit more about what may be going on.

Given the original log file at the seed of this thread, it seems like sample 515rcbc98 has 10x greater unique fragments contained relative to the other samples, which seems unusual. My best guess right now is that MAFFT is struggling to align > 100k sequences, which is certainly quite high. How many total sequences are in that file? It may be worth investigating the raw sequence data for the sample, spot checking some of the sequences, as there may be technical artifacts present.

All the best,
Daniel