Hi Colin,
I've been digging in the meantime and realize that the issue I raised was nonsense.
The issue had to do with a yet not identified issue of cleaned raw data by Trimmomatic (as wrapped by the Kneaddata function of Biobakery3.
It appears that the just adapter and quality trimmed PE data works properly when2- input as reads.qza into q2 assembly assemble-megahit.
However, after subsequent removal of repeats, which is output also created by Kneaddata, the problem arises. To be clear, the import itself works fine:
qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path "$sample_dir/sample/{sample}_manifest.tsv"
--input-format PairedEndFastqManifestPhred33V2
--output-path "$output_file"
But the internal stdout of q2 assembly assemble-megahit is where problems are encountered when reading in sequences:
/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/megahit.py:100: UserWarning: The presets parameter overrides settings for the min_count and k_list parameters. The settings of
min_count and k_list registered in provenance may not reflect the actual settings used by the presets parameter. Re
fer to the megahit documentation for more details, and refer to the presets values of these parameters when interpre
ting or reporting your results.
warnings.warn(warning, UserWarning)
2024-08-13 16:38:26 - MEGAHIT v1.2.9
2024-08-13 16:38:26 - Using megahit_core with POPCNT and BMI2 support
2024-08-13 16:38:27 - Convert reads to binary library
2024-08-13 16:39:03 - b'INFO sequence/io/sequence_lib.cpp : 75 - Lib 0 (/export/microlab/temp/temp_354/qiime2/mi
crolab/data/f67e5fda-04da-411a-8a03-33fcd04eb6be/data/IDIN03_0_L001_R1_001.fastq.gz,/export/microlab/temp/temp_354/q
iime2/microlab/data/f67e5fda-04da-411a-8a03-33fcd04eb6be/data/IDIN03_1_L001_R2_001.fastq.gz): pe, 27970432 reads, 15
0 max length'
2024-08-13 16:39:35 - b'INFO utils/utils.h : 152 - Real: 67.8597\tuser: 25.0557\tsys: 3.6088\tmaxr
ss: 246808'
2024-08-13 16:39:35 - Error occurs when reading inputs
Traceback (most recent call last):
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/me
gahit.py", line 78, in _process_sample
run_command(cmd, verbose=True)
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/_utils.py"
, line 32, in run_command
subprocess.run(cmd, check=True)
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['megahit', '-1', '/export/microlab/temp/temp_354/qiime2/microlab/data/f67e5
fda-04da-411a-8a03-33fcd04eb6be/data/IDIN03_0_L001_R1_001.fastq.gz', '-2', '/export/microlab/temp/temp_354/qiime2/mi
crolab/data/f67e5fda-04da-411a-8a03-33fcd04eb6be/data/IDIN03_1_L001_R2_001.fastq.gz', '-o', '/export/microlab/temp/t
emp_354/tmpjoku9vdv/results', '--presets', 'meta-sensitive', '--min-count', '2', '--k-list', '21,29,39,59,79,99,119,
141', '--bubble-level', '2', '--prune-level', '2', '--prune-depth', '2', '--disconnect-ratio', '0.1', '--low-local-r
atio', '0.2', '--cleaning-rounds', '5', '--memory', '0.9', '--mem-flag', '1', '--num-cpu-threads', '16', '--min-cont
ig-len', '200']' returned non-zero exit status 141.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2cli/commands.py", li
ne 520, in call
results = self._execute_action(
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2cli/commands.py", li
ne 586, in _execute_action
results = action(**arguments)
File "", line 2, in assemble_megahit
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py",
line 342, in bound_callable
outputs = self.callable_executor(
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py",
line 657, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/megahit.py", line 157, in assemble_megahit
(contig,) = _assemble_megahit(seq, **kwargs)
File "", line 2, in _assemble_megahit
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/qiime2/sdk/context.py", line 143, in deferred_action
return action_obj._bind(
File "", line 2, in _assemble_megahit
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 576, in callable_executor
output_views = self._callable(**view_args)
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/megahit.py", line 208, in _assemble_megahit
return assemble_megahit_helper(
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/megahit.py", line 246, in assemble_megahit_helper
_process_sample(samp, fwd, rev, common_args, result)
File "/export/microlab/miniconda3/envs/qiime2-metagenome-2024.5/lib/python3.9/site-packages/q2_assembly/megahit/megahit.py", line 80, in _process_sample
raise Exception(
Exception: An error was encountered while running MEGAHIT, (return code 141), please inspect stdout and stderr to learn more.
Sorry for changing the subject, but I think still useful to understand.
Thanks in advance.
Best,
Pieter