feature-table filter-features

Hi
I'm using the command :
qiime feature-table filter-features --i-table qza_path("clean_table.qza") --p-min-samples 3 --p-min-frequency 10 --o-filtered-table qza_path("feature-frequency-filtered-table.qza")
and I'm getting an empty table, maybe I should not use --p min-samples 3 --p min-frequency 10 ?
if I should not use it this way, how should I use it?
thanks in advance!

Hi!
Did you check the summary of "clean_table.qza" by creating its visualization?
You can take a look at it and try to answer two questions:

  1. How many features in the table are found at least in 3 samples?
  2. How many features in the table have frequency >= 10?

If you see that many features satisfy both requirements, then something is wrong with the command you are running (it looks fine to me). If not, then the command you are running is correct, but you will need to check why you don't have such features and if it makes any sense given the experimental design and samples' origin. If it is not making sense, check previous steps like data2 or filtering (if any).

Best,

@timanix thanks for answering!!
how can I create its visualization?

Here is the plugin for it.

thanks!
here are the results:


should i even filter it that way? ( --p-min-samples 3 --p-min-frequency 10 )

Thank you for sharing the results.
Looks like you have only 2 samples. Is it intentional (you have only 2 samples at the very beginning) or you lost samples at certain step?
By indicating minimum samples 3 in the filtering command all features were filtered out because you have only 2 samples. Also, you have only 2 features with total frequency 7,that is lower than 10 in the command.

You need to check all preceeding steps to check at which step most of the reads or / and samples were filtered out. I suggest starting with Dada2 outputs.

Best,

my first question is if I even need to filter the samples? if so how can i decide what is the pattern?
here is the output from dada2:

and here is the output from qiime demux:

what is the meaning of 'features' here?

thanks again!!

Here we see that instead of 2 samples you should have 248!

Features are unique sequences that you have. After Dada2 features are ASVs.

You have a lot of sequences in the demux file but after Dada2 only 2 sequences retain. That means that you lost basically all sequences with Dada2. Please check Dada2 stats output to see if they were filtered out at the filtering step or after merging reads.

Are you working with V3-V4? Reverse reads not of good quality. I would play with Dada2 parameters (truncation) to see if I can successfully merge them, otherwise I would proceed only with forward reads.

Best,

by running: (qiime2) -bash-5.1$ qiime dada2 denoise-paired --i-demultiplexed-seqs demux-paired-end.qza --o-table dada2_table.qza --o-representative-sequences dada2_rep-seqs.qza --o-denoising-stats dada2_denoising-stats.qza --p-trim-left-f 10 --p-trim-left-r 0 --p-trunc-len-f 180 --p-trunc-len-r 0 --p-n-threads 48 --verbose

i got :

The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada.R --input_directory /tmp/tmpc8u23hsq/forward --input_directory_reverse /tmp/tmpc8u23hsq/reverse --output_path /tmp/tmpc8u23hsq/output.tsv.biom --output_track /tmp/tmpc8u23hsq/track.tsv --filtered_directory /tmp/tmpc8u23hsq/filt_f --filtered_directory_reverse /tmp/tmpc8u23hsq/filt_r --truncation_length 180 --truncation_length_reverse 0 --trim_left 10 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 48 --learn_min_reads 1000000

Warning message:
package ‘optparse’ was built under R version 4.2.3 
R version 4.2.2 (2022-10-31) 
Loading required package: Rcpp
DADA2: 1.26.0 / Rcpp: 1.0.10 / RcppParallel: 5.1.6 
2) Filtering The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231830_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231830_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231943_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231943_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231904_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231904_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231910_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231910_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231950_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231950_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231918_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231918_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231985_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231985_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231908_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231908_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231956_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231956_L001_R2_001.fastq.gz not written.
The filter removed all reads: /tmp/tmpc8u23hsq/filt_f/ERR1231923_L001_R1_001.fastq.gz and /tmp/tmpc8u23hsq/filt_r/ERR1231923_L001_R2_001.fastq.gz not written.
Some input samples had no reads pass the filter.
........................................................x.........................................................................x...x.x.......x....x...................x......x.....x............................x..............................................
3) Learning Error Rates
7102770 total bases in 41781 reads from 248 samples will be used for learning the error rates.
12540315 total bases in 41781 reads from 248 samples will be used for learning the error rates.
3) Denoise samples ........................................................................................................................................................................................................................................................
........................................................................................................................................................................................................................................................
5) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) : 
  Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
3: stop("Input must be a valid sequence table.")
2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose)
1: removeBimeraDenovo(seqtab, method = chimeraMethod, minFoldParentOverAbundance = minParentFold, 
       allowOneOff = allowOneOff, multithread = multithread)
Traceback (most recent call last):
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 326, in denoise_paired
    run_commands([cmd])
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmpc8u23hsq/forward', '--input_directory_reverse', '/tmp/tmpc8u23hsq/reverse', '--output_path', '/tmp/tmpc8u23hsq/output.tsv.biom', '--output_track', '/tmp/tmpc8u23hsq/track.tsv', '--filtered_directory', '/tmp/tmpc8u23hsq/filt_f', '--filtered_directory_reverse', '/tmp/tmpc8u23hsq/filt_r', '--truncation_length', '180', '--truncation_length_reverse', '0', '--trim_left', '10', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '48', '--learn_min_reads', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/q2cli/commands.py", line 352, in __call__
    results = action(**arguments)
  File "<decorator-gen-56>", line 2, in denoise_paired
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 234, in bound_callable
    outputs = self._callable_executor_(scope, callable_args,
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 381, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/galilev/.conda/envs/qiime2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 339, in denoise_paired
    raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info. 

I'm using q2cli version 2023.2.0
I didn't take any reverse reads, by looking at the reverse graph it looks like there is no range to take. so I took just the forward reads.

again,thanks so much for helping!

I would try to do the following:

  1. Remove primers from the reads with cutadapt (I think I can see primers in your forward reads, based on the quality plot)
  2. Run dada2 for single reads (it will process only forward reads even if reverse reads are also imported).
  3. I will not trim reads (if primers will be removed at step 1) but truncate it at position 230-240.
  4. Check dada2 stats and table to decide filtering parameters after dada2.

Best,

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.