Adding modifiable inputs to plugin functions which have default values

Developer Discussion
,
1

Finally getting back to working on my plugin!

I’m trying to add some user-modifiable parameters to my function. For example, we have a default threshold for filtering OTUs (e.g. an OTU has to be present in 30% of samples to not be thrown out), but I’d like to let users change that if they want. How do I specify this in the plugin_setup.py plugin.methods.register_function call?

Here’s what I’ve tried:

plugin.methods.register_function(
    function=percentile_normalize,
...
    parameters={'metadata': MetadataColumn[Categorical],
                'N_control_thresh': int,
                'otu_thresh': float,
                'zero_val': float
    },
...

Right now, I’m getting this error when I try to run my plugin_setup.py:

TypeError: Parameter 'N_control_thresh' must be a primitive QIIME type, not <class 'int'>

And looking forward: is there anything special that I need to do in the actual function definition to specify these defaults? Currently this is my approach:

def percentile_normalize(table: biom.Table,
                         metadata: qiime2.CategoricalMetadataColumn,
                         N_control_thresh: int=10,
                         otu_thresh: float=0.3,
                         zero_val: float=1e-9) -> biom.Table:

Thanks!

1 Like
3

Hi @cduvallet!

You are 99% the way there, but in plugin_setup you need to use qiime2.plugin.Int (and qiime2.plugin.Float) instead of int (and float). The names should match the Python types, they just start with an uppercase letter so that things aren't impossible to tell apart.

You can also add some predicates to help limit the input, for example:

In [1]: import qiime2.plugin as q2p
                           # or Range(0, 101) since it's an integer
In [2]: PERCENT = q2p.Int % q2p.Range(0, 100, inclusive_end=True)  

In [3]: -1 in PERCENT
Out[3]: False

In [4]: 0 in PERCENT
Out[4]: True

In [5]: 100 in PERCENT
Out[5]: True

In [6]: 101 in PERCENT
Out[6]: False

That works on Float as well.

Also, as a minor note, most of our plugins use proportions ([0, 1]) instead of percentages, but that's entirely up to you!

Nope, that looks exactly right to me!

1 Like
5

Great, thanks so much! Figured it was something simple like that.

And thanks for the catch on the proportions vs. percent - I did mean to specify a proportion ([0, 1]), just was lazy in my wording. I’ll clarify in the documentation!

6

Ok, I made the change you recommended and now running my plugin_setup.py script works without errors, as does installing the plugin.

However, if I try to run my percentile-normalize function (after clearing the dev cache and running python setup.py install), I get a KeyError:

(qiime2-2018.2) 14:06-claire:~/github/q2-perc-norm$ qiime perc-norm percentile-normalize
Traceback (most recent call last):
  File "/Users/claire/anaconda/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 213, in __call__
    arguments, missing_in, verbose, quiet = self.handle_in_params(kwargs)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 289, in handle_in_params
    kwargs, fallback=cmd_fallback
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/handlers.py", line 633, in get_value
    value = self._locate_value(arguments, fallback)
  File "/Users/claire/anaconda/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/handlers.py", line 54, in _locate_value
    v = arguments[self.click_name]
KeyError: 'p_N_control_thresh'

If I play around with changing either the function’s inputs or the parameters in plugin_setup.py to have the p_ prefix, then I get errors in running plugin_setup.py/installing the plugin. Any thoughts on what could be causing this error?

7

Oh no :frowning:
It does seem like something is wrong on our end here, most likely with q2cli. Are you able to execute that action from a different interface (for example, the Artifact API):

from qiime2.plugins.perc_norm.actions as perc_norm

perc_norm.percentile_normalize(...)  # hand it qiime2.Artifact() instances

That makes sense, the p prefix is something q2cli invents to keep everything straight on the command line.

It occurs to me, we've never really tried using upper-casing in our parameters, does changing the N_control_thresh to n_control_thresh in your registration and function change anything? Otherwise, would you be able to link a source repo? (EDIT: nevermind, just saw the link in your original post again)

8

Was able to reproduce, I threw a debug-print into q2cli and saw this is what click sees:

{'output_dir': None, 'verbose': None, 'm_metadata_file': (), 'cmd_config': None, 
'p_n_control_thresh': None, 'i_table': None, 'o_perc_norm_table': None, 
'quiet': None, 'p_otu_thresh': None, 'm_metadata_column': None}

It looks like either we or click are mangling the casing. I’ve filed an issue to fix that.

In the meanwhile, switching it to n_control_thresh will technically fix it (it might be easier for users to type also).

9

Interesting! Glad we found this bug, and glad there’s an easy fix for now. :smiley:

I changed it to n_control_thresh (which, you’re right, is better anyway) and the KeyError goes away.

2 Likes