Semantic types for inputs and outputs during plugin development

Hi everyone,
I am trying to develop my first Qiime2 plugin and I am currently struggling with how to create the plugin given my data types.

In general, all my functions (either methods or visualizers) only take parameters of the following kind: str, int, bool, dict, list, tuple; and they output either a pandas dataframe or List[pd.Dataframe] or generate a plot. Thus, they are mostly primitive and I do not seem to use any of the "standard" semantic types.

Specifically, assume for example, I wish to register the following method:

def bacdive_call(
    bacdive_id: str = "",
    bacdive_password: str = "",
    input_via_file: int = 0,
    input_lists: Dict[str, List[str]] = {},
    search_by_id: bool = False,
    output_dir: str = "./") -> pd.DataFrame:

In my plugin_setup.py I write the following:

plugin.methods.register_function(
    function=q2_test._bacdive_caller.bacdive_call,
    inputs={},
    parameters={"bacdive_id" : Str,
                "bacdive_password" : Str,
                "input_via_file" : Int,
                "input_lists" : Dict[Str, List[Str]],
                "search_by_id" : Bool, 
                "output_dir" : Str},
    outputs=[("resulting_df", pd.DataFrame)],
    input_descriptions={},
    parameter_descriptions={...},
...
...

As you can see, I have left input and input_descriptions empty.

Also, note that I have simply labeled "resulting_df" as a pandas dataframe. Can I do this or does it rather belong to the category Metadata? However, my "resulting_df" (which is a pandas dataframe and my function's output) does not have "standard" column types of being either categorical/numeric. It may contain lists or lists with dictionaries as well as standard numerical and categorical entries...

Here you can see an example of how the "resulting_df" might look like:

So, how do I do the method registration?

Many thanks in advance.

2 Likes

Hi @Mahima_Arunkumar ,

Welcome to the Q2 forum and to the :qiime2: plugin dev community!

Let me see if I can answer your questions.

This is fine. It is possible to have a plugin that has no inputs, only parameters.

However, depending on the purpose of your plugin/action(s) it might make sense to create a new type for your input. E.g., maybe you could have a list of multiple bacdive_ids as a Metadata object or specific type if you want to query multiple simultaneously.

In the function signature (def bacdive_call(...) -> pd.DataFrame) this is fine. But in the plugin registration (in plugin_setup.py) you need to designate a semantic type as the output.

Which type you use is a question of how you plan to use these data downstream. Some options I see:

  1. create a new semantic type. This would be best, as you can then control how the data are used downstream. You have a very specific output, and probably want to control which plugins/actions this can be passed to, and so a new type gives you exquisite control. So you probably do not want to use a pre-existing type.
  2. make your action a visualizer instead of a method. This would be more suitable if you do not plan to pass your resulting_df output to any other QIIME 2 actions, but instead create something like a summary table or figures.
3 Likes

Thank you Nicholas! I will try both ways you suggested :slight_smile:

2 Likes

Hi again,

I have decided to take your advice and make my function a visualizer instead of a method. However, I keep getting an error...

Here is my function which is supposed to write multiple files along with index.html to output_dir:

def bacdive_call(output_dir: str ="./", bacdive_id: str ="", bacdive_password: str ="", input_lists: Dict[str, List[str]] = {}, sample_names: List[str] = [], print_res_df_ToFile:int = 1, print_access_stats: int = 1) -> None:

In plugin_setup.py I do the following:

plugin.visualizers.register_function(
    function= q2b.bacdive_call,
    inputs={},
    input_descriptions={},
    parameters={"output_dir": Str, "bacdive_id": Str, "bacdive_password": Str, "input_lists": Dict[Str, List[Str]], "sample_names": List[Str], "print_res_df_ToFile": Int, "print_access_stats": Int},
    parameter_descriptions={"<...>"},
    name="<...>",
    description=("<...>")
)

Now when I go to the terminal, activate the qiime2-2022.8 conda environment, and run pip install -e . from the main directory which contains setup.py, it successfully builds my egg-info directory. Next, I tried to debug with: python3 plugin_setup.py.

At this point I get the following error:

I am not sure what I am doing wrong here...

Perhaps it is the Dict-part? Since I did not find a specific type for standard Python dictionaries in qiime2.core.type.primitive nor in qiime2.core.type.collection, I reused the Dict from the typing package (which I have also used for the data type annotation for this function as shown above). Maybe I was looking in the wrong place?

I would be grateful for your help in solving this error :slight_smile:

1 Like

Hi @Mahima_Arunkumar ,

Correct, this looks like the issue. Currently there is no way to register a dict as an input to a QIIME 2 action, as far as I am aware. I recommend:

  1. for short dicts: instead input two separate lists that must be of equal length.
  2. for long dicts: accept metadata (or create a new type, depending on how strict this should be). Parse row names as keys, other columns as vals.

Sorry that's a somewhat unappealing answer... but hopefully this is a workaround to get you moving forward.

Good luck!

2 Likes