longitudinal pairwise difference error

Blockquote (qiime2-2019.10) Jinbings-iMac:fatigue jinbingbai$ qiime longitudinal pairwise-differences
–m-metadata-file sample-metadata.tsv
–m-metadata-file core-metrics-results/Shannon_vector.qza
–p-metric shannon
–p-group-column Fatigue2
–p-state-column time
–p-state-1 1
–p-state-2 3
–p-individual-id-column SubjectName
–p-replicate-handling random
–o-visualization pairwise-differences-shannon-fatigue2.qzv
Plugin error from longitudinal:

local variable ‘t’ referenced before assignment

Debug info has been saved to /var/folders/sd/vlrr9d916gg5qj3yzzf3srfw0000gn/T/qiime2-q2cli-err-bqj8xccb.log

Hi @Bing,
Could you please provide the full error message? (Rerun the command with --verbose added to the command).

I tried again and got another error. See below:
(qiime2-2019.10) Jinbings-iMac:fatigue jinbingbai$ qiime longitudinal pairwise-distances \

–i-distance-matrix core-metrics-results/jaccard_distance_matrix.qza
–m-metadata-file sample-metadata.tsv
–p-group-column physicalActivity2
–p-state-column time
–p-state-1 1
–p-state-2 3
–p-individual-id-column SubjectName
–p-replicate-handling random
–o-visualization pairwise-distances-jaccard-physicalactivity2.qzv
–verbose \

Traceback (most recent call last):
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2cli/commands.py”, line 328, in call
results = action(**arguments)
File “</Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/decorator.py:decorator-gen-240>”, line 2, in pairwise_distances
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 240, in bound_callable
output_types, provenance)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 445, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_longitudinal.py”, line 102, in pairwise_distances
state_column, state_1, state_2)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 59, in _validate_input_values
df1, state, state_column, group)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 70, in _validate_state_in_dataframe
‘group_column or state value.’.format(state, group))
ValueError: State 1.0 is not represented by any members of nan group in metadata. Consider using a different group_column or state value.

Plugin error from longitudinal:

State 1.0 is not represented by any members of nan group in metadata. Consider using a different group_column or state value.

See above for debug info.sample-metadata.tsv (3.7 KB)

I attached the metadata I have and thanks.

I tried shannon and still got the error. Please help me check about what is wrong with the metadata and thanks.

(qiime2-2019.10) Jinbings-iMac:fatigue jinbingbai$ qiime longitudinal pairwise-differences \

–m-metadata-file sample-metadata.tsv
–m-metadata-file core-metrics-results/Shannon_vector.qza
–p-metric shannon
–p-group-column Fatigue2
–p-state-column time
–p-state-1 1
–p-state-2 3
–p-individual-id-column SubjectName
–p-replicate-handling random
–o-visualization pairwise-differences.qzv
–verbose \

Traceback (most recent call last):
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2cli/commands.py”, line 328, in call
results = action(**arguments)
File “</Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/decorator.py:decorator-gen-236>”, line 2, in pairwise_differences
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 240, in bound_callable
output_types, provenance)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 445, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_longitudinal.py”, line 50, in pairwise_differences
group_column, state_column, state_1, state_2)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 59, in _validate_input_values
df1, state, state_column, group)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 70, in _validate_state_in_dataframe
‘group_column or state value.’.format(state, group))
ValueError: State 1.0 is not represented by any members of nan group in metadata. Consider using a different group_column or state value.

Plugin error from longitudinal:

State 1.0 is not represented by any members of nan group in metadata. Consider using a different group_column or state value.

See above for debug info.

The metadata column you are using (“Fatigue2”) contains empty values. You should filter your samples so that you are only comparing samples that have values in this column, or add values for those samples that are missing.

@Nicholas_Bokulich
I fixed the variables. Then i got the same error as before and thanks.

(qiime2-2019.10) Jinbings-iMac:fatigue jinbingbai$ qiime longitudinal pairwise-differences \

–m-metadata-file sample-metadata.tsv
–m-metadata-file core-metrics-results/Shannon_vector.qza
–p-metric shannon
–p-group-column Fatigue2
–p-state-column time
–p-state-1 1
–p-state-2 3
–p-individual-id-column SubjectName
–p-replicate-handling random
–o-visualization Shannon-pairwise-differences-fatigue2.qzv
–verbose \

/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py:116: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
individual_at_state_idx = group_md[_state & _ind].index
/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/scipy/stats/stats.py:5700: RuntimeWarning: divide by zero encountered in double_scalars
z = (bigu - meanrank) / sd
Traceback (most recent call last):
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2cli/commands.py”, line 328, in call
results = action(**arguments)
File “</Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/decorator.py:decorator-gen-236>”, line 2, in pairwise_differences
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 240, in bound_callable
output_types, provenance)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 445, in callable_executor
ret_val = self._callable(output_dir=temp_dir, **view_args)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_longitudinal.py”, line 89, in pairwise_differences
boxplot=True)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 571, in _stats_and_visuals
pairs, parametric=parametric)
File “/Users/jinbingbai/miniconda3/envs/qiime2-2019.10/lib/python3.6/site-packages/q2_longitudinal/_utilities.py”, line 244, in _compare_pairwise_differences
pvals.append((name, t, p))
UnboundLocalError: local variable ‘t’ referenced before assignment

Plugin error from longitudinal:

local variable ‘t’ referenced before assignment

See above for debug info.

Thanks @Bing, you’ve found a nice little bug :bug:

This error is effectively being raised because there is no variance between your groups, and should fail more gracefully.

I suspect this boils down to an issue with your data/metadata. We have not uncovered this error before because usually the groups being compared do not have zero variance. So either all of the samples in one group have zero alpha diversity (unlikely) or you have one group that contains only 1 sample and hence has no variance.

So the workaround for now is to fix your data/metadata: drop any groups that have only 1 sample (in both the shannon data and the metadata), then re-run this test. You can also use qiime metadata tabulate to inspect the shannon results to make sure they look okay.

Good luck!

1 Like