Checkerboard score?

Ideas and Suggestions
1

I’m wondering if the checkerboard score has been implemented in QIIME 2? I know it was part of QIIME 1 and wasn’t sure if its been ported?

Thanks,
Justine

2

Hi @jwdebelius,
No, we do not have that implemented anywhere in QIIME 2. Looking back over this, I think the reason is that the checkerboard score may not be appropriate for compositional data (correct me if I am wrong! That is the reason why none of the correlation methods used in observation_metadata_correlation.py were ported over).

1 Like
3

Hi @Nicholas_Bokulich,

Thanks for the reply.

Checkerboard score is a presence/absence method, or should be based on my reading of the paper. So, Im not thinking it’s as sensitive to compositionality as abundance-based methods. (Although if one of the people who is better at compositionality than I am, like @mortonjt wants to weigh in and let me know if I’m wrong, it would be good.)

I’m looking for something better than abundance-based tests when I’m dealing with data with a stronger signal in unweighted metrics than weighted metrics. I could apply something like a chi-square or logistic regression as well, but I’m a little bit wary of those tests at ASV scales.

Thanks,
Justine

1 Like
4

Got it — we may have thrown out the baby with the bathwater operating on the assumption that all methods in observation_metadata_correlation.py had the same test assumptions.

I will wait for @mortonjt and others to weigh in — if this sounds like a useful test to add to QIIME 2 I will open an issue for it.

Thanks!

5

I think the default point of view for qiime2 is to not endorse any particular method - so any developers that want to contribute a published method can. It’s allow make it easier for others to benchmark said method.

Wrt my opinion - its highly doubtful that this will work on compositional data. When implementing, it’ll be important to run simulations to confirm this (for example negative binomial sampling from lokta volterra simulated dynamics)

1 Like
6

Thanks @mortonjt. for your insight. I’m looking (continously) for presence-absence based methods. Maybe I should double check Sophie’s paper, to see if she benchmarked them, and if not, potentially undertake some benchmarking on m yown.