Been having some fun with q2-longitudinal recently and I was just wondering if there is a way to perform a paired-test on distance matrices?
Example: I have 4 groups sampled at time 1 and time 2.
I am interested whether there was a significant shift in distance matrix between the two times in each group. The pairwise-distances method calculates the differences between time-points fine, but doesn’t compare those to zero, instead it compares the delta values to other groups only.
On the other hand, pairwise-differences does exactly what I want as it performs Wilcoxon test comparing the change to zero first, followed by a Kruskal-Wallis test across groups. However, this test requires a vector and not a matrix, calculating 1st difference doesn’t work either because it still requires 2 time points as input.
PERMANOVA and adonis can deliver similar answers to my questions but I do worry that I’ll be violating independent assumptions with these tests. Perhaps a Procrustes + mantel can accomplish something similar?
I’m guessing there is a specific reason for exclusion of this feature but I can’t figure it out.
Thanks!
No clear (or good) answer, sorry, but some thoughts. (Take them for what they’re worth.) But, one question that comes up for me is whether a significant shift compared to 0 is really the best measurement. I’m assuming you’re working on rarified data, or something with an error measurement. Do you have any replicates of the same timepoint? Because I might try to compare the within sample pseudo-change with a paired distance. (Essentially, do you significantly overcome the potential technical variation?) Because I’d expect to see a significant difference from 0, just based on noise in the data. …Which may not be a great answer, but is potentially something to consider.
Always happy to hear your thoughts on these things, so thanks for your time!
I should have mentioned that each group has ~n=8 biological replicates (no technical ones), so 4 groups x n=8 x 2 time points. Data is rarefied too but if I can get the method to work I'll probably focus on DEICODE distance which would work without rarefying. I also originally thought the same about what does a significant shift even mean compared to zero and convinced myself it would answer a low-level question of was the community significantly different across time. Also, having seen it at work with first-difference + LME I saw potential in it that I wanted to exploit . Of course I would pair this with some differential abundance testing to extract something tangible but for now I just wanted to get a confirmation that there was indeed a significant shift in the overall community.
I'm not entirely sure I understand what you mean by:
Do you mean that with technical replicates you would expect to detect a difference to zero? I suppose that may be the case, but one could argue that with any test (alpha or differential abundance) no?
I'm actually saying here that with technical replicates, experience says I shouldn't detect a zero difference. At some point, I need to get my crap together and find somewhere to publish the rarefaction plots I invariably make every time I play with this stuff, but basically, the within-sample technical variability Im used to seeing is maybe about 0.1-0.2 in unweighted UniFrac (higher for Jaccard, lower for BC, lowest for weighted unifrac), with biological variability higher. So, essentially, if you want to test if things have changed, compare the within-replicate (or psuedo-replicate) variability with the between timepoint variability.
I have, BTW, no idea if this is actually kosher, but it almost makes more sense to me than comparing to 0, just because of the nature of the data.
That’s an interesting approach, it kind of sounds like a PERMANOVA but with technical replicates? The big problem being though how often is anyone running so many technical replicates to actually perform these kinds of tests? I don’t think I’ve ever seen more than 3, with overwhelming majority running only 1…
It also probably violates the assumptions of independence too, thus my original problem
Kind of? But maybe not entirely. But, its kind of an answer to the meta question if the difference is greater than 0. (Because in my experience it is, at least in the metrics Im used to.) But, it does rely on having a sense of a technical replicate distribution, which is a problem. I mean, you coudl amybe do it with biological replicates at the same timepoint, (if you have them, which you probably don’t) but it does have the independence issue.
because the distances can only be > 0, and hence the equivalent test (null hypothesis: distance == 0) will almost always return a significant result. This is pretty much meaningless, and hence the only statistical test performed by that command is the comparison between groups (null hypothesis: mean delta distance is equal between groups).
If you want to see if there is a directional change in beta diversity, you could run pcoa and input the ordination results to pairwise-differences. It sounds like maybe that is what you are after?
I have a follow-up question. If we use the pcoa data for the pairwise-differences method, which information should be described in the metric parameter?
You’re right, any change certainly will be significant compared to 0. I’ll play with the PCoA results as input a bit, hadn’t thought of that, guess its reliability depends on how much of the variation is explained by a single Axis. Thanks @Nicholas_Bokulich!
. Of course I would pair this with some differential abundance testing to extract something tangible but for now I just wanted to get a confirmation that there was indeed a significant shift in the overall community.