That is a great question! Alpha diversity is usually calculated using ASVs/OTUs, which means that taxonomic assignment (which may be affected by the region of interest) doesn’t have a role. Shannon diversity will be high if you have many ASVs (evenly distributed) and that’s it. That is, it doesn’t matter if your ASVs are V3-V4 or V4 amplicons. In this sense, if in sample A you have 2 reads, representing 2 ASVs that come from V3-V4; and in sample B you have 2 reads, representing 2 ASVs that come from V4, the Shannon index for both samples will be the exact same.
Therefore, it makes sense that V3-V4 samples have overall higher Shannon diversity, because as you are interested in 2 variable regions, there is a bigger chance you will find more ASVs than when looking to V4 only.
Actually, for this reason, as far as I know it is not advised to compare diversity metrics between samples sequenced with different primer pairs.
Hope this helps!