Permafrost represents an important understudied genetic resource. Soil microorganisms play important roles in regulating biogeochemical cycles and maintaining ecosystem function. However, our knowledge of patterns and drivers of permafrost microbial communities is limited over broad geographic scales. Using high-throughput Illumina sequencing, this study compared soil bacterial, archaeal and fungal communities between the active and permafrost layers on the Tibetan Plateau. Our results indicated that microbial alpha diversity was significantly higher in the active layer than in the permafrost layer with the exception of fungal Shannon–Wiener index and Simpson's diversity index, and microbial community structures were significantly different between the two layers. Our results also revealed that environmental factors such as soil fertility (soil organic carbon, dissolved organic carbon and total nitrogen contents) were the primary drivers of the beta diversity of bacterial, archaeal and fungal communities in the active layer. In contrast, environmental variables such as the mean annual precipitation and total phosphorus played dominant roles in driving the microbial beta diversity in the permafrost layer. Spatial distance was important for predicting the bacterial and archaeal beta diversity in both the active and permafrost layers, but not for fungal communities. Collectively, these results demonstrated different driving factors of microbial beta diversity between the active layer and permafrost layer, implying that the drivers of the microbial beta diversity observed in the active layer cannot be used to predict the biogeographic patterns of the microbial beta diversity in the permafrost layer.