Soil and root-associated fungi are important in ecosystem functioning, and it is essential to understand driving factors of these fungi in natural ecosystems. In the present study, soil and root fungal communities in a fine-scale grassland were determined using high through-put sequencing, and our aims are to evaluate the relative importance of plant composition, soil elements and space factors on these two fungal communities. Our results showed that (1) fungal communities in soil and plant roots were distinct from each other, and OTU richness in soil was significantly higher than that in plant roots; (2) biomass of perennial rhizome grass, soil carbon content, and soil C/P ratio were key drivers for both soil and root fungal community; (3) dissimilarity of the soil fungal community significantly correlated with spatial distance, while no significant correlations were found between dissimilarity of root-associated fungal community and spatial distance. These results suggest the different spatial patterns of root and soil fungal communities which may contribute to a better understanding of the mechanisms maintaining root and soil fungal communities at small scales in this grassland ecosystem.