Understanding the impacts of biodiversity loss on ecosystem functioning and services has been a central issue in ecology. Experiments in synthetic communities suggest that biodiversity loss may erode a set of ecosystem functions, but studies in natural communities indicate that the effects of biodiversity loss are usually weak and that multiple functions can be sustained by relatively few species. Yet, the mechanisms by which natural ecosystems are able to maintain multiple functions in the face of diversity loss remain poorly understood. With a long-term and large-scale removal experiment in the Inner Mongolian grassland, here we showed that losses of plant functional groups (PFGs) can reduce multiple ecosystem functions, including biomass production, soil NO₃-N use, net ecosystem carbon exchange, gross ecosystem productivity, and ecosystem respiration, but the magnitudes of these effects depended largely on which PFGs were removed. Removing the two dominant PFGs (perennial rhizomatous grasses and perennial bunchgrasses) simultaneously resulted in dramatic declines in all examined functions, but such declines were circumvented when either dominant PFG was present. We identify the major mechanism for this as a compensation effect by which each dominant PFG can mitigate the losses of others. This study provides evidence that compensation ensuing from PFG losses can mitigate their negative consequence, and thus natural communities may be more resilient to biodiversity loss than currently thought if the remaining PFGs have strong compensation capabilities. On the other hand, ecosystems without well-developed compensatory functional diversity may be much more vulnerable to biodiversity loss.