Grazing removal is widely used in grassland management. Plant responses following grazing removal at different organizational levels, however, are not well understood. We examined plant responses at different stocking rates in an Inner Mongolia grassland ecosystem dominated by Leymus chinensis and Stipa grandis. Our results indicated that plant response patterns differed significantly among stocking rates, at different levels of organization, and between wet and dry years. Community aboveground net primary production (ANPP) recovered more quickly at low and moderate stocking rates than those at high stocking rates. Response of aboveground net primary production (RANPP) was significantly positively correlated with both individual biomass and density responses of L. chinensis. Overcompensation of L. chinensis after grazing removal contributed greatly to positive RANPP at the community level. Significant compensatory effects were found between the two dominant species and between dominant species and the remaining non-dominant species. Variation in precipitation significantly affected community ANPP, relationships between community and species responses, and compensatory effects between species. Our study suggests that periodic grazing removal is likely to be a useful method for grassland management and that a combination of species with compensatory effects can be advantageous for reseeding practices in grassland restoration.