The legacy effects of precipitation, defined as the impact of precipitation from previous years on current-year net primary productivity (NPP), have been well recognized in field experiments and site-level observational studies. However, it remains unclear how consecutive dry or wet years influence the directions and magnitude of legacies and broad-scale patterns of NPP. Here, we quantified the spatial and temporal patterns of legacy effect across different numbers of consecutive wet and dry years from 2000 to 2020 in the Inner Mongolia grassland. Spatially, the response of legacies to precipitation transitions was asymmetric along the mean annual precipitation gradient. The positive response of legacies to precipitation decrease was found for desert steppe and typical steppe after preceding wetness. In contrast, the response of legacies to precipitation increase was negative for desert steppe and meadow steppe after preceding drought. Moreover, legacies were more sensitive in desert steppe than that in typical steppe and meadow steppe for both the preceding wetness and preceding drought. However, the meadow steppe showed the highest response to the inter-annual precipitation change among the three grassland types. Furthermore, the legacies responded more strongly to precipitation decrease (preceding consecutive wet years) than the increase (preceding consecutive dry years) for meadow steppe and typical steppe. Yet, the desert steppe showed the opposite response to precipitation change. Temporally, the legacy effects lasted for only one year and could be transmitted year by year. Our findings highlight that consecutive wet and dry years play essential roles in controlling the NPP responses to climate fluctuations in arid and semi-arid grasslands. Understanding ecosystem responses to climatic periods can provide insights into the effects of directional changes in rainfall associated with climate change.