Soil water sources for shrub encroachment in arid and semiarid regions have not been fully understood. This study used stable δ²H and δ¹⁸O isotope ratios of plants, soil water and precipitation, coupled with proportional similarity index of species to identify relations between shrub encroachment and water sources for four shrub and grass species at various shrub encroachment stages in a semiarid grassland of Inner Mongolia. We found that shallow rooted grasses (Agriophyllum squarrosum and Stipa bungeana) predominantly used shallow soil water (0–20 cm) during growing season. In contrast, deeply-rooted semi-shrubs (Artemisia ordosica and Artemisia sphaerocephala) used middle and deep soil water in July but shallow and middle soil depth water in May and September. Plants competition for soil water in different shrub encroachment stages were different depending on plant water use sources. The highest plant competition existed in the A. ordosica community in semi-fixed sandy land, followed by A. squarrosum community in shifting sandy land, A. ordosica community in fixed sandy land, and S. bungeana community in fixed sandy land. Competition for soil water and differences of water use sources of grass and shrub species were crucial in driving shrub encroachment in semiarid grasslands. Communities dominated by A. ordosica were not stable, with an increase in surface soil water availability, the dominance was replaced by S. bungeana. Our findings on the linkage between plant water use sources and patterns and shrub encroachment shall help future arid and semiarid grassland management under changing climate conditions.