Sustainable production of second-generation energy crops on marginal land holds a great potential for renewable energy development. Because a vast area of marginal land is located in the arid and semiarid regions of the world, water shortage is the most serious environmental limitation. In this study, we developed a water balance model to address the question of whether Miscanthus energy crops can be sustainably produced in the Loess Plateau of China, a region of more than 60 million hectares particularly abundant in semiarid marginal land. The simulation of 20-year soil water content in bare soil, the winter wheat field, and the Miscanthus field across the Loess Plateau suggested that the long-term production of Miscanthus would not cause water depletion in deep soil. This finding addressed a serious concern that growing high-biomass plants in the Loess Plateau might lead to deep-soil water depletion, which was suggested to be the cause of previous failure of afforestation. Planting Miscanthus was effective in reducing surface runoff and consequently preventing water and soil loss in this heavily eroded region. The model and analyses illustrated where in the Loess Plateau this perennial energy crop could be produced with stable and sufficient yield.