Medicago falcata L is a perennial legume species native to semiarid grasslands in northern China, and distinguished by its superior tolerance to abiotic stresses such as drought and cold. But there have been few studies to investigate the physiological mechanism underlying salt tolerance in M. falcata. In the present study, we compared the response of non-nodulated M. falcata with legume model plant Medicago truncatula Garetn at physiological and molecular level by submitting them to a salt stock. M. falcata exhibited higher survival rate and less chlorosis than M. truncatula seedlings upon exposure to salt stress, suggesting that M. falcata is more tolerant to salt stress than M. truncatula. The ability to maintain a high Kt and low Na+ contents by plants is an important trait for tolerance to salt stress. M falcata accumulated less amount of Na+ in their shoots than M. truncatula when exposed to NaCl, leading to a higher K+/(K+ + Na+) ratio in M. falcata than in M. truncatula under saline conditions. There was a similar difference in Na+ concentration and K+/(K+ + Na+) ratio in roots and xylem sap between the two species under saline conditions. Results from grafting experiments revealed that seedlings grafted with M. falcata either as scions or rootstocks can significantly reduce Na+ accumulation and increase K+ + Na+) ratio in both shoots and roots under saline conditions. Expression levels of SOS1 that encodes a Na+-H+ antiporter responsible for Na+ extrusion from cells in M. falcata were much higher than in M. truncatula under both control and saline conditions. Abundance of SOS1 transcripts in seedlings grafted with M. falcata as either rootstocks or scions was significantly higher than in seedlings self-grafted with M. truncatula. These results strongly suggested that SOS1-dependent Na+ extrusion is an important mechanism to maintain a relatively low Na+ concentration and high K+/(K+ + Na+) ratio in non-nodulated M. falcata under saline conditions, thus conferring tolerance to salt stress.