The history of Mount Abu is as diverse as the city itself. It was once a part of the Chauhan kingdom of Rajasthan and served as a summer resort for the Rajput kings of the region. After that Mount Abu was leased by the British government from the then Maharaja of Sirohi for use as the headquarter of the resident to Rajputana (another name for Rajasthan). During the British rule in India, it used to be the favorite summer destination of the British who came here to escape the dusty, dry heat of the plains particularly Rajasthan. It also served as a sanatorium for the troops. The small huts and cottages made by the British here tell the story of those times even today.

Mount Abu used to be the home of many saints and sages in the old times. Legend has it that all the three hundred and thirty million gods and goddesses of the Hindu pantheon used to visit this holy mountain. It is also the place where the great saint Vashishth lived and performed a yagna (sacrificial worship on a fire pit) to create four Agnikula (four clans of fire) to protect the earth from demons. The yagna was supposed to have been performed near a natural spring, which emerged from a rock shaped like a cow's head.

According to another legend, once sage Vashishth's cow Nandini was trapped in a deep gorge and could not free herself. The sage appealed to Lord Shiva for assistance. The Lord sent Saraswati, the divine stream, to help flood the gorge so that the cow could float up. Vashishth then decided to ensure that such mishaps would not occur in future and asked the youngest son of Himalaya, the king of mountains to fill the chasm permanently. This he did with the assistance of Arbud, the mighty snake. This spot came to be known as Mount Arbud and was later changed to its present form-Mount Abu.

This place is held in reverence by Jains as well since Jain scriptures record that Lord Mahavira, the 24th Jain Tirthankar (spiritual leader), also visited Mount Abu and blessed the city.