An excavation by Benjamin Mazar at the southern foot of the Temple Mount was conducted and found a stone with Hebrew inscription says "To the Trumpeting Place"

Construction of the Second Temple began under Cyrus in 538 BCE, and was completed on the sixth year of Darius the Great in 516 BCE, 70 years after the exile to Babylonia.

A drawing of Ezekiel's Visionary Temple from the Book of Ezekiel Around 19 BCE, Herod the Great expanded the Temple Mount and rebuilt the Temple, but In the course of the First Jewish-Roman War Titus in 70 CE destroyed it. The Romans did not topple the Western Wall though. Upon the destruction of the Temple, the Rabbis revised prayers, and introduced new ones to request the speedy rebuilding of the Temple and the city of Jerusalem. They also instituted the saying of the portions of the Torah commanding the bringing of the sacrifices in place of the sacrifices themselves.

During the time of the Byzantine Empire, it is believed that Constantine's mother, St. Helena, built a small church on the Mount in the 4th century, calling it the Church of St. Cyrus and St. John, later on it was enlarged and called the Church of the Holy Wisdom. The church was later destroyed and on its ruins the Dome of the Rock was built.

In 363, Emperor Julian II, on his way to engage Persia, stopped at the ruins of the Second Temple in Jerusalem. In keeping with his effort to foster religions other than Christianity, Julian ordered the Temple rebuilt. A personal friend of his, Ammianus Marcellinus, wrote this about the effort:

"Julian thought to rebuild at an extravagant expense the proud Temple once at Jerusalem, and committed this task to Alypius of Antioch. Alypius set vigorously to work, and was seconded by the governor of the province; when fearful balls of fire, breaking out near the foundations, continued their attacks, till the workmen, after repeated scorchings, could approach no more: and he gave up the attempt."

In 691 an octagonal Muslim building topped by a dome was built by the Caliph Abd al Malik around the rock, for political reasons, in violation of the Caliph Omar's teachings. The shrine became known as the Dome of the Rock. The dome itself was covered in gold in 1920.

In 715 the Umayyads led by the Caliph al-Walid I, rebuilt the Temple's nearby Chanuyos into a mosque which they named al-Masjid al-Aqsa, the Al-Aqsa Mosque or in translation "the furthest mosque", corresponding to the Muslim belief of Muhammad's miraculous nocturnal journey as recounted in the Quran and hadith. The term al-Haram al-Sharif (the Noble Sanctuary) refers to the whole area that surrounds that Rock as was called later by the Mamluks and Ottomans

The structures have been ruined or destroyed several times in earthquakes; the current version dates from the first half of the 11th century. For Muslims, the importance of the Dome of the Rock and Al-Aqsa Mosque make Jerusalem the third-holiest city after Mecca and Medina. The mosque and shrine are currently administered by a Waqf (an Islamic trust).

In 1867, a team from the Royal Engineers, led by Lieutenant Charles Warren (later the London police commissioner of Jack the Ripper fame) and financed by the Palestine Exploration Fund, discovered a series of tunnels beneath Jerusalem and the Temple Mount, some of which were directly underneath the headquarters of the Knights Templar. Various small artifacts were found which indicated that Templars had used some of the tunnels, though it is unclear whom exactly first dug them. Some of the ruins, which Warren discovered, came from century’s earlier, and other tunnels that his team discovered had evidently been used for a water system, as they led to a series of cisterns.