Formerly it was a pilgrim's shelter. During a restoration in 1837, an altar with three advocations (St. Leocadia, St. Agatha and St. Barbara) was installed.

Although it retained the title of St. John the Evangelist, it was gradually replaced by St. Leocadia, because of the popular devotion to her relic.