During medieval times, the canons used part of the gardens as a cemetery and the remainder to grow herbs, vegetables and medicinal plants. They also planted extensive orchards, which lay to the east of the Abbey. In the 19th century the Carys kept an orchard and also a flower garden to supply cut flowers for the house.

A wooden palm house was added and this was replaced in 1969 with the existing structure. At nearly 50 years old it had exceeded its expected life span but has now been fully restored. The restoration work meant cutting back the original plant collection but a star feature that has survived intact is a Nolina, Elephant's foot palm. Kew Gardens have supplied a number of new palms to enable the gardening team to reinstate this impressive collection.

The garden also features an New Zealand border, a fine specimen Chinese Yellow Wood tree and one of a pair of Mulberry trees planted in 1827 to mark the marriage of Henry George Cary to Emily Shedden.

The peaceful gardens that surround the Abbey today were laid out by Torbay Council and
the council is delighted that Four Seasons has agreed to run an education facility and oversee the renovation of the gardens. During the restoration work the students have been gaining an opportunity for valuable learning experiences and the council has been able to maximise the improvements to the garden within the limited funding available.