The official website for The Settlers of Catan

Question about Wizards & Dragons Magic Books - Can I use Magic Books that are placed in Citadels without Wizards?

Answer

Yes, provided that you have Wizards in other Citadels.