The official website for The Settlers of Catan

Question about Wizards & Dragons Change Places! - Can I use “Change Places!” to have my Bordello switch place with my Robber Baron’s Castle?

Answer

No, the magic doesn’t go that far.