Palacio de Piria makes most sense as a short place of interest while you are already near Montevideo. A quick visit may be enough for casual travelers.