Four inmates are missing after they "walked away" from a satellite campus of a federal prison over the weekend, authorities said.