Hamlet refers to Denmark as a prison because he feels trapped inside the city, that he knows he can't escape Claudius The King. Yet he also knows that whatever he says to Guildenstern & Rosencrantz will be repeated again to the King.(Recall that the King sent them to spy on Hamlet) Hamlet is basically indirectly sending a message to the King, telling him that he known he's being kept a prisoner in the city of Denmark
