He is saying that he's decided to keep quiet about his private life and just do the job he's paid to do--act. Be serious about his work and enjoy his time off, while still being quiet and responsible. He doesn't think he should talk about his private life to the public, because it's none of the public's business, and they'll get upset no matter what he says--to put his words bluntly.