Places you can see but can't get into: The Senate House, the Temple of Concord, the Tullian Jail, the Temple of Castor and Pollux, the Cloaca Maxima (no, not even through that grate in the Lacus Curtius tunnel!), the alcoves along the Via Nova, the Tiber River, the Temple of Hercules (at the Tiber); the pool in the House of the Vestal Virgins, the Temple of Jupiter.
Places you've heard about but can't get into: The Valve House (except during the Xanthus ending), the cistern under the Regia, the Imperial Palace, the reservoir on the Palatine Hill, the mundus, the Field of Wickedness (where Verania was buried alive), the Colosseum.
Places you can get into but that hold nothing of importance (except possibly generating notebook entries): The Basilica Julia, the Basilica Aemilia, the Rostra, the top of the Arch of Severus (unless you really need that clue to the Aerarium door in December).
Things that do nothing: The gates, stone blocks, and grate in the Lacus Curtius; the mysterious wooden doors on the stairs of the Via Nova -- they only open for the Xanthus ending.
Things you can play with but that do nothing important: Most of the items on the characters' desks make some kind of appropriate noise when you click on them; the chains across the doorways in the Lacus Curtius tunnel rattle, and some of the closed gates make a clanging sound; the livestock in the Cattle Market make barnyard sounds; wall pictures not mentioned above often display some kind of fun animation.