There is, of course, an exception to every rule.  So there are stage directions that I wouldn’t think about ignoring.

Samuel Beckett’s plays are a good example.  Waiting for Godot specifies a single tree in a barren landscape.  To populate the set with some scrub brush as well would be to damage Beckett’s intention.

To have Hamm stand or Clov sit in Endgame would similarly harm the play.

In Equus and in Christopher Schario’s A Christmas Carol, the playwright calls for the actors to be on stage at all times, seated on benches at the sides when they are not part of the action.  This is a choice that should be honored in the production.

Thornton Wilder makes no such request for Our Town, but productions of his play have been staged this way, and I don’t see it as problematic.

Schario’s play calls for a fiddler on stage, who plays music at various points throughout the night.  When I directed the play, there was no fiddler available.  We turned four of the actors into a singing quartet who fulfilled Schario’s intention faithfully, I think.  However, striking the musical component entirely would have lessened the play.

Noel Coward’s estate insists that his plays be staged with complete fidelity to the stage directions, including the smoking.  I’m not sure that every cigarette in Coward’s oeuvre must stand lest his plays be harmed.  (A red pen to some of his dialogue would strengthen the plays, but alas, we must draw the line there.)

Full realistic sets for Our Town would completely contradict the playwright’s purpose; however, if you indicated the gardens with something other than arched trellises, I doubt an audience would be disturbed.

In other words, examine the stage directions, playwright’s notes, and dialogue for the playwright’s intention.  Honor the intention.  If you succeed in this goal, then whatever you do will be all right.