The victory gave the Red Sox a spotless 5-0 mark against their pinstriped rivals this season after a three-game sweep of the Yankees last month at Fenway Park.
Bay also supplied some critical firepower in Monday's 6-4 victory with a two-run homer in the seventh inning.
Boston improved to 17-10, while New York dipped to 13-13.
Bay's first-inning blast gave Boston a 4-0 lead after the first three batters in their lineup singled off Yankees starter Joba Chamberlain (1-1) to make it 1-0.
Chamberlain, who was tagged with the loss, clamped down after that by striking out a career-high 12 batters over five and two-thirds innings.
The Yankees clawed back to 4-3 after a three-run homer by Johnny Damon in the third but were unable to score again.
Boston tacked on two insurance runs in the eighth and one in the ninth on just one hit -- a soft, RBI-single by Nick Green in the eighth—as the Red Sox tallied on an assortment of walks, an error, a hit batsman and a sacrifice fly.
Red Sox starter Josh Beckett improved to 3-2, yielding 10 hits in six innings.
Japanese relievers Hideki Okajima and Takashi Saito finished up with three perfect innings -- Okajima retired the six batters he faced and Saito set them down in the ninth.
After his bumpy first inning, Chamberlain enthralled the home crowd with his power pitching but the damage was already done.
"This one was on me because I came out and didn't get our team off to a good start," the 23-year-old Chamberlain, who broke in last season as a dominant reliever, told reporters.
"It was a terrible start."
Yankees manager Joe Girardi found it hard to look on the positive side of Chamberlain's performance.
"We need him to have his best stuff from the first inning on because you can lose a game in the first inning," he said.
Yankees captain Derek Jeter was also downbeat.
"It couldn't be any worse. They beat us five straight," he said.
"But we play them so many times, when we go and play them again you can't be thinking of these five games. You've got to be able to move on."
The arch rivals play each other 13 more times in the regular season.
