Defending a 2-0 lead from the home leg of the first round tie, Steve McClaren's side rarely looked in trouble and with better finishing could have stretched their advantage.
After last weekend's disappointing 2-0 home loss against Sunderland McClaren made four changes to his team, with Jimmy Floyd Hasselbaink, Massimo Maccarone, Chris Riggott and Doriva all handed starting berths.
Out went Mark Viduka and Yakubu, together with the injured Ugo Ehiogu and cup-tied Fabio Rochemback.
The changes seemed to have an unsettling effect on Boro and Xanthi went close to opening the scoring after just three minutes.
Brazilian striker Luciano broke away from the visiting defence and unleashed a powerful left-foot shot that Brad Jones just managed to push away.
The Greek side controlled possession in the early stages but the well-organised Boro back-line restricted them to long-range efforts until the 26th minute, when Gareth Southgate made a vital block to deny Levan Maghradze.
However, Boro dominated the rest of the half and Hasselbaink was close to breaking the deadlock 13 minutes before the interval, when he latched onto a James Morrison ball and smashed a left-foot drive that home goalkeeper Petr Pizanowski had to dive full-length to tip away.
Maccarone had a goal chalked off by an offside flag moments later, but Boro still went in happy at half-time.
Xanthi regained some measure of control after the break but again struggled to create clear-cut chances, with Doriva particularly impressive in snuffing out promising moves from the Greek side.
With time running out Boro substitute Szilard Nemeth twice squandered good opportunities to win the game.
First he shot straight at Pizanowski when a pass was the better option, and then he dragged a shot wide when well placed.
However, the Slovakian redeemed himself when he made a goal line clearance to prevent Emerson scoring against his former club and ensure the match finished goalless.