It was the first time the Addicks have reached the fifth round for five years but they were taken all the way by the League Two leaders, who gave a tremendous performance which was much appreciated by the 4,000 travelling West Country fans in the 22,873 crowd.
In the past Charlton have struggled against lesser teams in cup matches but they finally broke their jinx to deny Yeovil a possible giantkilling, but not before they were given a few frights on the way.
Yeovil could have taken the lead after 21 minutes when Paul Terry, watched from the stands by his brother, Chelsea and England star John, hit the post which Addicks keeper Dean Kiely beaten.
But the home side took the lead against the run of the play when Bryan Hughes volleyed home after Talal El Karkouri's free-kick had been headed goalwards by Hermann Hreidarsson.
Yeovil levelled right on half- time through Terry, who volleyed home after a fantastic 40-yard run by Kevin Gall had set up the chance.
Two goals in the space of seven minutes early in the second half looked to have killed off the second division side, but Yeovil hit back to make the score 3-2 and set up a tremendous late finish.
Charlton took the lead on 50 minutes when Danny Murphy's right-wing cross was tapped home from close range by Francis Jeffers and they increased their lead after good work from Jerome Thomas set up Shaun Bartlett and his shot went into the net off the inside of the post.
Glovers sub Arron Davies brought his side back to the match with a 25-yard effort midway through the second period.
Twenty-goal Phil Jevons had a great chance to level things for the visitors, but saw his close-range effort scrambled away and the Premiership side were able to hang on for victory against their brave opponents.
Man of the Match: Danny Murphy – The former Liverpool midfielder was at the centre of most of his side's creative efforts and he also did much to quell the Yeovil fire.