Anderson, 38, denied wicketkeeper-batsman Niroshan Dickwella a century after sending him back on 92 to claim his 30th five-wicket haul and second in Sri Lanka -- his previous best was 5-75 in Galle in 2012.
But the dismissal was skipper Joe Root's mastermind after he reviewed the umpire's call, which was not out, and replays suggested the ball had grazed Mathews' bat.
Perera also dug in after Dickwella's wicket and scored his seventh Test half-century to frustrate the England bowlers before falling to Sam Curran.
Root and Bairstow got together to thwart the Sri Lankan attack as they put on an unbroken 93-run stand till close of play.