United’s stand-in goalkeeper saved a penalty from Martin Odegaard in normal time and then denied Kai Havertz in the shootout as the wasteful Gunners had another day to forget in front of goal.
Bruno Fernandes had fired the visitors into the lead early in the second half before Diogo Dalot was sent-off on the hour mark for a second bookable offence.
Gabriel Magalhaes quickly levelled for Arsenal, but they wasted a host of chances to win the tie in 90 minutes and were made to pay.
United were perfect from the spot as the much-criticised Josua Zirkzee stroked home the winning penalty in front of the jubilant travelling support.
“We deserved to pass through this round because we suffered all together and showed character,” said United boss Ruben Amorim.
United’s reward is a meeting with the club’s legendary former striker and caretaker boss Ruud van Nistelrooy, who will return to Old Trafford as Leicester manager in round four.
Defeat is another hammer blow to Mikel Arteta’s attempts to end a five-year trophy drought for Arsenal, who also lost the first leg of their League Cup semi-final 2-0 to Newcastle on Tuesday.
“It was unbelievable,” said Arteta. “(Based on) The performance we deserved to win the game by a mile, but the reality is that we are out and the only thing we will be judged on is that.”
After four consecutive defeats in all competitions, United were much improved in a 2-2 draw away at Premier League leaders Liverpool last weekend.
Amorim’s gameplan was a similar one as the visitors were happy to sit back and contain Arsenal before looking to spring on the counter-attack.
Jesus adds to injury woes
However, neither side posed much attacking threat in a flat first half.
And Arsenal’s injury problems mounted before the break when Gabriel Jesus was carried off on a stretcher with a suspected serious knee injury.
If the first 45 minutes lacked for action, the second period more than compensated.
United hit the Gunners with a sucker punch seven minutes after the restart.
