To build off this idea of tradition vs. law.... prior to the 20th and 25th Amendments, there were several unsettled constitutional questions about presidential succession.
For instance, when Tippecanoe died, the country was actually without a POTUS for a day. Harrison died on April 4th, but Tyler wasn't inaugurated until April 6th due to the delay in receiving the information. He actually took the oath at a hotel that was located at 6th & Pennsylvania, which is now Seward Square. Even though it wasn't much disputed that the VP would become ACTING president, there was debate as to for how long. There was a valid argument put forth, I believe by JQA and Henry Clay, that the VP was only acting POTUS until a new POTUS was determined, as some had argued, via a new election. That's how Article II, Section 1, Clause 6 reads ("or a President shall be elected.") That's what the Succession Act of 1792 seems to indicate. Thing is, Tyler basically just asserted for himself this authority to BECOME president and serve out the rest of the term. No law clearly gave him that authority. But nobody really challenged it, and it was just accepted as tradition.
Tyler's precedent was followed by Fillmore when he replaced Taylor (died July 9th, 1850, inauguration on the 10th... 24hrs wo/POTUS). After Fillmore, there was no significant delay for Johnson/Lincoln, Garfield/Arthur, TR/McKinley, Coolidge/Harding, Truman/FDR or LBJ/JFK (post-25th).
But again, lots of tradition and not exactly law in all of these cases. Even the 20th Amendment refers to a VP etc. as ACTING POTUS, not permanent replacement, except on account of a POTUS-elect's death. These constitutional questions remained unsettled until the 25th Amendment with the explicit text declaring that the VP BECOMES president. This question wasn't resolved for over 150 years!
I believe this theory you've raised, as at the least as valid of an unsettled constitutional question. Americans have just accepted traditions because they've yet to be seriously challenge.
Very interesting theory. The Constitution doesn't specify WHEN or WHERE a presidential oath is to be taken, or WHO is to administer it or witness it. It's actually largely silent on oath procedures, aside from the proscribed oath itself and that it be taken "Before he enter on the Execution of his Office." The 20th Amendment simply defines when the prior term ends ("at noon on the 20th day of January") and after that, "the terms of their successors shall then begin." Much of the inauguration ceremony and traditions are just that, traditions. Traditions are not law.
Theoretically, Trump could have been inaugurated for his second term on the 10th, 11th, 12th, etc. and then upon the expiration of his first term (noon Jan 20th), if he's his own successor, he begins his second term. As crazy as it might seem, there's a very sound Constitutional argument to be made here. Excellent work.
It means that Atchison was correct, sort of, in that he was ACTING President for the day. Polk's term should have ended on the 4th, but if he technically left office a day early on the 3rd, while not a formal resignation, he's still vacated the office. So Polk's VP, George Dallas actually should have been ACTING President from noon on the 3rd until noon on the 4th, when his term as VP ended. Atchison was then technically ACTING President from noon on the 4th, until Taylor was inaugurated at noon on the 5th.
I believe this Sunday issue happened at least 2 or 3 more times since then, but can't remember the exact circumstances of those cases. Those who hold that the oath doesn't matter, should read the Constitution. It explicitly states that the oath MUST be taken before assuming office. Period. That's the law. There is no "Sundays exempted" in this case, like there is for legislative issues.
Frankly I'm utterly baffled as to how procedural failures like these happened as many times as they did and hardly or anyone at all, said anything!