Solution: Do not delete the AMP files that are present. Just paste your modpack files on top of it. (And as always, make a backup just in case)
I actually seem to have gotten it to work.
I had deleted every file in the Minecraft folder of AMP to replace it with the modpack’s server file. But I did do a backup of those AMP file just in case. So I put them back in alongside the new modpack’s server file (especially the forge-1.21-51.0.33-shim.jar file) and started the server. As expected, it did not give the error since the file was present but surprisingly, the server managed to start either way? Is it correctly using the correct version of Forge, which should be forge-1.20.1-47.3.7 or is it mistakenly using the 1.21 51.0.33 version?
(Note: At this point I am able to connect and play on the server I’m just not understanding what’s going on here)