Yeah, could be that the modpack is just sending too much over the network, or you have the render distance too high (default is 10)
If it’s not the render distance, you could try adding this mod to the server:
https://www.curseforge.com/minecraft/mc-mods/connectivity/files/all?page=1&pageSize=20&version=1.21.1&gameVersionTypeId=6