Make sure you have the right modloader and version selected on the server, otherwise the mods will not load. Double check that you have the same mods loaded onto the client as well.
The problem is not the joiningprocess its that the game launched with forge can’t find the server at all in the multiplayer menu. When I got a mod on the server and started minecraft vanilla, he was able to find it but (obviously) I was not able to join
If i would have a missmatching modlist, the server would tell me, but it doesn’t even show up on the list. But no, I had neither mods on the client nor on the server