If you want to skip the menu server and go directly into a specific server, you can use one of the IP Adresses on Minecraft: Java Edition 1.20.4
(Battle 1) Battle direct connect: battle1.legacyminigames.net
(Battle 2) Battle direct connect: battle2.legacyminigames.net
(Battle 3) Battle direct connect: battle3.legacyminigames.net
(Battle 4) Battle direct connect: battle4.legacyminigames.net
(Experimental) Battle direct connect: exp.battle1.legacyminigames.net
(Experimental) Battle direct connect: exp.battle2.legacyminigames.net
[Disabled] (Europe) Battle direct connect: eu.legacyminigames.net
Make sure you read the rules before joining