You can't have 2 separate sockets open on the same IP address / port. Each IP/Port is a unique mailbox, so to speak. (Technically, it is possible to have 2 processes attached to the same socket, under certain circumstances, but that isn't relevant to your problem). The rule is simple, but the workarounds are several.
I would ask if the client really needs to have a standard port number. Normally, UDP/TCP clients just use a random socket and communicate with the server. The server is usually the only side of the connection that needs a standard, fixed port. The UdpClient.Receive() method will populate the IPEndPoint with the ip/port of the sender of the packet so that you can reply to it.
Either don't bind to IPAddress.Any (that causes it to bind to all interfaces). Replace the Any with a specific IP address (one machine can have multiple if you setup IP aliasing or multiple adapters), or simpler, change the port number of the client to differ from the server. You should be able to bind one end to the primary ethernet interface IP address, and bind the client to the loopback (127.0.0.1) address.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…