c - Linux: Bind UDP listening socket to specific interface (or find out the interface a datagram came in from)?

Question

I have a daemon I'm working on that listens for UDP broadcast packets and responds also by UDP. When a packet comes in, I'd like to know which IP address (or NIC) the packet came TO so that I can respond with that IP address as the source. (For reasons involving a lot of pain, some users of our system want to connect two NICs on the same machine to the same subnet. We tell them not to, but they insist. I don't need to be reminded how ugly that is.)

There seems to be no way to examine a datagram and find out directly either its destination address or the interface it came in on. Based on a lot of googling, I find that the only way to find out the target of a datagram is to have one listening socket per interface and bind the sockets to their respective interfaces.

First of all, my listening socket is created this way:

s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)

To bind the socket, the first thing I tried was this, where nic is a char* to the name of an interface:

// Bind to a single interface
rc=setsockopt(s, SOL_SOCKET, SO_BINDTODEVICE, nic, strlen(nic));
if (rc != 0) { ... }

This has no effect at all and fails silently. Is the ASCII name (e.g. eth0) the correct type of name to pass to this call? Why would it fail silently? According to man 7 socket, "Note that this only works for some socket types, particularly AF_INET sockets. It is not supported for packet sockets (use normal bind(8) there)." I'm not sure what it means by 'packet sockets', but this is an AF_INET socket.

So the next thing I tried was this (based on bind vs SO_BINDTODEVICE socket):

struct sockaddr_ll sock_address;
memset(&sock_address, 0, sizeof(sock_address));
sock_address.sll_family = PF_PACKET;
sock_address.sll_protocol = htons(ETH_P_ALL);
sock_address.sll_ifindex = if_nametoindex(nic);
rc=bind(s, (struct sockaddr*) &sock_address, sizeof(sock_address));
if (rc < 0) { ... }

That fails too, but this time with the error Cannot assign requested address. I also tried changing the family to AF_INET, but it fails with the same error.

One option remains, which is to bind the sockets to specific IP addresses. I can look up interface addresses and bind to those. Unfortunately, is a bad option, because due to DHCP and hot-plugging ethernet cables, the addresses can change on the fly.

This option may also be bad when it comes to broadcasts and multicasts. I'm concerned that binding to a specific address will mean that I cannot receive broadcasts (which are to an address other than what I bound to). I'm actually going to test this later this evening and update this question.

Questions:

• Is it possible to bind a UDP listening socket specifically to an interface?
• Or alternatively, is there a mechanism I can employ that will inform my program that an interface's address has changed, at the moment that change occurs (as opposed to polling)?
• Is there another kind of listening socket I can create (I do have root privileges) that I can bind to a specific interface, which behaves otherwise identically to UDP (i.e other than raw sockets, where I would basically have to implement UDP myself)? For instance, can I use AF_PACKET with SOCK_DGRAM? I don't understand all the options.

Can anyone help me solve this problem? Thanks!

UPDATE:

Binding to specific IP addresses does not work properly. Specifically, I cannot then receive broadcast packets, which is specifically what I am trying to receive.

UPDATE:

I tried using IP_PKTINFO and recvmsg to get more information on packets being received. I can get the receiving interface, the receiving interface address, the target address of the sender, and the address of the sender. Here's an example of a report I get on receipt of one broadcast packet:

Got message from eth0
Peer address 192.168.115.11
Received from interface eth0
Receiving interface address 10.1.2.47
Desination address 10.1.2.47

What's really odd about this is that the address of eth0 is 10.1.2.9, and the address of ech1 is 10.1.2.47. So why in the world is eth0 receiving packets that should be received by eth1? This is definitely a problem.

Note that I enabled net.ipv4.conf.all.arp_filter, although I think that applies only to out-going packets.

Answer 1

The solution that I found to work is as follows. First of all, we have to change ARP and RP settings. To /etc/sysctl.conf, add the following and reboot (there's also a command to set this dynamically):

net.ipv4.conf.default.arp_filter = 1
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.arp_filter = 1
net.ipv4.conf.all.rp_filter = 2

The arp filter was necessary to allow responses from eth0 to route over a WAN. The rp filter option was necessary to strictly associate in-coming packets with the NIC they came in on (as opposed to the weak model that associates them with any NIC that matches the subnet). A comment from EJP led me to this critical step.

After that, SO_BINDTODEVICE started working. Each of two sockets was bound to its own NIC, and I could therefore tell which NIC a message came from based on the socket it came from.

s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
rc=setsockopt(s, SOL_SOCKET, SO_BINDTODEVICE, nic, IF_NAMESIZE);
memset((char *) &si_me, 0, sizeof(si_me));
si_me.sin_family = AF_INET;
si_me.sin_port = htons(LISTEN_PORT);
si_me.sin_addr.s_addr = htonl(INADDR_ANY);
rc=bind(s, (struct sockaddr *)&si_me, sizeof(si_me))

Next, I wanted to respond to in-coming datagrams with datagrams whose source address is that of the NIC the original request came from. The answer there is to just look up that NIC's address and bind the out-going socket to that address (using bind).

s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
get_nic_addr(nics, (struct sockaddr *)&sa)
sa.sin_port = 0;
rc = bind(s, (struct sockaddr *)&sa, sizeof(struct sockaddr));
sendto(s, ...);

int get_nic_addr(const char *nic, struct sockaddr *sa)
{
    struct ifreq ifr;
    int fd, r;
    fd = socket(AF_INET, SOCK_DGRAM, 0);
    if (fd < 0) return -1;
    ifr.ifr_addr.sa_family = AF_INET;
    strncpy(ifr.ifr_name, nic, IFNAMSIZ);
    r = ioctl(fd, SIOCGIFADDR, &ifr);
    if (r < 0) { ... }
    close(fd);
    *sa = *(struct sockaddr *)&ifr.ifr_addr;
    return 0;
}

(Maybe looking up the NIC's address every time seems like a waste, but it's way more code to get informed when an address changes, and these transactions occur only once every few seconds on a system that doesn't run on battery.)