I’m writing a simple network packet mangler in C++ which only changes the TTL value (for now). The main loop is receiving requests from the netfilter queue using recv(fd, buf, sizeof(buf), 0)
and then processes those requests using nfq_handle_packet(h, buf, rv)
. The buffer size is 500000000
bytes, which is about 500 megabytes or 4000 megabits. The internet speed is nowhere close to 4 gigabits (and even if it would be 4 gigabits, I’m not transferring that much data), yet the recv
function keeps on receiving the ENOBUFS
(“No buffer space available” when formatted using strerror) when I, for example, run a speed test on speed.cloudflare.com. I don’t understand how I’m running out of buffer space here. The code that receives the requests is:
// created before in the code: char buf[500000000] = {0};
fd = nfq_fd(h);
while (true) {
rv = recv(fd, buf, sizeof(buf), 0);
if (rv < 0) {
std::cout << "Error in recv: " << strerror(errno) << std::endl;
} else nfq_handle_packet(h, buf, rv);
}
The code that processes the packets is also very simple, I don’t think the reason behind the “No buffer space left” error is slow processing of packets. As I said before, it just changes the TTL (time-to-live) value.
static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nfmsg, struct nfq_data *nfa, void *_data) {
struct nfqnl_msg_packet_hdr *ph;
ph = nfq_get_msg_packet_hdr(nfa);
if (!ph) {
data->packetsRejected += 1;
return -1;
}
int id = ntohl(ph->packet_id);
unsigned char* packetData;
int length = nfq_get_payload(nfa, &packetData);
if (length == -1) {
int result = nfq_set_verdict(qh, id, NF_DROP, 0, NULL);
data->packetsRejected += 1;
return result;
}
uint8_t ipVersion = (packetData[0] & 0b11110000) >> 4;
if (ipVersion == 4) {
uint8_t headerLength = packetData[0] & 0b1111;
if (headerLength < 5 || headerLength > 15) {
int result = nfq_set_verdict(qh, id, NF_ACCEPT, 0, NULL);
data->packetsRejected += 1;
data->bytesRejected += length;
return result;
}
headerLength *= 4;
packetData[8] = 64;
fixIPv4Checksum(packetData, headerLength);
}
int result = nfq_set_verdict(qh, id, NF_ACCEPT, length, packetData);
data->packetsAccepted += 1;
data->bytesAccepted += length;
return result;
}
(before you say “meh google it”, I already did)