Make UDP communication more reliable in NodeJS
UDP ( User Datagram Protocol)
As Wikipedia says, UDP is a protocol that executes on internet protocol (IP) and it does not need any prior message passing to set up any communication channel or data path. Because of the minimum protocol mechanism, it does not have handshaking dialogues and there is no guarantee of delivery, ordering, or duplicate protection. But it has checksum for data integrity.
UPD message passing is heavily used in distributed system applications. But we cannot use row UDP, because it does not have any guarantee of delivery. Today we are going to add some reliability mechanisms for UPD message passing with NodeJS example.
You cannot send JSON or any other data through the UPD message. But you can stringify the JSON object and send it.
Before going any further, let me explain what we are going to do here. First, we create a datagram socket by using the package called dgram which is delivered with NodeJS (no npm install needed). Then bind a port to our socket and make it listen. Meanwhile, we will create another server (actually a client in this case. we use this one to send the messages) same as this one which is listening on a different port and sending a simple string message to the server that we created earlier.
As our application is to build a distributed system communication protocol, we should have only one application (same code). Therefore in my example, I am going to explain everything by using a single file but running in different port numbers. To execute the same code in a different manner I am going to use process.arvg an array to get parameters. Let's see how it going to work.
const dgram = require('dgram');
const udpServer = dgram.createSocket('udp4');
udpServer.on('listening', () => {
console.log('UDP server is listing...');
});
const init = (port) => {
udpServer.bind(port);
udpServer.on('message', (msgStream, rinfo) => {
console.log('msg is', msgStream.toString());
console.log('rinfo', rinfo);
});
};
const send = (target) => {
const data = 'HELLO WORLD.. !';
udpServer.send(data, 0, data.length, target, '127.0.0.1', (err, len) => {
console.log('error', err);
console.log('sent length', len);
});
};
// when you run this file make sure these parameter to be passed
// myPort, <s, c>, target (if c)
init(parseInt(process.argv[2]));
if (process.argv[3] === 'c') {
console.log('send message');
send(parseInt(process.argv[4]));
}I assume you know about the process.argv array. It is used for controlling the server as we want. When executing the file, the argument order is, 1) listing port number. 2) whether it is server or client by ‘s’ or ‘c’. 3) target port number. See the sample executing example below. Save the file server.js go to the path in the terminal and execute the following commands.
node server.js 3000 s # this will start one serverNow open up another terminal window and execute the following command
node server.js 4000 c 3000 # this will start the client and send a messageWhat actually happens here is the first command creates a server on port number 3000 and the second command creates a server on port number 4000. Then the second one (let's call it the client) sends a message to port 3000. We are having 3 command line arguments while executing the command and if you want to get more complex data you can have packages like minimis . At last, I will give you the full source code with those features. But as for now, we are just going to build the communications only.
Until now, we are sending a text messages. Let's make this more complex by sending JSON objects. Actually, we are using the same thing but the JSON which is converted to a string by JSON.stringify() Then we can easily manage the data we are sending.
Now define the structure of data packets passing through the network.
UPD_Packet : {
id -> unique id for each message
version -> version number
body -> message body(can be json or string)
type -> type of the udp packet (RES, ACK, REQ)
}id attribute represents a randomly generated id to uniquely identify a request and serve the response. If you want you can use packages uuid to get a randomized id for the messages.
Furthermore, the type attribute represents what kind of message is this.
REQ : Request ( client to server)
ACK : Acknowledgement ( server to client / acknowledging the
request is received)
RES : Response ( server to client / response message for a
particular request)Let’s discuss how this protocol going to be worked.
First, the client node makes a request ( REQ ) for a server node and waiting for an ACK from the server. In this example, we are sending the same request message with an increasing version number for every ‘500ms’ until an ACK receive it. It will check 5 times and in that time interval the acknowledgement is not received then an error will be thrown as Timeout .
After the message is sent, we bind the callback function and other useful information to the map (normal object) call responseHandlersMap with an id that we create when we send the request as the key.
In the server, the message is converted into JSON format and checks what type of message is received.
If it is REQ , the server sends an ACK to the client to indicate the message is successfully taken by the server. After that, we will create two objects that are carrying data about requests and responses. In the request object has body and the rinfo properties that have message body and sender information respectively. The response body has only one function called json to end the request by passing the response to the client. Inside that function, we take one argument that has the response of the server and wraps it inside UPD_Packet what we mentioned above. Then call the send method to send the stringified object to the client.
If the message type is ACK , it can be the acknowledgement of a request we made before this from this particular node. So our responseHandlersMap should have an entry for that request. Find it by its id and mark it as the acknowledgement is received (resHandler.ack = true;). As we are sure the server has received the request, we are not going to send requests anymore. To do so, stop calling the setTimout() method by clearing the timeout.
If the message type is RES , Find the proper entry from the responseHandlersMap and execute the handler function with the received data. If you look into the code well, you may see an attribute call done is set to true . This is because stop calling the same method over and over for every response.
It is enough reading…. Just see the code.
const dgram = require('dgram');
const udpServer = dgram.createSocket('udp4');
udpServer.on('listening', () => {
console.log('UDP server is listing...');
});
const REQUEST_TYPES = {
ACK: 'ACK',
REQ: 'REQ',
RES: 'RES',
};
const responseHandlersMap = {};
init = (port, cb) => {
udpServer.bind(port);
udpServer.on('message', (msgStream, rinfo) => {
const udpStream = JSON.parse(msgStream.toString());
let resHandler = responseHandlersMap[udpStream.id];
// create req, res object to pass handler.
if (udpStream.type === REQUEST_TYPES.REQ) {
// create the acknowledgement
const ack = JSON.stringify({ type: REQUEST_TYPES.ACK, ok: 1, id: udpStream.id });
// send ACK
udpServer.send(ack, 0, ack.length, rinfo.port, rinfo.address);
const request = {
// create request object
body: udpStream.body,
rinfo: rinfo,
};
const response = {
// response object
json: (data) => {
const resString = JSON.stringify({
body: data,
id: udpStream.id,
type: REQUEST_TYPES.RES,
});
udpServer.send(resString, 0, resString.length, rinfo.port, rinfo.address);
},
};
// execute the call back with request and response object.
if (cb) cb(request, response);
} else if (udpStream.type === REQUEST_TYPES.ACK) {
if (resHandler) {
resHandler.ack = true;
resHandler.stopSending();
}
} else if (udpStream.type === REQUEST_TYPES.RES) {
if (resHandler && !resHandler.done) {
resHandler.stopSending();
// not call the response handler two time
resHandler.done = true;
resHandler.handler(null, udpStream, udpStream.body);
}
}
});
};
send = (target, data, cb) => {
// generate a unique random number
const msgId = Math.random().toString().substring(2);
let version = 0; // message version
const fn = () => {
version += 1; // increase message version
if (version > 5) {
// message limit
return cb({ error: 'TIMEOUT' });
}
// message Object
let msg_send = {
id: msgId,
version: version,
body: data,
type: REQUEST_TYPES.REQ,
};
msg_send = JSON.stringify(msg_send);
// send the message
udpServer.send(msg_send, 0, msg_send.length, target.port, target.ip, (a, b) => {
let timeoutRef;
// append a response handler object to the map
responseHandlersMap[msgId] = {
handler: cb,
ack: false,
done: false,
stopSending: () => {
clearTimeout(timeoutRef);
},
};
timeoutRef = setTimeout(() => {
return fn();
}, 500);
});
};
// put an interval to send the message until get an ack
fn();
};
// when you run this file make sure these parameter to be passed
// node server.js myPort <s / c> target (add target only if c)
const myPort = parseInt(process.argv[2]);
const type = process.argv[3];
const serverPort = parseInt(process.argv[4] || '0');
// initialize the server
init(myPort, (req, res) => {
// this is the request handling function.
const { number1, number2 } = req.body;
res.json({ sum: number1 + number2 });
});
// make a request from the client
if (type === 'c') {
send({ port: serverPort, ip: '127.0.0.1' }, { number1: 39, number2: 34 }, (err, upstreamData, response) => {
console.log(response);
});
}You can paste this code segment to a file and execute it by following commands.
Start the server by:
node server.js 3000 s # this will start one serverAnd open a new terminal window and start the client by:
node server.js 4000 c 3000 # this will start the client and send a messageYou should see the response of summing up the two values passed as number1 and number2 from client.
Thanks and hope you enjoyed this.
