To understand how Verse works, and why it has a better network performance than many other solutions, we need to explain how networks work. Any data sent using using the Internet Protocol (IP) is divided into packets. Packets are usually have a size of around 1,500 bytes. When you send larger data you need to split up your data into multiple packets and reassemble them on the receiving end. To do this, TCP (Transmission Control Protocol) was written on top of IP. It takes large sets of data, puts them in a range of numbered packets and sends them out. The receiving end can then reassemble the original data by ordering the packets using their numbers. This would be simple and efficient in a perfect world, but on a real network, especially one as large and complex as the Internet, packets get lost or arrive in the wrong order. To combat this TCP sends back positive and negative acknowledgments for all packets. This allows the sending side to resend any packets that might have gotten lost. and since all packets are numbered it is easy to sort them in the correct order, and then reassemble the data. This approach is fine for many types of data sent over the Internet, such as mail, files (FTP) and web traffic. But in some cases it is not sufficient. If one packet gets lost on the net, the time it takes to tell the sender that a packet was lost and to receive a new copy it may be a problem. Since all packets must be re-assembled in order the entire process stops while waiting for the lost packet, even if other later packets have arrived. This is sometimes known as a "TCP hick up". For non-real time data this doesn't matter much, but if you are dealing with real time video, audio, VR, games or other applications where you want low latency feedback this sudden freeze will be a problem. Therefore some things on the Internet use an other, smaller protocol known as UDP (User Datagram Protocol) that doesn't re-send any packets. In real time applications this is just fine. If you are watching a broadcast of video online, a missed frame is acceptable, or if you are listening to an online radio station a break that lasts a fraction of a second is also often acceptable. Sometimes this type of broadcast is referred to as "streaming". Games work the same way because they put the entire game state in one or very few packets, and if they get lost they know that a new packet will arrive shortly and overwrite the old data anyway. Often streaming solutions have various ways of "smoothing over" lost packets. lost video packets often give stains or smears, audio streams may hide lost packets by replaying the last packet, and games use interpolation and client-side simulation to predict what the game host is doing. | | Verse however is a little bit more complicated. Verse also requires low-latency handling, and can't accept TCP "hick-ups". But on the other hand Verse can send large data sets over a network, and can not just accept lost data like streaming protocols can. Verse also packs commands into numbered packets, and if the receive end notices a lost packet it informs the sender of this so that the sender can resend the data. However the receive end does not (like TCP) wait for the lost packet to arrive before it parses any of the commands in subsequent packets. This can lead to some problems if the same data is written to twice. To combat this problem, Verse uses event compression. Consider this: a command is sent in a packet that tells the receiver that an object has changed color to red. But then shortly after, it sends a new command, in a new packet, that sets the color of the same object to blue. If it now happens that the first packet gets lost and has to be resent, the resent command may arrive after the second command. This will make the two sides data inconsistent since the sending side will claim the object previously was red but is now blue, while the receiving end will claim the that the object was blue but now has been changed to red. To solve this the sender divides the command in two parts, address and data. The address in this case would consist of "change color of object X", and the data part would be "red" or "blue". When the receiver tells the sender that the packet was lost, the sender can compare the commands' address part, with all commands sent since the lost packet. If any of them are the same, it means that the command that was lost has been replaced since it was first sent, and therefore this command will not be re-sent. This technique provides a powerful framework for Verse network traffic, that allows it to be fast and reliable without impacting severely on the application developer. In fact all this handling is hidden in the Verse API implementation, and invisible to the application developer. |