Integration Through Messaging Systems – Part 2

In case you want to catch-up or read the previous post in the series, here’s a link Click Here.

Part2: Application Layer Protocols

In Part 1 of our post, we discussed how application layer protocols use a transport protocol internally and implement data transfers without the users’ knowledge of the details underneath it. TCP or UDP (Transport Layer Protocols) implementation would only transfer one packet at a time, while the actual transfer of users’ data would certainly take up a lot more packets. Whether a user is transferring a file or a stream of data, it would still take some wrappers to package them into datagrams to prepare for the transfer. Usage of a transport layer protocol necessitated the users to build programs that would keep the status of transfer, a live connection, handshakes of control data between client and server, and the transfer of the actual data. As the usage of transmission protocols rose, the user community started to realize they were using the same tool for different use-cases. Some use-cases were carrying critical data that needed confirmation, while some carried data that is only critical within a certain amount of time. Sometimes, there was a large amount of data that needed to be transferred, other times, only a handful of signals were enough.

In this post, let us dive into how an application protocol, FTP, solves the use-case of a file transfer. We will investigate the various steps of the internal workings of FTP (File Transfer Protocol) and how this application layer protocol stitches together the various steps of the data transfer to accomplish the task.

FTP is a protocol that uses TCP for the actual transfer. FTP implements the file transfer using two TCP connections under the cover. While one connection is used exclusively to transmit and receive control information, the other is used for the data transfer.

In the current age, users transfer data through a multitude of Graphical User Interfaces (GUIs) that masks almost everything that happens underneath it. You would probably not realize that an FTP protocol was used at all. In the screenshot below you will find how early users of FTP protocol operated, and following the screenshot is a step-by-step process that FTP, as an Application Layer Protocol, orchestrates the transfer.

As I typed in the command to start an FTP session, ftp speedtest.telet2.net, the following steps would kick-off behind the scenes

• An ARP protocol would be used to broadcast the FTP server name, so that client would get the address to establish a TCP connection with. The purpose of the ARP broadcast is to send a request to all the hosts, and the host that recognizes its address would reply to the client, with the physical address for TCP to connect with. A DNS service is invoked in the process to get the real address of the server. Sometimes there would be a group of servers (mirror-sites) that are ready to connect
• When a response is received by the client, a TCP connection is initiated with the server’s address, through a typical TCP handshake to confirm the connection
• After the connection is established, to perform any transfer, client would authenticate, in this case, authenticated as an anonymous user
• On successful authentication, the FTP establishes two TCP connections each on port 21 and port 20. Port 21 is used for control messages that work on keeping the status of connection and the file transfer, while the connection on port 20 is used for transfer of file data.
• As you could see, the next few commands were issued to list the files on the remote server and set the mode of transfer to ASCII
• On the command get 1KB.zip, the file transfer was done; needless to say, the process of converting the file content to datagrams and the process of transfer of each of those datagrams from Server to Client was done
• And on command to quit (not shown in the screenshot), the FTPs session gets disconnected

The illustration above is to drive home the point that an application layer protocol’s implementation, by the nature of its abstraction, stitches together a functionality that makes life easy for a developer. In this case, the FTP implementation had to go through multiple stages of the transfer, and it abstracted the actual implementation by taking simple commands from the user.

While each application Layer protocol serves a specific purpose, it helps to understand how each of these fit into Systems Integration needs. Sometimes these protocols differ by the purpose they serve and sometimes they differ by the quality of service they provide. Some are good for interoperability, support Open standards, while others are good at security. In fact, there would be more than one market vendor that implements the same protocol but differs in the add-ons they provide to their customers. We at Prowess can help you remove the clutter to find the right fit for your needs.

In our next post, we shall go beyond the underlying protocols and discuss the messaging systems that are available in market and delve into them.

Pradeep Tallogu
_{Enterprise Architect}

Pradeep Tallogu brings numerous years of experience in building Enterprise Application Integration systems. He is a seasoned Integration Architect, takes ownership of building complex integration systems from scratch. He works with project stakeholders and technical teams and forms an effective liaison between the two. 
 
At Prowess Software, Pradeep works in Architecture Services. He works closely with Architects, Team Leads and PMs on full lifecycle of software development. He is also involved in building Technical Competency within the company. He constantly challenges his teams, prepares them for what is to come and strives to bring the best out of everyone. He is a team builder, mentor for people around him. Above all, he is a life-long learner. 

Linked In