Wireshark TCP Debugging

GProbe, my C/C++ port scanner project which uses a raw sockets implementation, has moved on to the next step. The TCP SYN scan functionality finally works. But there were some pitfalls until that point.

What to do, when your peer doesn’t reply

As soon as I accomplished the task of getting my tool to send TCP SYN packets to the peer, I noticed that I get no TCP SYN,ACK reply from it. After indepth analysis, I found out that the TCP header checksums were wrong. If you send handcrafted ip datagrams around, you have to make sure, that the TCP header checksum is valid. If it isn’t, it won’t be processed by your peer (and you never get a SYN/ACK TCP response packet.

Wireshark offers a TCP feature which verifies TCP header checksums. Make sure you have this feature enabled. On some system, it is disabled by default. This is a handy feature, when it goes to debugging your code. It is called “Validate TCP checksums if possible” and you’ll find it in the “Preferences” menu on the TCP protocol page.  After enabling this feature, Wireshark will verify the checksums for you.

Another thing which can be confusing when working with Wireshark ist the fact, that Wireshark does not show you the real TCP sequence numbers and TCP acknowledgment numbers which are important. In my case it was especially important for the TCP Three-Way-Handshake. Wireshark shows relative sequence numbers by default. For me it was far easier to let him show the REAL absolute sequence numbers. It made debugging for me a lot easier. You can set this feature on the TCP protocol page too. See the screenshot below which shows you which options I mean:

I then ran Wireshark a couple of times, corrected some and finally got my peer to send a proper response. You can see it in the picture below. Please note, that this is not a complete Three-Way-Handshake according to the RFCs. Since I am implementing a TCP SYN port scanner, the final ACK packet is not of interest to me. Therefore I do not generate it. After the close() call to the socket, a TCP RST packet ist sent to the peer to close the socket (this is the red line in the capture example below):

Since the Three-Way-Handshake is well documented, I won’t discuss that here. Should you have any questions, just ask 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *