In a quandary

I happened to stumble upon something, which has sure put me in a confused state. This was what I was arguing in the class, that since the TCP layer is on the kernel side of the stack [referring to figure 1.14 of Unix Network Programming], the kernel itself establishes connections [the 3-way handshake] and then queues it up, without sending the SYN packets up to the application. This whole thing initiated when I read a post on a forum I am frequenting [btw its the Linux forum on Orkut]. The user in question had to build a telnet-server which was to be serializing connections and not servicing multiple concurrent requests. But the server wasn't showing the right behavior! While a client was being serviced, another one could get connected [3-way handshake again]. What follows is the question the person asked, and a subsequent reply that threw me in a quandary: "-- I dont get it since a conn. can be established only when I accept(), right ? No you are mistaken. When someone tries to connect to a listening socket, they are not prevented. In fact, a TCP 3-way hand shake is done by the TCP/IP stack in the kernel. (This is independent of whether or not u call accept). This connection for which a TCP/IP 3 way hand shake is completed, is Queued by the Kernel to give it to u when u call accept. This is the same Queue, whose length you have specified in the listen call." And then when I searched around on the internet I came across a Secure Kernel paper by Ramakrishnan Lakshminarasimhan that said: "TCP handshake that involves the exchange of SYN packets takes place during the connect and accept calls of the client and server respectively. A successful 3-way handshake establishes a TCP connection between the client and server."

-- Kernel-based Network Security, Ramakrishnan Lakshminarasimhan 2002

Yet another PDF that has been describing sockets API in some detail had this to say:

"Establishing a passive mode TCP socketPassive mode:– Address already determined.– Tell the kernel to accept incoming connection requests directed at the socket address.• 3-way handshake– Tell the kernel to queue incoming connections for us."

-- TCP Sockets, Network Programming

From what I gather it looks like that the 3-way handshake takes place between the connect and accept calls. This works fine with that guy's problem too, and has cleanly identified the user-process and kernel segments of the OSI layers.

Now I just wanted to clarify my mind about the workings of this. What if the server doesn't want to accept a connection? Does this not result in a bottleneck, deadlocking all the requests behind? Or could a malicious request bypass the process [since the TCP connection is already established]?

Hoping that I get enlightened all of a sudden!