300 lines
9.8 KiB
HTML
300 lines
9.8 KiB
HTML
<style>
|
|
body {
|
|
font-size: 12pt;
|
|
font-family: Arial;
|
|
max-width: 40em;
|
|
margin: 1em;
|
|
}
|
|
|
|
a {
|
|
color: inherit;
|
|
}
|
|
|
|
a:hover {
|
|
color: red;
|
|
}
|
|
|
|
table td {
|
|
min-width: 10em;
|
|
vertical-align: top;
|
|
padding: 0.5em;
|
|
}
|
|
|
|
table td {
|
|
border-bottom: 1px solid #bbb;
|
|
}
|
|
|
|
table tr:first-child td {
|
|
border-top: 1px solid #bbb;
|
|
}
|
|
</style>
|
|
<h1>Asynchronous I/O in Windows for UNIX Programmers</h1>
|
|
|
|
<p>Ryan Dahl ry@tinyclouds.org
|
|
|
|
<p>This document assumes you are familiar with how non-blocking socket I/O
|
|
is done in UNIX.
|
|
|
|
<p>Windows has very different notions for how asynchronous and non-blocking I/O
|
|
are done. While Windows has <code>select()</code> it supports only 64
|
|
file descriptors. Obviously Microsoft does understand how to make
|
|
high-concurrency servers, they've simply choosen a different paradigm for
|
|
this called <a
|
|
href="http://msdn.microsoft.com/en-us/library/ms686358(v=vs.85).aspx">overlapped
|
|
I/O</a>. The mechanism in Windows by which multiple sockets are polled
|
|
for completion is called
|
|
<a href="http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx">I/O
|
|
completion ports</a>. More or less equivlant to <a
|
|
href="http://en.wikipedia.org/wiki/Kqueue">kqueue</a> (Macintosh,
|
|
FreeBSD, other BSDs), <a href="http://en.wikipedia.org/wiki/Epoll">epoll</a>
|
|
(Linux), <a
|
|
href="http://developers.sun.com/solaris/articles/event_completion.html">event
|
|
completion ports</a> (Solaris), <a href="">poll</a> (modern UNIXes), or <a
|
|
href="http://www.kernel.org/doc/man-pages/online/pages/man2/select.2.html">select</a>
|
|
(all operating systems). The main difference is that in UNIX you ask the
|
|
kernel to wait for file descriptors to change their readability or
|
|
writablity while in windows you wait for asynchronous functions to complete.
|
|
For example, instead of waiting for a socket to become writable and then
|
|
<a
|
|
href="http://www.kernel.org/doc/man-pages/online/pages/man2/write.2.html"><code>write(2)</code></a>
|
|
to it, as you do in UNIX operating systems, you rather <a
|
|
href="http://msdn.microsoft.com/en-us/library/ms742203(v=vs.85).aspx"><code>WSASend()</code></a>
|
|
a buffer and wait for it to have been sent.
|
|
The result is that non-blocking <code>write(2)</code> and <code>read(2)</code>
|
|
are non-portable to Windows. This tends to throw the poor sap assigned with
|
|
the job of porting your app to Windows into compulsive nervous twitches.
|
|
|
|
<p>
|
|
Almost every socket operation that you're familar with has an
|
|
overlapped counter-part (<a href="#table-foot">see table</a>).
|
|
|
|
|
|
<p id="table-foot">
|
|
<table cellspacing=0>
|
|
<!-- TODO: links -->
|
|
<tr>
|
|
<td></td>
|
|
<td>
|
|
<pre>int fd;</pre>
|
|
</td>
|
|
<td>
|
|
<pre>HANDLE handle;</pre>
|
|
<pre>SOCKET socket;</pre>
|
|
(the two are the same type)
|
|
</tr>
|
|
<tr>
|
|
<td>socket or pipe</td>
|
|
<td>
|
|
<code>send(2)</code>,
|
|
<code>write(2)</code>
|
|
</td>
|
|
<td>
|
|
<a href="http://msdn.microsoft.com/en-us/library/ms742203(v=vs.85).aspx"><code>WSASend()</code></a>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>socket or pipe</td>
|
|
<td>
|
|
<code>recv(2)</code>,
|
|
<code>read(2)</code>
|
|
</td>
|
|
<td>
|
|
<a href="http://msdn.microsoft.com/en-us/library/ms741688(v=VS.85).aspx"><code>WSARecv()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>socket</td>
|
|
<td>
|
|
<pre>connect(2)</pre>
|
|
Non-blocking <code>connect()</code> is has difficult semantics in
|
|
UNIX. The proper way to connect to a remote host is this: call
|
|
<code>connect(2)</code> which will usually return <code>EAGAIN</code>.
|
|
Poll on the file descriptor for writablity. Then use
|
|
<pre>int error;
|
|
socklen_t len = sizeof(int);
|
|
getsockopt(fd, SOL_SOCKET, SO_ERROR, &error, &len);</pre>
|
|
The <code>error</code> should be zero if the connection succeeded.
|
|
(Documented in <code>connect(2)</code> under <code>EINPROGRESS</code>
|
|
on the Linux man page.)
|
|
</td>
|
|
<td>
|
|
<a href="http://msdn.microsoft.com/en-us/library/ms737606(VS.85).aspx"><code>ConnectEx()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>pipe</td>
|
|
<td>
|
|
<pre>connect(2)</pre>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/aa365146(v=VS.85).aspx"><code>ConnectNamedPipe()</code></a>
|
|
|
|
Be sure to set <code>PIPE_NOWAIT</code> in <code>CreateNamedPipe()</code>
|
|
</td>
|
|
</tr>
|
|
|
|
|
|
<tr>
|
|
<td>socket</td>
|
|
<td>
|
|
<pre>accept(2)</pre>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/ms737524(v=VS.85).aspx"><code>AcceptEx()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>pipe</td>
|
|
<td>
|
|
<pre>accept(2)</pre>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/aa365146(v=VS.85).aspx"><code>ConnectNamedPipe()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
|
|
<tr>
|
|
<td>file</td>
|
|
<td>
|
|
<code>write(2)</code>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/aa365748(v=VS.85).aspx"><code>WriteFileEx()</code></a>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>file</td>
|
|
<td>
|
|
<code>read(2)</code>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/aa365468(v=VS.85).aspx"><code>ReadFileEx()</code></a>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>socket and file</td>
|
|
<td>
|
|
<code>sendfile()</code> [<a href="#sendfile-foot">1</a>]
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/ms740565(v=VS.85).aspx"><code>TransmitFile()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>tty</td>
|
|
<td>
|
|
<a
|
|
href="http://www.kernel.org/doc/man-pages/online/pages/man3/tcsetattr.3.html"><code>tcsetattr(3)</code></a>
|
|
</td>
|
|
<td>
|
|
<a href="http://msdn.microsoft.com/en-us/library/ms686033(VS.85).aspx"><code>SetConsoleMode()</code></a>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>tty</td>
|
|
<td>
|
|
<code>read(2)</code>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/ms684958(v=VS.85).aspx"><code>ReadConsole()</code></a>
|
|
and
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/ms684961(v=VS.85).aspx"><code>ReadConsoleInput()</code></a>
|
|
do not support overlapped I/O and there are no overlapped
|
|
counter-parts. One strategy to get around this is
|
|
<pre><a
|
|
href="http://msdn.microsoft.com/en-us/library/ms685061(VS.85).aspx">RegisterWaitForSingleObject</a>(&tty_wait_handle, tty_handle,
|
|
tty_want_poll, NULL, INFINITE, WT_EXECUTEINWAITTHREAD |
|
|
WT_EXECUTEONLYONCE)</pre>
|
|
which will execute <code>tty_want_poll()</code> in a different thread.
|
|
You can use this to notify the calling thread that
|
|
<code>ReadConsoleInput()</code> will not block.
|
|
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>tty</td>
|
|
<td>
|
|
<code>write(2)</code>
|
|
</td>
|
|
<td>
|
|
<a
|
|
href="http://msdn.microsoft.com/en-us/library/ms687401(v=VS.85).aspx"><code>WriteConsole()</code></a>
|
|
is also blocking but this is probably acceptable.
|
|
</td>
|
|
|
|
</tr>
|
|
|
|
</table>
|
|
|
|
<p id="sendfile-foot">[1] <code>sendfile()</code> on UNIX has not been agreed
|
|
on yet. Each operating system has a slightly different API.
|
|
|
|
|
|
<p id="foot2">
|
|
<p>
|
|
tips
|
|
<ul>
|
|
<li> overlapped = non-blocking.
|
|
<li> There is no overlapped <a href="http://msdn.microsoft.com/en-us/library/ms738518(VS.85).aspx"><code>GetAddrInfoEx()</code></a> function. It seems Asynchronous Procedure Calls must be used instead.
|
|
<li> <a href=http://msdn.microsoft.com/en-us/library/ms740673(VS.85).aspx"><code>Windows Sockets 2</code></a>
|
|
</ul>
|
|
|
|
<p>
|
|
IOCP:
|
|
<ul>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms686358(v=vs.85).aspx">Synchronization and Overlapped Input and Output</a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms741665(v=VS.85).aspx"><code>WSAOVERLAPPED</code> Structure</a>
|
|
<ul>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms683209(v=VS.85).aspx"><code>GetOverlappedResult()</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms683244(v=VS.85).aspx"><code>HasOverlappedIoCompleted()</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/aa363792(v=vs.85).aspx"><code>CancelIoEx()</code></a>
|
|
— cancels an overlapped operation.
|
|
</ul>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms742203(v=vs.85).aspx"><code>WSASend()</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms741688(v=VS.85).aspx"><code>WSARecv()</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms737606(VS.85).aspx"><code>ConnectEx()</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms740565(v=VS.85).aspx"><code>TransmitFile()</code></a>
|
|
— an async <code>sendfile()</code> for windows.
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms741565(v=VS.85).aspx"><code>WSADuplicateSocket()</code></a>
|
|
— describes how to share a socket between two processes.
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/6e3b887c.aspx"><code>_setmaxstdio()</code></a>
|
|
— something like setting the maximum number of file decriptors
|
|
and <a
|
|
href="http://www.kernel.org/doc/man-pages/online/pages/man2/setrlimit.2.html"><code>setrlimit(3)</code></a>
|
|
AKA <code>ulimit -n</code>. Note the file descriptor limit on windows is
|
|
2048.
|
|
</ul>
|
|
|
|
<p>
|
|
APC:
|
|
<ul>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms681951(v=vs.85).aspx">Asynchronous Procedure Calls</a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/ms682016"><code>DNSQuery()</code></a>
|
|
— General purpose DNS query function like <code>res_query()</code> on UNIX.
|
|
</ul>
|
|
|
|
|
|
Pipes:
|
|
<ul>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/aa365781(v=VS.85).aspx"><code>Pipe functions</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/aa365150(VS.85).aspx"><code>CreateNamedPipe</code></a>
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/aa365144(v=VS.85).aspx"><code>CallNamedPipe</code></a>
|
|
— like <code>accept</code> is for UNIX pipes.
|
|
<li><a href="http://msdn.microsoft.com/en-us/library/aa365146(v=VS.85).aspx"><code>ConnectNamedPipe</code></a>
|
|
</ul>
|