Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt.
Some years later this got revisited due to needing to use the same thing under C# on Win10 and while it was better it was still a major embarrassment how big the performance gap was.
asveikau 1 hours ago [-]
Some years back Windows added AF_UNIX sockets, I wonder how those would perform relative to Win32 pipes. My guess is better.
SoftTalker 47 minutes ago [-]
Well POSIX only defines behavior, not performance. Every platform and OS will have its own performance idiosyncracies.
andrewmcwatters 1 hours ago [-]
Did you find that you needed interprocess communication to replace the gap?
johnisgood 3 hours ago [-]
FWIW there is readv() / writev(), splice(), sendfile(), funopen(), and io_buffer() as well.
splice() is great when transferring data between pipes and UNIX sockets with zero-copy, but it is Linux-only.
splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal.
Sadly on BSDs, for pipes, readv() / writev() is the most performant way to achieve the same if I am not mistaken. Please correct me if I am wrong.
At any rate, this is a great article.
messe 2 hours ago [-]
> sendfile() is file-to-socket (zero-copy as well), and has very high performance as well, for both Linux and BSDs. It only supports file-to-socket, however, and well, to stay relevant, sendmsg() can't be used with pipes in the general case, it is for UNIX domain sockets, INET sockets, and other socket types.
On Linux, sendfile supports more than just file to socket, as it's implemented using splice. I've used it for file-to-block-device in the past.
johnisgood 59 minutes ago [-]
On BSDs probably not, as they don't have splice, but that is good to know. I wonder if on BSDs it really is readv() and writev() that are the fastest way to achieve the same thing as has been done in the article. Maybe I am missing something. I would like to be corrected.
messe 50 minutes ago [-]
AFAIK, neither OpenBSD nor NetBSD has sendfile. On FreeBSD, I think you're correct regarding it being file-to-socket only.
zambal 37 minutes ago [-]
Indeed, if I'm not mistaken Netflix at least used to use (and commit to kernel) FreeBSD on content servers because of its superior sendfile performance
Some years later this got revisited due to needing to use the same thing under C# on Win10 and while it was better it was still a major embarrassment how big the performance gap was.
splice() is great when transferring data between pipes and UNIX sockets with zero-copy, but it is Linux-only.
splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal.
Sadly on BSDs, for pipes, readv() / writev() is the most performant way to achieve the same if I am not mistaken. Please correct me if I am wrong.
At any rate, this is a great article.
On Linux, sendfile supports more than just file to socket, as it's implemented using splice. I've used it for file-to-block-device in the past.
https://news.ycombinator.com/item?id=31592934 (200 comments)
https://news.ycombinator.com/item?id=37782493 (105 comments)
I'd like to use splice more, but the end of the article talked about the security implications and some ABI breaking.
I'm curious to know if long term plans are to keep splice around?
I'd also be curious how hard it would be to patch the default pipe to always use splice for performance improvements.