libasync (everyone has one) and asio

An Architecture for Internet Data Transfer | Tolia, Kaminsky, Andersen, Patil

Niraj Tolia, Michael Kaminsky, David G. Andersen, Swapnil Patil; An Architecture for Internet Data Transfer; In Proceedings of the Symposium on Networked Systems Design and Implementation (NSDI), in the USENIX sphere; 2006.

tl;dr → A “content-centric networking” concept with cryptographic hashes on blobs of data, which can be transferred out of order and across time & space. Hard to get that out of the paper though.

Abstract

This paper presents the design and implementation of DOT, a flexible architecture for data transfer. This architecture separates content negotiation from the data transfer itself. Applications determine what data they need to send and then use a new transfer service to send it. This transfer service acts as a common interface between applications and the lower-level network layers, facilitating innovation both above and below. The transfer service frees developers from re-inventing transfer mechanisms in each new application. New transfer mechanisms, in turn, can be easily deployed without modifying existing applications.

We discuss the benefits that arise from separating data transfer into a service and the challenges this service must overcome. The paper then examines the implementation of DOT and its plugin framework for creating new data transfer mechanisms. A set of microbenchmarks shows that the DOT prototype performs well, and that the overhead it imposes is unnoticeable in the wide-area. End-to-end experiments using more complex configurations demonstrate DOT’s ability to implement effective, new data delivery mechanisms underneath existing services. Finally, we evaluate a production mail server modified to use DOT using trace data gathered from a live email server. Converting the mail server required only 184 lines-of-code changes to the server, and the resulting system reduces the bandwidth needed to send email by up to 20%.

Mentions

  • No code (this was before the era of githubbery.
    • Only a protocol proposal; no reference implementation.
    • Perhaps there was an experimental implementation, but it is not available.
  • David Mazières, Frank Dabek, Eric Peterson, Thomer M. Gil. Using libasync; Technical Report; PDOS, CSAIL, MIT; 2004.12 pages; pdf; followup separately filled.