The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing | Akidau et al. (Google)

Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernandez-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, ́ Frances Perry, Eric Schmidt, Sam Whittle; The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing; In Proceedings of the Conference on Very Large Data Bases (VLDB), Volume 8, Number 12; 2015-08-31; 12 pages; Google, paywall


Unbounded, unordered, global-scale datasets are increasingly common in day-to-day business (e.g. Web logs, mobile usage statistics, and sensor networks). At the same time, consumers of these datasets have evolved sophisticated requirements, such as event-time ordering and windowing by features of the data themselves, in addition to an insatiable hunger for faster answers. Meanwhile, practicality dictates that one can never fully optimize along all dimensions of correctness, latency, and cost for these types of input. As a result, data processing practitioners are left with the quandary of how to reconcile the tensions between these seemingly competing propositions, often resulting in disparate implementations and systems.

We propose that a fundamental shift of approach is necessary to deal with these evolved requirements in modern data processing. We as a field must stop trying to groom unbounded datasets into finite pools of information that eventually become complete, and instead live and breathe under the assumption that we will never know if or when we have seen all of our data, only that new data will arrive, old data may be retracted, and the only way to make this problem tractable is via principled abstractions that allow the practitioner the choice of appropriate tradeoffs along the axes of interest: correctness, latency, and cost.

In this paper, we present one such approach, the Dataflow Mode , along with a detailed examination of the semantics it enables, an overview of the core principles that guided its design, and a validation of the model itself via the real-world experiences that led to its development

Separately noted.


Big Other: Surveillance Capitalism and the Prospects of an Information Civilization | Zuboff

Shoshana Zuboff (Harvard); Big Other: Surveillance Capitalistm and the Prospects of an Information Civilization; In Journal of Information Technology; Issue 30; 2015-04-04; pages 75–89 (15 pages). doi:10.1057/jit.2015.5; landing, ssrn:2594754.

tl;dr → <quote>Google is to surveillance capitalism what General Motors was to managerial capitalis</quote>; Hal Varian is an apologist; surveillance subverts capitalism (which is old, storied, normative and valid).  What will happen?  Only time will tell.


This article describes an emergent logic of accumulation in the networked sphere, ‘surveillance capitalism,’ and considers its implications for ‘information civilization.’ The institutionalizing practices and operational assumptions of Google Inc. are the primary lens for this analysis as they are rendered in two recent articles authored by Google Chief Economist Hal Varian. Varian asserts four uses that follow from computer-mediated transactions: ‘data extraction and analysis,’ ‘new contractual forms due to better monitoring,’ ‘personalization and customization,’ and ‘continuous experiments.’ An examination of the nature and consequences of these uses sheds light on the implicit logic of surveillance capitalism and the global architecture of computer mediation upon which it depends. This architecture produces a distributed and largely uncontested new expression of power that I christen: ‘Big Other.’ It is constituted by unexpected and often illegible mechanisms of extraction, commodification, and control that effectively exile persons from their own behavior while producing new markets of behavioral prediction and modification. Surveillance capitalism challenges democratic norms and departs in key ways from the centuries-long evolution of market capitalism.


  • Big Other is a made up word, constructed for effect; to sound ominous.
  • Hal Varian
    • chief economist, Google
    • is an apologist
    • The Four Uses of Computers (quoted)
      1. data extraction and analysis
      2. new contractual forms due to better monitoring
      3. personalization and customization
      4. continuous experiments
  • Eric Schmidt
    • Google
  • A tour of the opinements of others strung together as a directional argument.
  • A Google hit piece purveyed as muckraking in what passes for academic sensemaking.
  • Google is bad.


<quote>Will surveillance capitalism be the hegemonic logic of accumulation in our time, or will it be an evolutionary dead-end that cedes to other emerging information-based market forms? What alternative trajectories to the future might be associated with these competing forms? I suggest that the prospects of information civilization rest on the answers to these questions. There are many dimensions of surveillance capitalism that require careful analysis and theorization if we are to reckon with these prospects. One obvious dimension is the imbrication of public and private authority in the surveillance project. <snip/> Another key set of issues involves the relationship of surveillance capitalism – and its potential competitors – to overarching global concerns such as equality and climate disruptions that effect all our future prospects. A third issue concerns the velocity of social evolution compared to that at which the surveillance project is institutionalized. </quote>


  • Big Other
  • surveillance capitalism


Lots of newspaper trend pices & oped essays…

Shoshana Zuboff is

    • Charles Edward Wilson Professor of Business Administration (Emerita), Harvard University

Faculty Associate at the Berkman Center for Internet and Society, Harvard Law School.


  • Master or Slave? The Fight for the Soul of Our Information Civilization (forthcoming, Public Affairs and Eichborn 2016).
  • In the Age of the Smart Machine: The future of work and power
  • The Support Economy: Why corporations are failing individuals and the next episode of capitalism

What Happens To Privacy When The Internet Is In Everything? | TechCrunch

What Happens To Privacy When The Internet Is In Everything?; ; In TechCrunch; 2015-01-25.

Original Sources

Harvard professors warn ‘privacy is dead’ and predict mosquito-sized robots that steal samples of your DNA; Afp, Mark Prigg; In The Daily Mail; 2014-01-22.

  • Researchers told Davos that privacy is already non existent
  • Say technology will allow governemtns and insurance firms to steal DNA
  • Also claims the same technology could help eradicate disease


  • Margo Seltzer, professor, Harvard.
    • <paraphrased>privacy is effectively dead</paraphrased>
    • <quote>Privacy as we knew it in the past is no longer feasible… How we conventionally think of privacy is dead.</quote>
  • Joseph Nye, professor, Harvard.
  • Sophia Roosth, (postdoc?) Harvard.
  • Rodney Brooks, Rethink Robotics
  • Anthony Goldbloom, self.


  • World Economic Forum, Davos.
  • Eric Schmidt, Google
  • Edith Ramirez, Chairwoman, Federal Trade Commissino (FTC)
    • Quoted at CES.
  • Deep Mind, bought by Google

Silicon Valley’s Paydays Are Outrageous—So, Where’s the Outrage? | James Kwak

Silicon Valley’s Paydays Are Outrageous—So, Where’s the Outrage?; ; In The Atlantic; 2014-02.21.
Teaser: Lavishing executives with $20 million—or $100 million—is pointless and wasteful, whether the CEOs live in the Financial District or Mountain View.

Original Sources

Introducing a Theory of Creepy | Tene, Polonetsky

Introducing a Theory of Creepy; Omer Tene, Jules Polonetsky; In Re/code; 2014-04-18.

Original Sources


  • <quote>The word “creepy” has become something of a term of art in privacy policy to denote situations where the two [social values, technical capabilites] do not line up</quote>
  • <quote>“There is what I call the creepy line. The Google policy on a lot of things is to get right up to the creepy line and not cross it.”</quote> attributed to Eric Schmidt.

Silicon Valley’s Youth Problem | NYT

Silicon Valley’s Youth Problem; Yiren Lu; In The New York Times; 2014-03-12.

The End Of Anonymity | Popular Science

Erik Sofge; The End Of Anonymity; In Popular Science; 2014-01-15.
Teaser: Technology that matches faces to names can already single out criminals. What happens when it can identify anyone?

Surveillance Valley | Yasha Levine, Pando Daily

; Surveillance Valley; the tag, the running series; In Her Venue, Pando Daily; 2013-12 -> 2014-01

Secularizing the Tech Debate | Dissent Magazine

Geoff Shullenberger; Secularizing the Tech Debate; In Dissent Magazine; 2013-11-11.

Book reviews

Original Sources

  • Jaron Lanier; Who Owns the Future?; Simon & Schuster, 2013, 396 pp.
  • Evgeny Morozov; To Save Everything, Click Here: The Folly of Technological Solutionism; Public Affairs, 2013, 415 pp.
  • Nicholas Carr; The Shallows

The Coming Tech-Lash | The Economist

Adrian Wooldrich; The Coming Tech-Lash; In The Economist; 2013-11-18.
Teaser: The tech elite will join bankers and oilmen in public demonology, predicts Adrian Wooldridge


As a litany of an indictment.

  • <quote>So far they have succeeded in protecting themselves from the tax authorities and shareholders alike.</quote>
  • <quote>Geeks have turned out to be some of the most ruthless capitalists around.</quote>
  • <quote>They employ remarkably few people</quote>
  • <quote>At the same time the tech tycoons have displayed a banker-like enthusiasm for hoovering up public subsidies and then avoiding taxes.</quote>
  • <quote>But tech giants have structured their businesses so that they give as little back as possible.</quote>
  • <quote>Top techies are upping their profile in politics. This is partly by design: they are employing an army of Washington lobbyists to advance their interests.</quote>