“Information Bottleneck” Theory Cracks Open the Black Box of Deep Learning | Quanta Magazine

New Theory Cracks Open the Black Box of Deep Learning; Natalie Wolchover; In Quanta Magazine, also syndicated out to copied onto Wired.com; 2017-10-09; pdf.
Teaser: A new idea called the “information bottleneck” is helping to explain the puzzling success of today’s artificial-intelligence algorithms — and might also explain how human brains learn.

tl;dr → the “information bottleneck,” an explainer; as the metaphor.
and → <quote><snip/> that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts.</quote>


Phases of Deep Learning

“fitting” or “memorization”
Is shorter (than the longer phase).The network learns labels for training data.
“compression” or “forgetting”
Is longer (than the shorter phase).
The network observes new data, to generalize against it. The network
optimizes (“becomes good at”) generalization, as measured differential with the (new) test data.


  • Alex Alemi, Staff, Google.
    …quoted for color, background & verisimilitude; a booster.
  • William Bialek, Princeton University.
  • Kyle Cranmer, physics, New York University.
    …quoted for color, background & verisimilitude; a skeptic.
  • Geoffrey Hinton,
    …quoted for color, background & verisimilitude; is non-committal, “It’s extremely interesting.”

    • Staff, Google
    • Faculty, University of Toronto
  • Brenden Lake, assistant professor, psychology & data science statistics, New York University.
    In which a data scientist is a statistician who performs statistics on a Macintosh computer in San Francisco; and Prof. Lake’s employer is the university system of the State of New York.
  • Pankaj Mehta
  • Ilya Nemenman, faculty, biophysics, Emory University.
  • Fernando Pereira, staff, Google.
  • David Schwab
  • Andrew Saxe, staff, Harvard University.
    Expertise: Artificial Intelligence, The Theory of The Science of The Study of The Neuron; a.k.a. neuroscience.
  • Ravid Shwartz-Ziv, graduate student, Hebrew University, Jerusalem, IL.
    Advisor: Naftali Tishby
  • Naftali Tishby, Hebrew University, Jerusalem, IL.
  • Noga Zaslavsky, graduate student, Emory Univerity.
    Advisor: Ilya Nemenman.


  • Stuart Russell, éminence grise.
  • Claude Shannon, theorist.


Yes, there were papers referrenced.  See notes..


In archaeological order, in Quanta Magazine

Separately noted..

The Undue Influence of Surveillance Technology Companies on Policing | Elizabeth Joh

Elizabeth E. Joh; The Undue Influence of Surveillance Technology Companies on Policing; In Law Review Online, New York University (NYU); 2017-09; N pages; landing.
Elizabeth E. Joh is Professor of Law, School of Law, U.C. Davis.


Conventional wisdom assumes that the police are in control of their investigative tools. But with surveillance technologies, this is not always the case. Increasingly, police departments are consumers of surveillance technologies that are created, sold, and controlled by private companies. These surveillance technology companies exercise an undue influence over the police today in ways that aren’t widely acknowledged, but that have enormous consequences for civil liberties and police oversight. Three seemingly unrelated examples—stingray cellphone surveillance, body cameras, and big data software—demonstrate varieties of this undue influence. The companies which provide these technologies act out of private self-interest, but their decisions have considerable public impact. The harms of this private influence include the distortion of Fourth Amendment law, the undermining of accountability by design, and the erosion of transparency norms. This Essay demonstrates the increasing degree to which surveillance technology vendors can guide, shape, and limit policing in ways that are not widely recognized. Any vision of increased police accountability today cannot be complete without consideration of the role surveillance technology companies play.


    1. Stingray Cellphone Surveillance and Nondisclosure Agreements
      1. Nondisclosure Agreements
      2. Stingrays and the Fourth Amendment
      3. Secret Stingray Use
    2. Cornering the Market on Police Body Cameras
      1. When Product Design Is Policy
      2. Market Dominance
    3. Big Data Software and Proprietary Information
    1. Fourth Amendment Distortion
    2. Accountability by Design
    3. Outsourcing Suspicion and Obscuring Transparency
    1. Local Surveillance Oversight
    2. Public Records Requests as Oversight

Web of Things (WoT), Architecture, Thing Description, Scripting API | W3C


Web of Things (WoT) Architecture, 2017-09-14.
The “building blocks”

  1. WoT Thing Description
  2. WoT Scripting API
  3. WoT Binding Templates.

Web of Things (WoT) Thing Description, 2017-09-14.
Describes the metadata and interfaces of Things.

Web of Things (WoT) Scripting API, 2017-09-14.
Operates on Things characterized by Properties, Actions and Events.



Separately noted.


Fingerprinting web users through font metrics | Filfield, Egelman

David Filfield, Serge Egelman; Fingerprinting web users through font metrics; In International Conference on Financial Cryptography and Data Security (FC) 2015: Financial Cryptography and Data Security; 2015; Lecture Notes in Computer Science (LNCS, volume 8975); 2015-07-16; pages 107-124;


We describe a web browser fingerprinting technique based on measuring the onscreen dimensions of font glyphs. Font rendering in web browsers is affected by many factors—browser version, what fonts are installed, and hinting and antialiasing settings, to name a few — that are sources of fingerprintable variation in end-user systems. We show that even the relatively crude tool of measuring glyph bounding boxes can yield a strong fingerprint, and is a threat to users’ privacy. Through a user experiment involving over 1,000 web browsers and an exhaustive survey of the allocated space of Unicode, we find that font metrics are more diverse than User-Agent strings, uniquely identifying 34% of participants, and putting others into smaller anonymity sets. Fingerprinting is easy and takes only milliseconds. We show that of the over 125,000 code points examined, it suffices to test only 43 in order to account for all the variation seen in our experiment. Font metrics, being orthogonal to many other fingerprinting techniques, can augment and sharpen those other techniques.

We seek ways for privacy-oriented web browsers to reduce the effectiveness of font metric–based fingerprinting, without unduly harming usability. As part of the same user experiment of 1,000 web browsers, we find that whitelisting a set of standard font files has the potential to more than quadruple the size of anonymity sets on average, and reduce the fraction of users with a unique font fingerprint below 10%. We discuss other potential countermeasures.

The Value of Luxury Poseurs | New Yorker

The Value of Luxury Poseurs; Paul Hiebert; In The New Yorker; 2014-06-27.

tl;dr → [Brand] Tourists OK; [Brand] immigrants not so.
and → yet more social “truth” manufactured through Amazon Mechanical Turk.

Original Sources

Silvia Bellezza, Anat Keinan; Brand Tourists: How Non–Core Users Enhance the Brand Image by Eliciting Pride; In Journal of Consumer Research, Vol. 41, No. 2 (August 2014); pages 397-417 (21 pages); Oxford University Press; doi10.1086/676679; paywall.

  • Silvia Bellezza, [then-] doctoral candidate, Business School, Harvard University.
  • Anat Keinan, professor, marketing, Business School, Harvard University.

This research examines how core consumers of selective brands react when non–core users obtain access to the brand. Contrary to the view that non–core users and downward brand extensions pose a threat to the brand, this work investigates the conditions under which these non–core users enhance rather than dilute the brand image. A distinction between two types of non–core users based on how they are perceived by current users of core products is introduced: “brand immigrants” who claim to be part of the in-group of core users of the brand and “brand tourists” who do not claim any membership status to the brand community. A series of studies show that core consumers respond positively to non–core users when they are perceived as brand tourists. The brand tourism effect is mediated by core users’ pride and moderated by brand patriotism and selectiveness of the brand.


  • Brand Citizen (an archetype model)
  • Brand Immigrants
  • Brand Tourists
  • Core  Users
  • Selective brands
  • Brand patriotism
  • anger, Core User Anger
  • pride, Core User Pride
  • Prestige image
  • Something about Brand Extensions
  • Brand Emigrants
    <quote>the “brand emigrants”: those who could claim in-group status but willingly decided not to claim it. These would include, for instance, a consumer who owns a Ferrari car but decides to replace it with a different luxury sports car or a full-time undergraduate student at Harvard who transfers to another institution (e.g., MIT) to complete the degree. Brand emigrants inspire negative reactions from core users of the brand, just as citizens might feel betrayed by compatriots who decided to leave the country and live elsewhere.</quote>
  • Something about Brand Counterfeits.

Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks | Melicher, Ur, Segreti, Komanduri, Bauer, Christin, Cranor

William Melicher, Blase Ur, Sean M. Segreti, Saranga Komanduri, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor (CMU); Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks; In Proceedings of Some USENIX Security Symposium Type Event; 2016; 17 pages.


Human-chosen text passwords, today’s dominant form of authentication, are vulnerable to guessing attacks. Unfortunately, existing approaches for evaluating password strength by modeling adversarial password guessing are either inaccurate or orders of magnitude too large and too slow for real-time, client-side password checking. We propose using artificial neural networks to model text passwords’ resistance to guessing attacks and explore how different architectures and training methods impact neural networks’ guessing effectiveness. We show that neural networks can often guess passwords more effectively than state-of-the-art approaches, such as probabilistic context-free grammars and Markov models. We also show that our neural networks can be highly compressed—to as little as hundreds of kilobytes—without substantially worsening guessing effectiveness. Building on these results, we implement in JavaScript the first principled client-side model of password guessing, which analyzes a password’s resistance to a guessing attack of arbitrary duration with sub-second latency. Together, our contributions enable more accurate and practical password checking than was previously possible.

“Anonymized” data really isn’t—and here’s why not | Ars Technica

“Anonymized” data really isn’t—and here’s why not; Nate Anderson; In Ars Technica; 2009-09-08 (eight years ago).


  • Latanya Sweeney? [maybe]; Some Paper; In LIDAP, WP4; WHEN?
    tl;dr → persons are uniquely identified using only three bits of information, most of the time.
  • Paul Ohm; Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization; 2009-08-13; ssrn:1450006.


In archaeological order, in Ars Technica

PicoRadio Supports Ad Hoc Ultra-Low Power Wireless Networking

Jan M. Rabaey, M. Josie Ammer, Julio L. da Silva Jr., Danny Patel, Shad Roundy (UCB); PicoRadio Supports Ad Hoc Ultra-Low Power Wireless Networking; In IEEE Computer; 2000; 8 pages.
Teaser: One of the most compelling challenges of the next decade is the “last- meter” problem—extending the expanding data network into end-user data-collection and monitoring devices. PicoRadio supports the assembly of an ad hoc wireless network of self-contained mesoscale, low-cost, low-energy sensor and monitor nodes.

tl;dr → early a eighteen years ago; nearly one human generation ago.


  • Pico Radio
  • Berkeley Wireless Research Center (BWRC)
  • University of California Berkeley (UCB)