Phenomenal World

February 2nd, 2019

Phenomenal World

The Summit


Moving beyond computational questions in digital ethics research

In the ever expanding digital ethics literature, a number of researchers have been advocating a turn away from enticing technical questions—how to mathematically define fairness, for example—and towards a more expansive, foundational approach to the ethics of designing digital decision systems.

A 2018 paper by RODRIGO OCHIGAME, CHELSEA BARABAS, KARTHIK DINAKAR, MADARS VIRZA, and JOICHI ITO is an exemplary paper along these lines. The authors dissect the three most-discussed categories in the digital ethics space—fairness, interpretability, and accuracy—and argue that current approaches to these topics may unwittingly amount to a legitimation system for unjust practices. From the introduction:

“To contend with issues of fairness and interpretability, it is necessary to change the core methods and practices of machine learning. But the necessary changes go beyond those proposed by the existing literature on fair and interpretable machine learning. To date, ML researchers have generally relied on reductive understandings of fairness and interpretability, as well as a limited understanding of accuracy. This is a consequence of viewing these complex ethical, political, and epistemological issues as strictly computational problems. Fairness becomes a mathematical property of classification algorithms. Interpretability becomes the mere exposition of an algorithm as a sequence of steps or a combination of factors. Accuracy becomes a simple matter of ROC curves.

In order to deepen our understandings of fairness, interpretability, and accuracy, we should avoid reductionism and consider aspects of ML practice that are largely overlooked. While researchers devote significant attention to computational processes, they often lack rigor in other crucial aspects of ML practice. Accuracy requires close scrutiny not only of the computational processes that generate models but also of the historical processes that generate data. Interpretability requires rigorous explanations of the background assumptions of models. And any claim of fairness requires a critical evaluation of the ethical and political implications of deploying a model in a specific social context.

Ultimately, the main outcome of research on fair and interpretable machine learning might be to provide easy answers to concerns of regulatory compliance and public controversy"

⤷ Full Article

January 26th, 2019

Bone Mobile


Learning about long-term effects of interventions, and designing interventions to facilitate the long view

A new paper from the Center for Effective Global Action at Berkeley surveys a topic important to our researchers here at JFI: the question of long-run effects of interventions. In our literature review of cash transfer studies, we identified the need for more work beyond the bounds of a short-term randomized controlled trial. This is especially crucial for basic income, which is among those policies intended to be permanent.

The authors of the new Berkeley report, Adrien Bouguen, Yue Huang, Michael Kremer, and Edward Miguel, note that it’s a particularly apt moment for this kind of work: “Given the large numbers of RCTs launched in the 2000’s, every year that goes by means that more and more RCT studies are ‘aging into’ a phase where the assessment of long-run impacts becomes possible.”

The report includes a summary of what we know about long-run impacts so far:

"Section 2 summarizes and evaluates the growing body of evidence from RCTs on the long-term impacts of international development interventions, and find most (though not all) provide evidence for positive and meaningful effects on individual economic productivity and living standards. Most of these studies examine existing cash transfer, child health, or education interventions, and shed light on important theoretical questions such as the existence of poverty traps (Bandiera et al., 2018) and returns to human capital investments in the long term."

Also notable is the last section, which contains considerations for study design, "lessons from our experience in conducting long-term tracking studies, as well as innovative data approaches." Link to the full paper.

  • In his paper "When are Cash Transfers Transformative?," Bruce Wydick also notes the need for long-run analysis: "Whether or not these positive impacts have long-term transformative effects—and under what conditions—is a question that is less settled and remains an active subject of research." The rest of the paper is of interest as well, including Wydick's five factors that tend to signal that a cash transfer will be transformative. Link.
  • For more on the rising popularity of RCTs, a 2016 paper by major RCT influencers Banerjee, Duflo, and Kremer quantifies that growth and discusses the impact of RCTs. Link. Here’s the PowerPoint version of that paper. David McKenzie at the World Bank responds to the paper, disputing some of its claims. Link.
⤷ Full Article

January 19th, 2019



A look at China's social credit system

In a recent newsletter, we noted a spate of reporting drawing attention to the authoritarianism of China's growing Social Credit System. This week, we are sharing a paper by YU-JIE CHEN, CHING-FU LIN, AND HAN-WEI LIU that casts light on the details of the program's workings, corrects common misconceptions, proposes some likely and disturbing future scenarios, and offers a useful frame for understanding the significant shift it is bringing about in Chinese governance.

"A new mode of governance is emerging with the rise of China’s 'Social Credit System' (shehui xinyong zhidu) or SCS. The SCS is an unusual, comprehensive governance regime designed to tackle challenges that are commonly seen as a result of China’s 'trustless' society that has featured official corruption, business scandals and other fraudulent activities. The operation of the SCS relies on a number of important and distinctive features—information gathering, information sharing, labeling, and credit sanctions—which together constitute four essential elements of the system.

In our view, the regime of the SCS reflects what we call the 'rule of trust,' which has significant implications for the legal system and social control in China. We define the 'rule of trust' as a governance mode that imposes arbitrary restrictions—loosely defined and broadly interpreted trust-related rules—to condition, shape, and compel the behavior of the governed subjects… The 'rule of trust' is in fact undermining 'rule of law.'

In the context of governance, the unbounded notion of 'trust' and the unrestrained development of technology are a dangerous combination."

Link to the paper.

⤷ Full Article

January 12th, 2019



Another kind of cybersecurity risk: the destruction of common knowledge

In a report for the Berkman Klein center, Henry Farrell and Bruce Schneier identify a gap in current approaches to cybersecurity. National cybersecurity officials still base their thinking on Cold War-type threats, where technologists focus on hackers. Combining both approaches, Farrell and Schneier make a wider argument about collective knowledge in democratic systems—and the dangers of its diminishment.

From the abstract:

"We demonstrate systematic differences between how autocracies and democracies work as information systems, because they rely on different mixes of common and contested political knowledge. Stable autocracies will have common knowledge over who is in charge and their associated ideological or policy goals, but will generate contested knowledge over who the various political actors in society are, and how they might form coalitions and gain public support, so as to make it more difficult for coalitions to displace the regime. Stable democracies will have contested knowledge over who is in charge, but common knowledge over who the political actors are, and how they may form coalitions and gain public support... democracies are vulnerable to measures that 'flood' public debate and disrupt shared decentralized understandings of actors and coalitions, in ways that autocracies are not."

One compelling metaresearch point from the paper is that autocratic governments receive analysis of information trade-offs, while democratic governments do not:

"There is existing research literature on the informational trade-offs or 'dictators' dilemmas' that autocrats face, in seeking to balance between their own need for useful information and economic growth, and the risk that others can use available information to undermine their rule. There is no corresponding literature on the informational trade-offs that democracies face between desiderata like availability and stability."

Full paper available on SSRN here.

  • Farrell summarizes the work on Crooked Timber: "In other words, the same fake news techniques that benefit autocracies by making everyone unsure about political alternatives undermine democracies by making people question the common political systems that bind their society." Many substantive comments follow. Link.
  • Jeremy Wallace, an expert on authoritarianism, weighs in on Twitter: "Insiders, inevitably, have even more information about the contours of these debates. On the other hand, there's a lot that dictators don't know--about their own regimes, the threats that they are facing, etc." Link to Wallace's work on the topic.
  • Related reading recommended by Wallace, from Daniel Little, a 2016 paper on propaganda: "Surprisingly, the government tends to pick a high level of propaganda precisely when it is ineffective." Link.
⤷ Full Article

January 5th, 2019

Aunt Eliza


Bringing evidence to bear on policy

Happy 2019. We’re beginning with a report from Evidence in Practice, a project from the Yale School of Management. The report focuses on how to integrate rigorously researched evidence with policy and practice, with an emphasis on international development. The needs numerous stakeholders involved in research and policymaking are enumerated, along with their own needs and priorities: funders, researchers, intermediaries, policymakers, and implementers each receive consideration. One of the strengths of the report is its quotations from dozens of interviews across these groups, which give a sense of the messy, at times frustrating, always collaborative business of effecting change in the world. As to the question of what works:

⤷ Full Article

December 22nd, 2018

Frohes Fest | Year in Review

The JFI Letter has grown and morphed over the past twelve months; thank you to our readers for opening, skimming, clicking, and writing us every week. We'll be offline until January 5. In the meantime, here's a list of our favorite spotlights from last year and a list of favorite researchers to watch in the next.

⤷ Full Article

December 15th, 2018

Space Dance


A new book examines the economic and social impacts of R&D

Last May, we highlighted a report on workforce training and technological competitiveness which outlined trends in research and development investment. The report found that despite "total U.S. R&D funding reaching an all-time high in 2015," it's shifted dramatically to the private sector: "federal funding for R&D, which goes overwhelmingly to basic scientific research, has declined steadily and is now at the lowest level since the early 1950s." This week, we take a look at the returns to these investments and discuss how best to measure and trace the ways research spending affects economic activity and policy.

In the most recent Issues in Science and Technology, IRWIN FELLER reviewsMeasuring the Economic Value of Research, a technical monograph that discusses how best to measure the impact and value of research on policy objectives. Notably, the book highlights UMETRICS, a unified dataset from a consortium of universities "that can be used to better inform decisions relating to the level, apportionment, human capital needs, and physical facility requirements of public investments in R&D and the returns of these investments." While it represents a big data approach to program evaluation, Feller notes that UMETRICS' strength is in the "small data, theory-driven, and exacting construction of its constituent datasets," all of which offer insight into the importance of human capital in successful R&D:

"The book’s characterization of the ways in which scientific ideas are transmitted to and constitute value to the broader economy encompasses publications and patents, but most importantly includes the employment of people trained in food safety research. This emphasis on human capital reflects a core proposition of UMETRICS, namely the 'importance of people—students, principal investigators, postdoctoral researchers, and research staff—who conduct research, create new knowledge, and transmit that knowledge into the broader economy.'

In particular, the chapters on workforce dynamics relating to employment, earnings, occupations, and early careers highlight the nuanced, disaggregated, and policy-relevant information made possible by UMETRICS. These data provide much-needed reinforcement to the historic proposition advanced by research-oriented universities that their major contribution to societal well-being—economic and beyond—is through the joint production of research and graduate education, more than patents or other metrics of technology transfer or firm formation."

The UMETRICS dataset traces the social and economic returns of research universities and allows for a larger examination of universities as sociopolitical anchors and scientific infrastructure.

⤷ Full Article

December 1st, 2018

Energy Field


A discussion of different approaches to climate policy

Last week, the U.S. government released the Fourth National Climate Assessmentwhich outlined the dire economic and environmental consequences of climate change. Instead of highlighting key findings of the report—two good summaries are available here and here—we'll contextualize the current climate debate within legal history, which shows the limitations of current economically-focused arguments for climate policy.

A 2010 Yale Law Journal article by Jedediah Purdy situates the current climate debate within the long tradition of political argument about the natural world, and challenges assumptions that environmental values which appeal to moral and civic duty are too weak and vague to spur political action. In fact, Purdy argues that major environmental legislation emerged from "democratic argument over the value of the natural world and its role in competing ideas of citizenship, national purpose, and the role and scale of government." Purdy does more than just argue that environmental public language is more coherent than conventionally understood, he argues that understanding climate policy through economic self-interest diminishes the role political struggle plays in shaping national values and interests:

"Consider one example that makes little sense through the lens of narrow self-interest, much more as part of an ongoing debate over environmental values: the organizing project that has led 1015 city governments to adopt the goals of the Kyoto Protocol (a seven percent reduction in greenhouse-gas emissions from 1990 levels by 2012) through an instrument called the Mayors Climate Protection Agreement. Since the costs are not zero, and the benefits, in theory, are almost exactly that, the question of motivation is still fairly sharply presented.


In private interviews and public statements, city officials explain their efforts in several ways. They are quick to cite the advantage certain regions hope to enjoy from early adoption and manufacture of technologies that may later become standard. They embrace a simple public-choice motive: city governments hope to benefit from green-development block grants and, in the longer term, density-friendly economic development, and early efforts may position them to do both.
They also regard themselves as engaged in political persuasion that they hope will induce others to take similar action. Whether this is plausible is partly endogenous to the politics itself. This politics seeks to affect the reasons—specifically those grounded in environmental values—that people understand themselves to have for joining collective undertakings. Rather than a specimen of an independently established logic of collective action, it is an engagement with that logic itself."

Link to full paper.

  • In a 2018 article, Purdy looks more deeply at the history of environmental justice, and why its concerns were left out of mainstream environmental law: "Mainstream environmental law was the last major legal product of 'the great exception,' the decades of the mid-twentieth century when, unlike any other time in modern history, economic inequality was declining and robust growth was widely shared." Link.

  • A 2017 dissertation examines the environment as an object of politics, as opposed to natural capital, and argues that the environment is a "political problem that entails ongoing negotiations over the legitimacy of market rule, the role of the state in relation to the market, and the value of ecological stewardship." Link.

  • The new climate reports have brought attention back to solar geoengineering, which the Guardian, covering a Gernot Wagner paper, notes is extremely inexpensive and possibly an option for desperate circumstances: “The IPCC [Intergovernmental Panel on Climate Change] report said geoengineering might be adopted as a temporary “remedial measure” in extreme circumstances.”Link.

⤷ Full Article

November 17th, 2018

Poetry Machine



The inefficiency of lobbying

A few weeks ago, we spotlighted work by Elliott Ash et. al. on the startling influence of the Manne economics seminars in shaping judicial decision-making. This week we’re looking at an industry that, conversely, seems extremely influential, but is frequently ineffectual: lobbying.

In a 2009 book, "Lobbying and Policy Change: Who Wins, Who Loses, and Why," FRANK R. BAUMGARTNER et. al. take an unprecedentedly thorough look at lobbying in Washington, scrutinizing "ninety-eight randomly selected policy issues in which interest groups were involved and then followed those issues across two Congresses." What they find is complexity and gridlock:

"Since we followed our issues for four years, we know a lot about what eventually occurred (if anything did). In fact, as we outline in the chapters to come, for the majority of our issues, little happened.

If what they are supposed to be doing is producing change, interest groups are a surprisingly ineffectual lot. A focus by the media and many academics on explaining political change or sensational examples of lobbying success obscures the fact that lobbyists often toil with little success in gaining attention to their causes or they meet such opposition to their efforts that the resulting battle leads to a stalemate.

Of course, many lobbyists are active because their organizations benefit from the status quo and they want to make sure that it stays in place. We will show that one of the best single predictors of success in the lobbying game is not how much money an organization has on its side, but simply whether it is attempting to protect the policy that is already in place."

Preview on Google Books here.

⤷ Full Article

September 16th, 2019

Live Airborne System


Immanuel Wallerstein's contributions to research in the social sciences

Two weeks ago today marked the passing of the great Immanual Wallerstein. His work has had resounding influence across fields: from literature, to legal theory, education, development studies, and international relations. Among his foremost contributions is the four volume Modern World System series, which recount the transformation of feudalism into global capitalism through progressive incorporation of new regions into the European capitalist core. Complementing this history was world-systems theory, an analytical approach which challenged the tendency of social science research to identify simplified and direct causal relationships.

Wallerstein argued that purely economic, historical, or political analyses of society exclude more factors than they incorporate, casting doubt on both their internal and external validity. From the introduction to World Systems Analysis:

The phenomena dealt with in these separate boxes are so closely intermeshed that each presumes the other, each affects the other, each is incomprehensible without taking into account the other boxes. The separate boxes of analysis are an obstacle, not an aid, to understanding the world. Structurally, the social reality within which we live has not been the multiple national states of which we are citizens but something larger, which we call a world-system. This world-system has had many institutions—states and the interstate system, productive firms, households, classes, identity groups of all sorts—which form a matrix which permits the system to operate but at the same time stimulates both the conflicts and the contradictions which permeate it.

The world-system is a social creation, with a history, whose origins need to be explained, whose ongoing mechanisms need to be delineated, and whose inevitable terminal crisis needs to be discerned. For this reason, it is important to look anew not only at how the world in which we live works but also at how we have come to think about this world.

Link to the book's first pages.

  • "My intellectual development led me to historicize social movements, not only to better understand how they came to do the things they did, but also in order to better formulate the political options that were truly available in the present." On his website, Wallerstein reflects on the questions and contradictions that informed his life's work. Link.
  • "The Modern World-System is a theoretically ambitious work that deserves to be critically analyzed as such." Theda Skocpol's sympathetic scrutiny of the weaknesses in Wallerstein's major work, from the 1977 Review of American Sociology. Link.
  • Wallerstein's account of feudal breakdown, which stressed external factors like increased trade, countered that of historians like Robert Brenner, who focused instead on internal factors like peasant revolts. Robert A. Denemark and Kenneth P. Thomas give an overview of the debate. Link.
⤷ Full Article