In September 2015 the University of Michigan announced a data science initiative of 100 million dollars that would hire 35 new faculty. David Donoho, preparing a lecture for the centenary of the statistician John Tukey, noticed something odd about the announcement: its description of data science, "the collection, management, processing, analysis, visualization, and interpretation of vast amounts of heterogeneous data", would strike most applied statisticians as a description of the work they had done for their whole careers (Donoho, 2017). Statisticians, he wrote, "see administrators touting, as new, activities that statisticians have already been pursuing daily".
That tension is a good place to start a history of data science, because the field has two histories that run on different clocks. One is the history of the activity, learning from data, which is centuries old. The other is the history of the name, which is young, contested, and has meant different things to different people. This article traces both, using the original sources wherever they could be read, and ends by mapping Donoho's account of what the field contains onto the topics of this Data Science collection.
Data analysis before the name
Three studies spread over two centuries show the whole working pattern of data science long before it had a name: a question, data collected by someone else for another purpose, a judgment about how far those data can be trusted, a summary, a comparison, and an argument to a reader who must act.
Graunt and the bills of mortality (1662)
London's parishes published weekly bills of mortality, counts of burials and christenings with the cause of each death. John Graunt's Natural and Political Observations upon the Bills of Mortality (1662) treated those bills as data. Before using them, he described how they were made. When someone died, "the Searchers (who are ancient Matrons, sworn to their Office) repair to the place where the dead Corps lies, and by view of the same, and by other enquiries, they examine by what Disease or Casualty the Corps died", and report to the parish clerk, who sends the week's account to be printed (Graunt, 1662, in Hull's 1899 edition). Graunt then asked which causes the Searchers could plausibly recognize by sight, and concluded that for some, such as death in old age or a wasting consumption, their report "might be sufficient" for his purposes even if a physician would dispute the exact disease. That is a judgment about measurement error made before any analysis, in the same spirit as the checks in Cleaning Data.
His best-known estimate combines several causes. Of 229,250 deaths over twenty years, he counted those from diseases that he guessed fell almost entirely on young children, added a share of deaths from smallpox and measles, set aside the deaths from plague, and concluded that "about thirty six per Centum of all quick conceptions died before six years old". He went further and built a table of how many of 100 conceptions survive to each decade of age: 36 dead within six years and, by his assumption, one left alive at 76, with the decades between filled in by "mean proportional numbers", a geometric interpolation (Graunt, 1662). The endpoints come from data; the shape between them is a model. Hull's edition prints the work among the writings of William Petty, with a title page that calls it "more probably by Captain John Graunt", a reminder that its authorship was once disputed.
Snow and the water companies (1855)
In 1852 the Lambeth water company moved its works upriver to Thames Ditton, "thus obtaining a supply of water quite free from the sewage of London", while the Southwark and Vauxhall Company went on drawing water from the Thames at Battersea. In several districts south of the river the two companies' pipes went "down all the streets", and neighboring houses were supplied by different companies according to choices made years earlier (Snow, 1855). John Snow saw what this arrangement made possible. Each company, he wrote, supplied "both rich and poor, both large houses and small", so that "no experiment could have been devised which would more thoroughly test the effect of water supply on the progress of cholera than this, which circumstances placed ready made before the observer". More than 300,000 people had been "divided into two groups without their choice, and, in most cases, without their knowledge" (Snow, 1855).
What the experiment needed was data that no one had collected: the water company of every house where someone died of cholera. Snow collected it, and summarized the first seven weeks of the epidemic in his Table IX.
| Water supply | Houses | Cholera deaths | Deaths per 10,000 houses (printed) | Recomputed |
|---|---|---|---|---|
| Southwark and Vauxhall | 40,046 | 1,263 | 315 | 315.4 |
| Lambeth | 26,107 | 98 | 37 | 37.5 |
| Rest of London | 256,423 | 1,422 | 59 | 55.5 |
Mortality in houses supplied by Southwark and Vauxhall was, in Snow's words, "between eight and nine times as great" as in houses supplied by Lambeth: the ratio of his printed rates is 8.51, and of the rates recomputed from his counts, 8.40. Recomputing the table, as the script tools/experiments/data-science/history-of-data-science.py does, also turns up a small puzzle. The first two printed rates are the recomputed ones with the decimals dropped, but the rest of London's printed rate of 59 does not follow from its printed counts, which give 55.5, and the text does not say which figure was used. The conclusion does not depend on it. Rereading a famous table this way is a habit worth having: numbers in print are data like any other.
Snow's comparison is a natural experiment: the assignment of houses to companies was not made by the investigator, but, as Snow argued, it was unrelated to the residents' circumstances, which is what a randomized experiment guarantees by design. It is an early answer to a causal question from observational data, the kind of question Why Data Science separates from description and prediction.
Nightingale and the army in the Crimea (1858)
Florence Nightingale's Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the British Army (1858), presented at the request of the Secretary of State for War, used the army's own returns to argue that soldiers in the Crimean War died mainly of disease that could have been prevented. For the troops in the Crimea from 1 October 1854 to 31 March 1855, not counting those in hospital at Scutari, the return she analyzed recorded 5,359 deaths: 373 from wounds and 4,986 from disease. Of the disease deaths, 4,465 were from what she called zymotic disease (scurvy, cholera, diarrhoea, dysentery and fevers), so that "near 90 per cent. died from preventible causes alone" (Nightingale, 1858, pp. 33–34). The exact share is $4{,}465 / 4{,}986 = 0.896$, and there were 13.4 deaths from disease for every death from wounds.
The same report presents the figures in pictures as well as tables, among them a "Diagram of the Causes of Mortality in the Army in the East" (p. 315). Choosing a form that a particular reader will take in is part of the work, as The Data Science Lifecycle discusses; which forms are read most accurately is the subject of Best Practices for Visualizing Data.
Tukey: data analysis as a science (1962)
The modern argument begins with John Tukey's "The Future of Data Analysis" (Tukey, 1962), published in the Annals of Mathematical Statistics, in Donoho's words "the central venue for mathematically-advanced statistical research of the day". Its opening is a confession: "For a long time I have thought I was a statistician, interested in inferences from the particular to the general. But as I have watched mathematical statistics evolve, I have had cause to wonder and to doubt." Tukey's central interest, he decided, was data analysis, which he took to include, "among other things", "procedures for analyzing data, techniques for interpreting the results of such procedures, ways of planning the gathering of data to make its analysis easier, more precise or more accurate, and all the machinery and results of (mathematical) statistics which apply to analyzing data" (Tukey, 1962, p. 2).
The paper's claim was that data analysis is a science, not a branch of mathematics. A science, Tukey argued, needs intellectual content, organization in an understandable form, and "reliance upon the test of experience as the ultimate standard of validity"; mathematics fails the third test, because its standard is logical proof. Data analysis passes all three, and he would regard it as "a science, one defined by a ubiquitous problem rather than by a concrete subject" (Tukey, 1962, pp. 5–6). Donoho (2017) adds a list of four influences acting on data analysis, which he quotes as Tukey's and calls "John's 1962 list": the formal theories of statistics, accelerating developments in computers and display devices, the challenge of ever larger bodies of data, and the emphasis on quantification in more and more disciplines. Donoho observes that the list "encompasses all the factors cited today in press releases touting today's Data Science initiatives", and that the shocking item in 1962 was the first, which made statistical theory only one part of the new science.
Tukey developed the practical side of this program in Exploratory Data Analysis (1977). Its central idea, that looking at data to form hypotheses is a different activity from testing them, is the subject of Performing Exploratory Data Analysis.
A name with another meaning: Naur's datalogy (1966 and 1974)
The words "data science" were in print before anyone used them in Tukey's sense, with a different meaning. The Danish computer scientist Peter Naur proposed the word "datalogy" in a letter to Communications of the ACM titled "The science of datalogy" (Naur, 1966), and in a 1968 paper defined it as "the science of data and of data processes" (as quoted by Cao, 2017). In the preface to his Concise Survey of Computer Methods (1974), as quoted by Cao (2017), he defined data science as "the science of dealing with data, once they have been established, while the relation of the data to what they represent is delegated to other fields and sciences".
Naur's definition is closer to what is now called computer science, or to the data engineering of this collection, than to data analysis. It deliberately leaves out the part that Graunt and Snow spent most of their effort on: the relation of the data to what they represent. That boundary, between handling data and drawing conclusions about the world from them, still divides the field's job titles, as Careers in Data Science describes.
Statisticians argue for a larger field (1993 to 2001)
Tukey's paper had little immediate effect on academic statistics. Donoho (2017) credits Tukey's colleagues at Bell Labs, outside the universities, with keeping the argument alive.
- John Chambers, a developer of the S language for statistical computing, argued in "Greater or Lesser Statistics" (1993) that the profession faced a choice between "continuing concentration on traditional topics" and "a broader viewpoint", one "based on an inclusive concept of learning from data", and that the first risked "seeing statistics become increasingly marginal" (Chambers, 1993, as quoted by Donoho, 2017).
- Jeff Wu, in a 1997 lecture titled "Statistics = Data Science?", suggested that statistics be renamed data science and statisticians data scientists (Cao, 2017; Donoho, 2017).
- William Cleveland, also of Bell Labs, published "Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics" (Cleveland, 2001). Its abstract, as Donoho quotes it, "sets out six technical areas of work for a university department and advocates a specific allocation of resources devoted to research in each area": multidisciplinary investigations (25%), models and methods for data (20%), computing with data (15%), pedagogy (15%), tool evaluation (5%) and theory (20%). Donoho considers this possibly the earliest use of the term "in which it is utterly clear that the writer means it in exactly the modern sense".
Cleveland's percentages are a statement of priorities. A department following them would spend more effort on computing and teaching combined than on theory; Donoho remarks that he could not think of an academic department that devoted 15% of its effort to pedagogy, or 15% to computing with data.
Breiman's two cultures (2001)
In the same year Leo Breiman, a Berkeley statistician who had spent years as a consultant, published "Statistical Modeling: The Two Cultures" (Breiman, 2001). He pictured the data as produced by a black box that turns inputs $x$ into responses $y$, and named two goals: prediction, "to be able to predict what the responses are going to be to future input variables", and information, "to extract some information about how nature is associating the response variables to the input variables".
The two cultures differ in what they put inside the box. The data modeling culture assumes a stochastic model, such as linear or logistic regression, estimates its parameters, and validates it with goodness-of-fit tests and residual checks; Breiman estimated that it held 98% of all statisticians. The algorithmic modeling culture treats the inside of the box as complex and unknown, looks for any function $f(x)$ that predicts $y$ well, such as a decision tree or a neural network, and validates it by predictive accuracy; it held 2% of statisticians, "many in other fields". Breiman argued that the exclusive commitment to data models had "led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems". In the published discussion, David Cox and Bradley Efron disagreed with parts of the argument; Efron wrote that the paper "overstates both its role and our profession's lack of interest" in prediction (as quoted by Donoho, 2017).
Both cultures appear throughout this collection. Linear Regression Assumptions and Logistic Regression belong to the first; Random Forests, Breiman's own method, and The Multilayer Perceptron to the second; and Introduction to Supervised Learning ends with the trade-off between prediction and explanation that Breiman's paper is about.
The common task framework
Donoho (2017) argues that Breiman left out the practice that made the predictive culture succeed, which the linguist Mark Liberman called the common task framework: a public training dataset, a set of competitors who all try to build the best prediction rule from it, and a referee who scores each submitted rule on test data the competitors cannot see. The Netflix Prize, in which competitors predicted subscribers' movie ratings for a prize of a million dollars, is Donoho's example. The framework turns "is this method better?" into a measurement, and it depends on the discipline of held-out test data described in Training, Validation, and Test Sets. The Netflix data reappear in Ethics in Data Science, because the anonymized ratings released for the contest turned out to identify people.
The data mining strand
Computer science arrived at the same activity from databases. Cao (2017) traces the community of the ACM SIGKDD conferences to a first workshop on Knowledge Discovery in Databases held with IJCAI in 1989, and associates terms such as "data mining" and "knowledge discovery" with that community and the conferences that followed. Industry needed a process as well as algorithms. In late 1996, according to the foreword of its guide, three companies with data mining practices (Daimler-Benz, the software firm ISL, later part of SPSS, and NCR) set out to define a standard, non-proprietary process; the result, the Cross-Industry Standard Process for Data Mining, was published as CRISP-DM 1.0 in 2000 (Chapman et al., 2000). Its six phases, from business understanding to deployment, remain a common description of how a project runs, and The Data Science Lifecycle follows them.
The job title (2008 to 2015)
The phrase became a job title in industry. DJ Patil recalls that, "starting in 2008, Jeff Hammerbacher ... and I sat down to share our experiences building the data and analytics groups at Facebook and LinkedIn". They needed a name for the people they hired. "Business analyst" seemed too limiting; "data analyst" might limit what people could do; "research scientist" suggested work isolated from the products. "The term that seemed to fit best was data scientist: those who use both data and science to create something new" (Patil, 2011).
The title reached a general business audience with Davenport and Patil's article in the October 2012 Harvard Business Review, "Data Scientist: The Sexiest Job of the 21st Century" (Davenport and Patil, 2012). On 18 February 2015 the White House named Patil the first U.S. Chief Data Scientist, describing his career as that of "a data scientist", a term "he helped coin" (The White House, 2015). The universities followed with degree programs and initiatives like Michigan's.
Donoho (2017) was skeptical of the reasons usually given for treating data science as new. Large datasets were not new: census data, he notes, have long been of today's "big data" scale. The skills most often cited, such as programming for clusters of computers, are in his words "coping skills for dealing with organizational artifacts of large-scale cluster computing" rather than new ways of drawing inferences. His summary of the field as it was being defined in universities was "a superset of the fields of statistics and machine learning which adds some technology for 'scaling up' to 'big data'", a choice "motivated by commercial rather than intellectual developments".
Code made the activity visible
Donoho's account of why data analysis finally became a recognizable field is about software rather than words. Chambers and Rick Becker developed the S environment at Bell Labs starting in the mid-1970s; Robert Gentleman and Ross Ihaka created R, an open-source work-alike, in the 1990s. Such environments run scripts that "codify precisely the steps of a computation", and once a community shares an environment, its analyses can be shared and rerun. What was "somewhat nebulous", such as "the prose description of some data analysis in a scientific paper", then "becomes instead tangible and useful" (Donoho, 2017). Python, first released in 1991, later became the other common language of the field; Programming with Python and Jupyter tells that story, and Data Science Packages describes the libraries that made it so.
This is also why every number in this collection comes from a script in the site's repository. A script is the form in which an analysis can be checked, which is the point Tukey made about data analysis as a science: its standard of validity is the test of experience.
What the field contains: Donoho's six divisions
Donoho's proposal was a "greater data science" defined by the activities of people who learn from data, organized into six divisions. They make a useful map of this collection.
| Donoho's division (2017) | What it covers | Where this collection treats it |
|---|---|---|
| Data exploration and preparation | Exploring data to check its basic properties; cleaning and recoding | Introduction to Data Wrangling, Cleaning Data, Performing Exploratory Data Analysis, Detecting Outliers |
| Data representation and transformation | Formats, databases, restructuring data into a more revealing form | Relational Databases and the SQL articles, Structuring Data, Data Engineering |
| Computing with data | Languages, workflows, packages, computation at scale | Programming with Python and Jupyter, The NumPy Package, The pandas Package, Queries in Python |
| Data visualization and presentation | Plots from exploration to the presentation of conclusions | Visualizing Data with One Feature, Visualizing Data with Multiple Features, Best Practices for Visualizing Data |
| Data modeling | Both of Breiman's cultures: generative models and predictive models | Introduction to Regression, Introduction to Supervised Learning, Introduction to Unsupervised Learning, Introduction to Ensemble Models, Introduction to Artificial Neural Networks |
| Science about data science | Studying how data analysis is actually done and how well it works | The Bias-Variance Tradeoff, Cross-Validation, Model Selection, and the seeded experiments throughout |
Donoho noted that a single division, data modeling, "dominates the representation of data science in today's academic departments", through statistics in some and machine learning in others, and that his scheme places computing and data representation "as major divisions alongside" it. The order of topics in this collection follows the same reasoning: tools and data handling come first, and the statistical core and the learning methods are built on them.
What the history explains
Three things about present-day data science follow from this history. The first is that its methods are older than its name, and many of the problems it inherits are old as well: Graunt worried about the quality of the Searchers' reports, Snow had to collect the variable no one had recorded, and Nightingale presented the same figures in tables and in diagrams. The second is that the field was assembled by people arguing from different starting points, statisticians who wanted their discipline to grow and computer scientists and companies who needed a name and a process, and the seams still show in the disagreement over whether data science is a new field or a rebranding of statistics. The third is that the distinction Breiman drew between prediction and understanding has since been sharpened into three kinds of question, description, prediction and causal inference, each with its own requirements (Hernán, Hsu and Healy, 2019). That classification is the starting point of Why Data Science.
References
- Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3), 199–231. https://doi.org/10.1214/ss/1009213726 record
- Cao, L. (2017). Data science: A comprehensive overview. ACM Computing Surveys, 50(3), Article 43. https://doi.org/10.1145/3076253 record
- Chambers, J. M. (1993). Greater or lesser statistics: A choice for future research. Statistics and Computing, 3(4), 182–184. https://doi.org/10.1007/BF00141776 record
- Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., Shearer, C., and Wirth, R. (2000). CRISP-DM 1.0: Step-by-step data mining guide. SPSS. record · 2 pages
- Cleveland, W. S. (2001). Data science: An action plan for expanding the technical areas of the field of statistics. International Statistical Review, 69(1), 21–26. https://doi.org/10.1111/j.1751-5823.2001.tb00477.x record
- Davenport, T. H., and Patil, D. J. (2012). Data scientist: The sexiest job of the 21st century. Harvard Business Review, 90(10), 70–76. record
- Donoho, D. (2017). 50 years of data science. Journal of Computational and Graphical Statistics, 26(4), 745–766. https://doi.org/10.1080/10618600.2017.1384734 record · 3 pages
- Graunt, J. (1662). Natural and Political Observations Mentioned in a Following Index, and Made upon the Bills of Mortality. London. Reprinted in C. H. Hull (Ed.), The Economic Writings of Sir William Petty, Vol. 2 (1899). Cambridge University Press. record
- Hernán, M. A., Hsu, J., and Healy, B. (2019). A second chance to get causal inference right: A classification of data science tasks. CHANCE, 32(1), 42–49. https://doi.org/10.1080/09332480.2019.1579578 record · 4 pages
- Naur, P. (1966). The science of datalogy [Letter]. Communications of the ACM, 9(7), 485. https://doi.org/10.1145/365719.366510 record
- Nightingale, F. (1858). Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the British Army. Harrison and Sons. record
- Patil, D. J. (2011). Building Data Science Teams. O'Reilly Media. record · 2 pages
- Snow, J. (1855). On the Mode of Communication of Cholera (2nd ed.). John Churchill. record · 2 pages
- The White House (2015, February 18). The White House names Dr. DJ Patil as the first U.S. Chief Data Scientist [Blog post]. https://obamawhitehouse.archives.gov/blog/2015/02/18/white-house-names-dr-dj-patil-first-us-chief-data-scientist record
- Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics, 33(1), 1–67. https://doi.org/10.1214/aoms/1177704711 record
- Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley. record · 2 pages