Karl Ho
School of Economic, Political and Policy Sciences
University of Texas at Dallas
New Data Analytics in Political and Policy Sciences
Presentation prepared for the National Chung-hsing University, Taichung, Taiwan, June 30th, 2018
Data Analytics vs.
Data Analysis
Data analytics refers to generation, acquisition, management, modeling and visualization of data.
Thomas Davenport and his colleagues (2007) emphasize the ability to "collect, analyze and act on data".
Davenport, Thomas H., and Jeanne G. Harris. 2007. Competing on analytics: The new science of winning. Harvard Business Press.
Data Analytics vs.
Data Analysis
Data analytics goes beyond only providing analysis of data but focuses on the action or decision making informed by data.
Social data was initially referred as to data generated from social media. It is now not confined to that generation mode but is more generally data generated by people or users.
Social Data Analytics: A journey just set afoot
Social data was initially referred to data generated from social media. It is now not confined to that generation mode but is more generally data generated by people or users.
Social data analytics focus on generalizations social data.
.... social science is beginning to shape the world of big data.
Much of big data is social data.... It is the responsibility of social scientists to assume their central place in the world of big data, to shape the questions we ask of big data, and to characterize what does and does not make for a convincing answer.
- Monroe, Pan, Roberts, Sen and Sinclair 2015
What is Data Science?
-
Science of Data
-
Understand Data Scientifically
-
Data is ever growing......
-
Moore's Law
-
Parkinson's Law
-
-
Science?
Moore's Law
Moore's Law
Computing power doubles every 18 months.
Parkinson's Law of Data
“Data expands to fill the space available for storage.”
"Data Lake" Ubiquitous
Massive raw data repository in its rawest form pending processing.
How Data are generated?
-
Computers
-
Web
-
Mobile devices
-
IoT
-
Further extension of human users (e.g. avatars)
The key word in "Data Science" is not Data....
- Jeff Leek
it is Science.
The long term impact of Data Science will be measured by the scientific questions we can answer with the data.
- Jeff Leek
Big data revolution is transforming almost all disciplines. In fact, much of the data are generated in public agencies or under the regulations of government agencies. Big data is important to the public and should be considered public goods.
Why Public Policy, Government and Non-profit management researchers are concerned?
Millions of open data-related jobs are begging not just business, computer science and mathematics programs but also Social Sciences including Political Science seeking candidates who can deal with complicated big data and statistical models.
Why Public Policy, Government and Non-profit management researchers are concerned?
How do we ready researchers and students to understand the science of data and to scientifically understand data?
Why Public Policy, Government and Non-profit management researchers are concerned?
Jobs, jobs and jobs
Sample job
Sample job
Sample job
Sample job
Sample job
Keywords
-
Data management
-
Data analytics
-
Data scientists
-
Data curation
-
Modeling
-
CRMs
-
Python
-
Survey
-
R
Lazer et al. 2009 Life in the network
This figure summarizes the link structure within a community of political blogs (from 2004), where red nodes indicate conservative blogs, and blue liberal. Orange links go from liberal to conservative, and purple ones from conservative to liberal. The size of each blog reflects the number of other blogs that link to it
Cumulated/Repeated Data
A Theory of Data: Understanding Data Generation Process
Data Generation
Administrative Data
Administrative data are defined as data which derive from the operation of administrative systems, typically by public sector agencies
- Connelly et al. 2016
Administrative Data
Big data has the potential to change the landscape of social science research, and administrative social science data may offer particular benefits.
- Connelly et al. 2016
Administrative Data
Data Drive Decisions.
- Davenport 2006
Administrative Data
-
Smart City
-
Policy evaluations
-
Political and Policy implications
Examples:
A Taxonomy of Data
-
Numbers
-
Text
-
Images
-
Audio
-
Video
-
Signals
-
Data of data: Metadata and Paradata
Data Methods
-
Survey
-
Experiments
-
Qualitative Data
-
Text Data
-
Web Data
-
Machine Data
-
Complex Data
-
Network Data
-
Multiple-source linked Data
-
Statistical Modeling:
The Two Cultures
Leo Breiman 2001: Statistical Science
| One assumes that the data are generated by a given stochastic data model. |
|---|
| The other uses algorithmic models and treats the data mechanism as unknown. |
|---|
| Data Model |
|---|
| Algorithmic Model |
|---|
| Small data |
|---|
| Complex, big data |
|---|
Theory:
Data Generation Process
Data are generated in many fashions. Picture this: independent variable x goes in one side of the box-- we call it nature for now-- and dependent variable y come out from the other side.
Theory:
Data Generation Process
Data Model
The analysis in this culture starts with assuming a stochastic data model for the inside of the black box. For example, a common data model is that data are generated by independent draws from response variables.
Response Variable= f(Predictor variables, random noise, parameters)
Reading the response variable is a function of a series of predictor/independent variables, plus random noise (normally distributed errors) and other parameters.
Theory:
Data Generation Process
Data Model
The values of the parameters are estimated from the data and the model then used for information and/or prediction.
Theory:
Data Generation Process
Algorithmic Modeling
The analysis in this approach considers the inside of the box complex and unknown. Their approach is to find a function f(x)-an algorithm that operates on x to predict the responses y.
The goal is to find algorithm that accurately predicts y.
Theory:
Data Generation Process
Algorithmic Modeling
Unsupervised Learning
Supervised Learning vs.
Source: https://www.mathworks.com
Algorithm and Inference
Very broadly speaking, algorithms are what statisticians do while inference says why they do them.
- Efron and Hastie 2017
Social (Data) Scientist's mission
Two major areas to which social scientists can contribute, based on decades of experience and work with end users, are:
-
Inference
-
Data quality.
- Foster et al. 2016
Let the dataset change your mindset.
- Hans Rosling
Hans Rosling
- Founded Gapminder Foundation
- Visualize historical data on public health and poverty
The story of Google Flu Trend
By using Big Data of search queries, Google Flu Trend (GFT) predicted the flu-like illness rate in a population.
However, the journal Nature where GFT published the findings on figured the GFT overestimated as much as twice than the actual data. Two political scientists helped fix and address the problem.
Lesson we learn:
Political Science can save the world!
The story of Google Flu Trend
Lazer, Kennedy, King and Vespignani (2014)
Traditional “small data” often offer information that is not contained (or containable) in big data, and the very factors that have enabled big data are enabling more traditional data collection (watch TED talk by Dr. Joel Selanikio). The Internet has opened the way for improving standard surveys, experiments, and health reporting. (Lazer et al. 2014 Science)
Another Google data story
In the United States, the word “Jewish” is used in 3.2 times more Google searches than “Mormon”. This does not mean that the Jewish population is 3.2 times larger than the Mormon population. (Stephens-Davidowitz and Varian 2015).
Another possible explanation is that Jewish people use the Internet in higher proportions or have more questions that require using the word “Jewish.” Thus Google search data are more useful for relative comparisons than for estimating absolute levels (Foster 2016).
Hal Varian
Chief Economist, Google
Professor of Economics, University of California, Berkeley.
Big Data: New Tricks for Econometrics
Machine Learning and Econometrics
- Hal Varian
The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill in the next decades.
- Mike Driscoll
“The Three Sexy Skills of Data Geeks”: “…with the Age of Data upon us, those who can model, munge, and visually communicate data...
Data Scientists
Nathan Yau (2009) calls them:
SDAR Roadmap
Introduction - Data theory
Data methods
Statistics
Programming
Data Visualization
Information Management
Data Curation
Spatial Models and Methods
Machine Learning
NLP/Text mining
SDAR Roadmap
-
Introduction - Data theory
-
Fundamentals
-
Data concepts
-
Data Generation Process (DGP)
-
-
Algorithm-based vs. Data-based approaches
-
Taxonomy
-
SDAR Roadmap
-
Data methods
-
Passive data
-
Data at will
-
Qualitative data
-
Complex data
-
Text data
-
SDAR Roadmap
-
Statistics
-
Sample and Population
-
Inference
-
Size and power
-
Representation
-
SDAR Roadmap
-
Programming
-
R
-
Python
-
HTML
-
Java script
-
Data Science Roadmap
-
Data Visualization
-
Tableau
-
ggplot2
-
Shiny
-
D3.js
-
Animation
-
Data Science Roadmap
-
Information Management
-
MapReduce
-
Hadoop
-
Cassandra
-
MongoDB
-
NoSQL
-
Data Science Roadmap
-
Data curation
-
Google OpenRefine
-
Sampling
-
Missing value concepts and management
-
Data Science Roadmap
-
Spatial Models and Methods
-
GIS
-
R/Leaflet
-
Python Map
-
Remote Sensing
-
Data Science Roadmap
-
Machine Learning
-
Supervised
-
Unsupervised
-
Regression methods
-
Neural Networks
-
Data Science Roadmap
-
NLP/Text Mining
-
Corpus
-
Text Analysis
-
Sentiment Analysis
-
Natural Language Processing
-
Machine Learning
Machine can excel at frequent and high-volume task, at much faster rate and lower variance.
Human can tackle noble situations.
Quick Analytics:
Taiwan Climate
Quick Analytics:
Taiwan Climate
Spatial Data: United States
Spatial Data: United States
Java: D3 Library
Sentiment Analysis
Sentiment Analysis
"1","RT @RealJack: *Last year*
Democrats: “TRUMP IS SUCH A TERRIBLE PRESIDENT HE WILL GET US NUKED BY NORTH KOREA!!”
*Trump meets with Kim*
D…"
"2","Trump Kim summit: US wants 'major N Korea disarmament' by 2020 https://t.co/htY2r4eXXj"
"3","RT @thehill: JUST IN: Norwegian lawmakers nominate Trump for Nobel Peace Prize after summit with Kim Jong Un https://t.co/Uer56GgE2A https:…"
"4","RT @JRubinBlogger: Pompeo is acting exactly like Kerry -- indignant, caught up in process. Convinced concessions aren'[t concessions. Pathe…"
"5","RT @SykesCharlie: On Wednesday morning, Chosun Ilbo, South Korea’s paper of record, published a bleak editorial: “Kim Jong-un Got Everythin…"
"6","RT @chuckwoolery: Yesterday Shepard Smith, gave a scathing report on Trump/Kim Singapore summit. Following the Lefts lead."
"7","RT @WhiteHouse: Leaders the world over spoke of the powerful significance of President Trump’s summit with Kim Jong Un this week.
Read mor…"
"8","RT @PalmerReport: Fuck Donald Trump
Fuck Kim Jong Un
Fuck their fake summit
Fuck Vladimir Putin
Fuck Dennis Rodman
Fuck the media for…"
"9","Dennis Rodman has been the link between Kim Jong-Un and Donald Trump. Very Scary times we are in @StephMillerShow… https://t.co/UR0JWs0hcp"
"10","RT @TomSteyer: .@realDonaldTrump repeatedly said, Kim Jong Un ""loves his people."" This is what love looks like to Trump:
Over 100,000 poli…"
"11","RT @TheUSASingers: I’m gonna lay it on the line.
- Obama isn’t a Muslim
- Hillary doesn’t eat babies
- Socialists aren’t Nazis
- Nazis are…"
-
Data Thinking
-
Multi-disciplinary Thinking
-
Machine Thinking
Conclusion: New Data Literacy
Thank you!
Wait... are you sure you have no questions?
What is Data?
Data is everything.
-
Examples