Archive for the 'Data' Category

Raster Analysis in R

Etienne Racine has published a Visual Raster Cheat Sheet in RPubs.

H/T Urban Demographics.

Statistical Atlas of the United States

Nathan Yau of Flowing Data set out to recreate the original 56 page Statistical Atlas of the United States, first published in 1874 using 1870 Census data. Yau’s version uses current, publicly available government data.

Now that the first 56 maps are complete, Yau has decided to continue the project and produce a more compete Statistical Atlas with more maps and chart. He plans to update these weekly.

Rules for Charts

Nathan Yau of Flowing Data discusses the handful of rules for charts and data visualization which should never be broken. These include baselines, pie slices, and encodings.

New York City Census FactFinder

The NYC Planning Department’s American Community Survey update to the NYC Census Factfinder application has been released. It is now possible to get 2009-2013 ACS profiles for Neighborhood Tabulation Areas and user defined census tract aggregations, in addition to demographic profiles from the 2000 and 2010 censuses.

See the full press release.

H/T Data Detectives

Cleaning Data and Extracting Data from PDFs

Flowing Data points out two useful data resources:

  • Marc Bellemare, an associate professor in the Department of Applied Economics at the University of Minnesota, provides practical tips on cleaning data.
  • Tabula, which is available for both Windows and Mac, has an update which makes it easier to extract data from PDF documents.

Repository for ‘Dark Data’

Researchers at the University of North Carolina at Chapel Hill are working on a project called DataBridge to create an archive for data sets and metadata that would otherwise be lost once the papers they were produced for are published.

Read The Chronicle of Higher Education’s Wired Campus article here.

U.S. Census Bureau International Data Base

The U.S. Census Bureau released revised estimates and projections for 24 countries, including China, Iraq, Malawi, South Africa and United States. See the release note tab for a full list of revised countries.

Big data and smart cities

Urban Demographics posted a presentation by Rob Kitchin based on his paper “The real-time city? Big data and smart urbanism” (gated version; working paper version).

Abstract:

‘Smart cities’ is a term that has gained traction in academia, business and government to describe cities that, on the one hand, are increasingly composed of and monitored by pervasive and ubiquitous computing and, on the other, whose economy and governance is being driven by innovation, creativity and entrepreneurship, enacted by smart people. This paper focuses on the former and, drawing on a number of examples, details how cities are being instrumented with digital devices and infrastructure that produce ‘big data’. Such data, smart city advocates argue enables real-time analysis of city life, new modes of urban governance, and provides the raw material for envisioning and enacting more efficient, sustainable, competitive, productive, open and transparent cities. The final section of the paper provides a critical reflection on the implications of big data and smart urbanism, examining five emerging concerns: the politics of big urban data, technocratic governance and city development, corporatisation of city governance and technological lock-ins, buggy, brittle and hackable cities, and the panoptic city.

Sage Stats

The University of Michigan Library has just acquired access to Sage Stats, a resource for local area statistics:

Sage Stats features data series on U.S. states, counties, cities, and metropolitan areas. Topics covered include the economy, education, crime, government finance, health, population, religion, social welfare, and transportation. Some series go back more than 20 years. Sage Stats makes it easy to download data, compare indicators or create simple visualizations of local area data.

Access is available to the Ann Arbor, Flint and Dearborn campuses at
http://www.lib.umich.edu/database/link/33391

Apple Research Kit: New Frontiers in Data Collection & Informed Consent

The Apple Research Kit allows researchers to develop an iPhone app, which interested respondents can download from the Apple Store. The respondent goes through an on-line consent form and then responds to questions, tasks (walking), etc. Some of the diagnostic tools are based on previously developed apps from the Apple Healthkit.

As of now, apps have been developed for collecting data for research projects on asthma, cardiovascular disease, diabetes, Parkinson’s, mind, body, and wellness after breast cancer, and for a population-based study, the LGBTQ population.

Here is a description of the informed consent process for these iPhone apps:
Participant-Centered Consent Toolkit

Listed below are a few press releases associated with the Pride Study – the population based study of the gay population. Following those posts are some more general critiques of this way of gathering data. The post from the Verge is probably the most critical raising issues of “on the internet no one knows you are a dog” and gaming the consent process (lying about eligibility for the study). On the plus side, the participant pool is going to be easier to sign up and won’t be limited to those who live close to research hospitals. Here is an excerpt from Business Insider to the reaction to the app launch for the Stanford Heart study:

It’s really incredible … in the first 24 hours of research kit we’ve had 11,000 people sign up for a study in cardiovascular disease through Stanford University’s app. And, to put that in perspective – Stanford has told us that it would have taken normally 50 medical centers an entire year to sign up that many participants. So, this is – research kit is an absolute game changer.

The participant pool is limited to iPhone users (no android version of these apps), although some will have a web interface (the Pride Study).

Launch of the Pride Study
UCSF Researchers Launch Landmark Study of LGBTQ Community Health
Jyoti Madhusoodanan | UCSF Press Release
June 25, 2015

A big LGBT health study is coming to the iPhone
Stephanie M. Lee | BuzzFeed
June 25, 2015

How The iPhone Is Powering A Massive LGBT Health Study
Kif Leswing | International Business Times
June 25, 2015

Critiques of the Apple ResearchKit
Apple’s new ResearchKit: ‘Ethics quagmire’ or medical research aid?
Arielle Duhaime-Ross | The Verge
March 10, 2015

In-Depth: Apple ResearchKit concerns, potential, analysis
mobilehealthnews
March 9, 2015