15 Apr
2009
15 Apr
'09
11:22 a.m.
From a state planning perspective, I think it is very
disappointing (whether it is due to new tighter disclosure rules or the statistical
reliability issue) that it appears that states and MPOs will not have the same quality of
CTPP data available in 2010 as in 2000 and previous census cycles. Particularly since
states are paying considerably more for the next CTPP than for 2000. Virginia's share
more than tripled in cost from 2000 to 2010. I agree with the earlier point that if states
are paying for the special tabulations, there should not be these new disclosure
restrictions governing the release of the data. I am not clear on why the disclosure
restrictions are even an issue since I am not aware of any past disclosure issues with the
2000 or prior CTPP and mining CTPP data would not appear to be a particularly effective
way for someone to try to find out personal info. about someone else. If this is such an
issue, why was it not a problem in the past?
While it may be too late to fix the problems associated with data quality/content for the
next CTPP, I'm wondering if there are ways that the ACS sample could be increased with
additional federal and/or state support in the future so that this statistical reliability
issue could be addressed, perhaps in a process similar to the way NHTS is done, or perhaps
CTPP data needs to come from a different source long term if the ACS data is not reliable
enough to meet state and metropolitan planning needs.
Decision makers increasingly want to see more robust technical tools and analysis which to
support planning analysis which requires more detailed data at the small area from sources
such as ACS, CTPP, NHTS, etc., and major conferences, e.g., TRB, and federal agencies have
supported better data for transportation planning for years. Therefore, from a state
perspective, the Census Bureau's proposed policy change run counter, not only to
prevailing trends, but to the policies from other federal transportation agencies.
-------------------------------------------
Paul T. Agnello
Travel Demand Modeling Manager
Virginia Department of Transportation
Transportation & Mobility Planning Division
1401 East Broad Street Telephone (804) 786-2531
Richmond, Virginia 23219-2000 Fax (804) 225-4785
E-mail: mailto:paul.agnello@VDOT.Virginia.gov
Website: http://www.virginiadot.org/
* CONFIDENTIALITY/PRIVACY NOTICE - The documents included in this transmission may
contain information that is confidential and/or legally privileged. If you are not the
intended recipient, or the employee or agent responsible for delivering the information to
the intended recipient, you are hereby notified that any disclosure, copying,
distribution, or action taken in reliance on the contents of these documents is strictly
prohibited. If you have received this document in error, please notify the sender
immediately to arrange for return or destruction of these documents.
-----Original Message-----
From: Ed Christopher [mailto:edc@berwyned.com]
Sent: Wednesday, April 15, 2009 12:07 AM
To: Graham, Todd; Penny Weinberger
Cc: Murakami, Elaine; Agnello, Paul; banningag(a)michigan.gov;
sandy.beaupre(a)dot.state.wi.us; kcooper(a)dot.state.nv.us;
jonette.kreideweis(a)dot.state.mn.us; phil.mescher(a)dot.state.ia.us; ayalew.adamu(a)dot.ca.gov;
pleasantmd(a)scdot.org; virginia.porta(a)arkansashighways.com; bobbi.retzlaff(a)dot.state.wi.us;
robbins(a)wsdot.wa.gov; huiwei.shen(a)dot.state.fl.us; amy.thomas(a)ky.gov;
donna.weaver(a)po.state.ct.us; willimasjs(a)dot.state.al.us; sharon.ju(a)h-gac.com;
kmiller(a)njtpa.org; Pickard, Andy, P.E.; creschovsky(a)mwcog.org;
grousseau(a)atlantaregional.com; Fred(a)NARC.org; rdenbow(a)ampo.org; dhardy(a)ampo.org;
rmccready(a)aashto.org; Memmott, Jeff <RITA>; nsrinivasan(a)nas.edu; Weiner, Ed
<OST>; Ron.fields(a)arkansashighways.com; nerlbaum(a)dot.state.ny.us
Subject: Re: Census Bureau Federal Register Notice on New Data Disclosure Restrictions
Todd--You should have posted to the full CTPP listserve. You make all
good points that people need to think about. One point of clarification
is that the AASHTO CTPP Oversight Board has sent a new proposal of
tables over to the CB for 3-year data and in fact the mode to work
questions are rolled up. I believe the largest roll-up is 3 modes:
auto, other and total. I took on the task of posting the new tables to
the Listserve but haven't done so yet. The tables are still fresh off
the press and I will not be able to get to it till Thursday.
One point that is missed in all of this is that with the 3-year data we
are talking about a zone system with 20,000 people per zone. That is
the size of 7 or so tracts. Pretty big when you are talking about
planning within a region. Even though you are right about the
statistical quality of the data the CB is not telling us there is a
statistical reason for suppressing the data. They are basing it solely
on disclosure requirements. Disclosure requirements and arguments that
can not be proven. Another point that is missed is that the old long
form data suffered from the same relative thinness at the tract, block
group and TAZ level. At those levels of geography many of the same
tables that are not passing the disclosure rules now would not have
passed then. Yes, the data today is a little thinner but that has never
been the issue.
One last point is that CTPP has always been a special tabulation and if
you push the issue far enough logic should dictate that someone
purchasing a special tabulation should be allowed to buy whatever data
they want no matter how crappy it is. In 1980 the CB used to sell us
the data with a "caveat emptor" sticker on it.
Putting all the cards on the table does point to why synthetic data for
small area analysis is so important. Unfortunately there are not only
statistical issues of methodology to deal with but also practical issues
of political acceptance within the community.
Graham, Todd wrote:
State and MPO colleagues--
Discussions about Census data disclosure have been making the rounds. I wanted to share
a few thoughts with the SCOP Census Data Work Group...
Re: protecting the individual confidentiality of respondents. True, the Bureau itself is
standing this up as their decision basis (it's a legally powerful position). But I
think the larger, latent, real concern among the Bureau statisticians is statistical
reliability.
We know ACS sampling is thin (1 in 8 households surveyed, 60-65% response rates?) and
temporally spread out... And we know there will be large numbers of individual data cells
in the planned CTPP-from-ACS tabs where estimates would be based on just 1 or 2
respondents. As a statistician, I really don't like this. Resulting estimates are not
robust. (There's great uncertainty around whether the 1 or 2 survey respondents should
represent 10-20 other people - or perhaps, by freak luck, the 1 or 2 persons are unique.
There is real probability of 1 or 2 respondents being *not* representative.) This is
particularly true in the most highly-detailed crosstabs. Hundreds of cells in a table
*will* result in many cells with small numbers.
I know some members of SCOP have been drafting comments in response to the Fed Register
Notice. Some of these comments will make emphatic proposals that we must have fully
populated CTPP tables. I worry that this line of reasoning won't have much traction
at Census Bureau... And really, do we believe that any numbers (regardless of
statistical reliability) are better than no numbers?
As a statistician, I disagree - and I think there are creative alternatives that are
viable: (1) SCOP and Census Data Workgroup have discussed data synthesis techniques to
simulate or synthesize the desired details. Or (2), a more conventional solution, more
highly aggregated ("rolled up") categorization in the CTPP-from-ACS tabs. What
SCOP requested in 2007 looks a lot like CTPP 2000 -- even though we knew that ACS Survey
sampling is thinner than Census 2000.
Sorry to bring this up, but here goes: AASHTO SCOP should revisit the CTPP-from-ACS
design, reopen it for discussion, go back to the drawing board, and consider more highly
aggregated ("rolled up") categorizations in the CTPP-from-ACS tabs. Do we
really need 10 (or 17) categories of mode of travel (can we live with fewer)? Do we
really need 25 categories of household income (can we live with fewer)? Do we really need
tabs with travel-start-time expressed in 15-minute intervals??
I know there are sunken costs already. Still, my candid advice: AASHTO SCOP and other
funding partners in the CTPP need to take a deep breath... and consider revising the
special tabs requests. And Census Bureau should cooperate and enable such a new plan.
I understand that people are up-at-arms about this. (Census Bureau, for their part,
waited until 2008 to clearly signal that there would be a tighter data disclosure regime
than experienced in CTPP 2000...) But the realpolitic is: Census Bureau statisticians
have already decided this matter -- isn't the Fed Register notice just a formality? --
and from the standpoint of good statistical science, their decision is right.
-- Todd Graham
________________________
Todd Graham
Principal Forecaster
Metropolitan Council
390 Robert Street North
Saint Paul, MN 55101
phone 651/602-1322
email todd.graham(a)metc.state.mn.us
web www.metrocouncil.org
www.metrocouncil.org/metroarea/stats.htm
________________________
--
Ed Christopher
708-283-3534 (V)
708-574-8131 (cell)
FHWA RC-TST-PLN
19900 Governors Dr
Olympia Fields, IL 60461