As is most likely the case for many other lurkers on the CTPP listserv, I have been
reading the recent slew of ACS-related emails with great interest. Exactly what to make
of all this--I am still not sure.
Conceptually speaking, the ACS concept of "continuous surveying" with
permanent/experienced Census Bureau staff, as opposed to once-every-10-years surveying
with a large group of temp employees, appears very sound. Even the idea of a "moving
five-year average" to get a finer level of area-to-area detail sounds both workable,
better, and timely than what we could ever get in the past. Once a lot of statistical
analysis has been performed, I would expect our leading university researchers to come up
with useful ways to turn various sets of "moving five year averages" into
reasonable specific-year datasets that would be adequate for use with our future local
household survey expansions, population synthesis procedures for travel microsimulations,
and basic "so this is what is happening in our region" data summaries.
Unfortunately, the ACS approach requires not only sufficient sample sizes for any given
year, but a good annual weighting (expansion) process to deal with sub-county geographic
and other biases. If I am understanding things correctly, the only weighting to be
applied will be at the County level? Maybe I am missing something, for I don't see
how this will result in anything but much too much year-by-year variation in household
characteristics--it puts way too much pressure on an underlying implicit assumption that
the number of households in a County that wind up in each year's dataset are truly
representative households for not only the County, but any future sub-County
disaggregations that are prepared. I suspect this is why some folks have been noting some
big year-by-year changes in important household statistics (autos per household, average
household size, etc.) that may not be reflecting real-world annual changes.
I can certainly understand some of the difficulties that would be encountered, but it
seems like some modified approach is needed to re-weight the yearly Household ACS samples
within Counties by something other than a single county-specific factor. If this was a
Year 2000 small-sample local household travel survey we would be able to correct for the
most significant within-County response biases by using the Year 2000 Census data as the
real-world "truth" for coming up with household-specific weighting factors. But
it's not clear what source could be used for a more realistic/useful weighting of
annually-collected ACS records. It's not the current ACS sample size that bothers me
(although I will be distressed if the sample size should ever get reduced from current
levels), but rather the ACS weighting/expansion procedures.
Or...maybe I am missing some important point about the ACS weightings, for which I will
actually be grateful to be publicly corrected. Just call me, "perplexed in
(Dallas-Fort Worth MPO)
[mailto:email@example.com]On Behalf Of Richard Lin
Sent: Tuesday, September 20, 2005 8:31 PM
To: ctpp-news(a)chrispy.net; tzakaria(a)dvrpc.org;
Subject: RE: [CTPP] 2004 ACS Data Release
I completely agree with your comments on the serious limitations
of ACS data. Comparing to your desire to apply the ACS data for
transportation, I am very disappointed about the quality and limitations
of ACS to serve as reference data for evaluating our annual county and
municipal estimates of population and households. However, there are
many other data users out there hoping to get timely demographic
characteristics for making decisions involving either to spend money or
to get money. Less accurate statistics information is still better than
no information for decision making.
Sample statistics from the decennial census long form (from 1/6 of
households or 17 million household samples in Census 2000) indeed are
very much better and reliable (accurate) than that from the 838,000
samples from the 2004 ACS survey in a single year. Without the 2004 ACS
data, we can only depend on the Census 2000 sample statistics. Do you
like that? That is your choice. If so, just don't bother about the ACS
survey data for transportation applications. I am still hope to find
something worthwhile from the annual ACS survey data to check on the
annual estimates of population and households. It is still better than
the March Supplement CPS (Current Population Survey) data which only
contain about 800 (household) samples for Colorado. As of 7-1-2004
Colorado has more than 2,026,000 households. Comparing to CPS, I
appreciate very much to have ACS.
How about canceling the ACS program and campaigning to re-install
Long Form for Census 2010? You all know that if the Senates do not fund
or under fund the ACS test programs for year 2006 and the years after,
it is highly likely that the Census Bureau has to abandon ACS program
and request funding to work on Long Form for 2010 Census. How many of
you would like to see that happen?
>> <TMarchwinski(a)njtransit.com> 9/17/2005
12:51 PM >>>
Dear Mr. Lin- I disagree with you about census long form vs. ACS. I
cannot use the ACS for any transportation planning analysis except maybe
to see where general trends are going. Even there, looking at NJ data
by county, there are some weird changes in mode data even from year to
year. I use the 2000 Census data almost every week, and the ACS, unless
it can get down to a smaller geography, does not do much good. Unless
you are doing some high level comparisons about average travel time or
average mode split etc., it cannot be used for what most transportation
planners need, such as trip distribution, mode split, auto occupancy,
vehicle ownership etc. If there is no long form, then we will have to
take what we can from the ACS and combine it with other data, surveys
[mailto:firstname.lastname@example.org] On Behalf Of Richard Lin
Sent: Friday, September 16, 2005 7:48 PM
To: ctpp-news(a)chrispy.net; tzakaria(a)dvrpc.org;
Subject: RE: [CTPP] 2004 ACS Data Release
We all realize the limitations and quality of the ACS data. Also,
we all are seeking better resolutions. First of all, we hope the Census
Bureau gets enough funding to carrying out ACS operations and to
improving the quality of data by accumulating greater samples. Hope by
Census 2010, the quality of the ACS data is equal to or even better than
that of the traditional long form. Without annual ACS survey, we will
fall back again to the once a 10-year long form data availability. Will
you prefer still using the Census 2000 Long Form data in 2005 or do you
appreciate the limited 2000-2004 ACS data? Without the annual ACS data,
we will go back to the dark again.
The good news is that if the Census Bureau gets funding to
continue ACS survey through Census 2010 then we should have much more
reliable and larger sample ACS data for better quality time series.
With the sampling error variation, we should be more cautious about
time series fluctuations and verify the true changes in numbers such as
the Jew Jersey's Mercer County having lesser workers riding bikes or
walk to work (5,450 or 3.5 in Census 2000 comparing to 2,924 or 1.9 in
ACS 2004). We should also be cautious when we compare apple to orange.
We may check the data at state (NJ) level by comparing the state's
Census 2000 numbers and ACS 2000 numbers to establish a ratio factor for
county adjustment for time series comparison.
>> "Murakami, Elaine"
Thank you for providing your opinion on the disutility of the ACS 2004
data for your applications, and I am glad that others are examining data
for their area and are sharing their findings with the listserv.
My earlier email was sent to CAUTION people who might try to compare
2000 decennial data to ACS 2004 data. As I said in my earlier email, it
is probably better to compare ACS results to ACS results, than to
compare ACS results to decennial census results, however, results from
ACS from 2000 are very limited. The geographic coverage of County level
data from the ACS in 2000 is sparse, which makes comparisons for ACS
between 2000 and 2004 possible only in some areas.
The ACS did not go into "full implementation" until 2005, and it will
require multiple years of data collection before data for small areas
will be reliable because of the small samples collected each month and
averaged over time. As Tom Marchwinski pointed out, averaging over
multiple years creates other problems. In 2005, the CB begin
differential non-response follow-up, so that in areas (tracts) with low
mail-back responses, there is a greater rate (1:2 and 2:5 instead of
1:3) made to follow-up non-respondents from the mail-back and CATI
portion of the ACS. Not surprisingly, the areas with low mail-back
responses are more likely to be low income, and higher shares of African
American and Hispanic populations. My guess is that this will cause a
shift in numbers between 2004 and 2005, and could impact variables such
as carpooling and transit use and number of vehicles in households.
The Census Bureau recommends that the ACS should be used to describe
characteristics, and not to use it for COUNTS. When I examine the
Mercer County NJ data comparing Census 2000 (workers in households) to
ACS 2004 (workers in hhlds), I find that about the only thing to say is
that "driving alone" appears to increase from 77 to 78%, and
"carpooling" to decrease from 11 to 10%, "rail" also appears to
from 4 to nearly 6%. (Worked at Home is not included in column in the
table below). HOWEVER, this is not taking into account the effects of
the different survey methods, where, generally speaking, the decennial
census has a greater share reporting "carpooling," which is why a
BRIDGE from decennial 2000 data is so important. Thus, it is probably
incorrect to say that carpooling is declining.
Mercer County, NJ
Workers in Hhlds Census 2000 ACS 2004
Number Pct Number Pct
Total 153,665 153,041
Drove alone 118,390 77.0 119,597 78.1
2-person CP 13,105 8.5 12,026 7.9
3+ person CP 4,580 3.0 2,722 1.8
Bus/trolley bus 4,585 3.0 4,390
Streetcar/trolleycar+ 195 0.1 1,168 0.8
Railroad or ferry 6,105 4.0 8,795
Bike or Walk* 5,450 3.5 2,924 1.9
Taxi/motorcycle/other 1,255 0.8 1,419 0.9
* 2000 is walk + bike, 2004 is walk only
The county estimates program which is used to weight the ACS data is
drawing considerable fire, as evidenced by the post by Jeffrey P. Levin
from the City of Oakland.
Things for State DOTs and MPOs to consider if you feel that the ACS
will not provide you with quality data:
1. Can your organization leverage enough political resources to bring
back a Census "long form" ?
2. Would improvements to the county estimates program make you feel
more comfortable with the ACS results?
3. Should your organization consider conducting a very large sample
survey, similar to the surveys conducted in the 1950's and 1960's where
sample sizes of 3 - 5 % of all households were asked to completed a
travel diary? One of the goals of these surveys was to produce an O/D
matrix for a limited number of zones. An area with 1 million population
might have 400,000 households, therefore a 4% sample would be 16,000
households. Let's estimate the cost of a household survey at a
conservative $150 per complete, resulting in a estimate of $2.4 million.
Keep in mind that the response rates to recent regional household
travel surveys have been between 25-30%, which is much lower than the
ACS, thus, risking much higher sample bias. Once you get the results,
you will need to determine a method to weight your results for regional
4. Should your organization implement a survey on group quarters
population, or do you believe that the ACS will include group quarters
in 2006, as planned.
5. Should you find an alternative data source for home-to-work flows.
Sorry for the long post, and hope that my table comes over without
FHWA Office of Planning