Re: [CTPP] FW: 2012-16 CTPP Question about Counts

Ed Thanks for pursuing this issue. As you’ve suggested, we will total the row in the table to get the correct figures for now and go from there.  I assume that the on-line data tool will get fixed, but this work may take some time. I want to give a shout out to my intern Bailey Werner who brought this to my attention. Cliff *From:*Ed Christopher &lt;edc(a)berwyned.com&gt; *Sent:* Thursday, August 15, 2019 10:20 AM *To:* Cook, Cliff &lt;ccook(a)cambridgema.gov&gt;ov>; ctpp-news(a)chrispy.net *Cc:* Werner, Bailey &lt;bwerner(a)cambridgema.gov&gt; *Subject:* Re: [CTPP] FW: 2012-16 CTPP Question about Counts Cliff et al--don't be perplexed ;-) When preparing large data sets stuff happens. Our friends at the bureau looked into Table 303100 and the issue with subcategories not adding up to total counts. Good news! It's not a data issue, it's a software tool issue. In the CTPP data tool, they were able to verify the discrepancies between the totals and subtotals observed for Cambridge. they then checked the numbers for the original tables delivered from the Census Bureau. Those tables are correct. That is, the subtotals add up to the totals (with typical minor rounding discrepancies, of course). For now the assumption is that  it is just a simple instance of the table drawing from the wrong total line in the tool. Needless to say there is some circling back to the software folks. On 8/14/2019 9:54 AM, Cook, Cliff wrote: Ed I thought you’d be interested in the following from Le Zhang from NYC DEP: I just checked the Table B303100 state-to-state flows. The estimated US total workers is over 232M, whereas the sum of all income brackets is only 145M. The latter makes much more sense to me. Therefore, I think the issue very likely has a national impact. We continue to be perplexed by this problem.  If anyone has any suggestions for the source we are interested to learn more. Cliff Cook *From:*Ed Christopher &lt;edc(a)berwyned.com&gt; <mailto:edc@berwyned.com> *Sent:* Monday, August 12, 2019 1:24 PM *To:* ctpp-news(a)chrispy.net <mailto:ctpp-news@chrispy.net>; Cook, Cliff &lt;ccook(a)cambridgema.gov&gt; <mailto:ccook@cambridgema.gov> *Cc:* Werner, Bailey &lt;bwerner(a)cambridgema.gov&gt; <mailto:bwerner@cambridgema.gov> *Subject:* Re: [CTPP] FW: 2012-16 CTPP Question about Counts Cliff--The geocoding issue that was mentioned is not affecting or part of your issue. That is a whole different issue. As to what you are seeing in Cambridge I still do not have an answer for the Part 3 income tables (B303100 and B303201) but I think we could be getting close to figuring it out. I just checked the numbers for my town which is much smaller and has no Group Quarters. That means that I should get consistent numbers across all the Part 3 tables which I did except for the income tables. BTW, my town of Berwyn IL has a total of 2,600 internal worker flows, people living and working in Berwyn. Of course tables A302100 and A302103 (total workers and worker by mode) gave me 2,595 workers but we know that difference is due to disclosure proofing. Now on to the income table. Typically the cells in a CTPP table will not add up to the total because the totals are rounded independently of the cells (more disclosure proofing). However they would never be as far off as we are seeing. In checking my town's Part 3 income tables I discovered that the cells added to the total number I was expecting (2,600) while the total was published at 4,000. The question now becomes what does the total represent? Whatever it is looks to be systematic which means we will be able to figure it out. More digging needed though. On 8/12/2019 11:22 AM, Cook, Cliff wrote: Dara Thank you for these materials.  They are very helpful.  After taking a closer look at the CTPP flow tables we are still left with a question about why the components of certain tables do not add up to the table total.  For example when we filter for Cambridge, MA as both residence and workplace, the “Total” shown in Table B303100 is 37,500 while the component cells add up to 22,470.  This problem appears in all the part 3 tables whose universe is household workers.  (We do not see a similar issue in all tables based purely in worker characteristics, regardless of living arrangement.  Those seem to consistently sum to 27,725.) Can anyone shed light on why the table components to sum to the sum stated in the table? Thanks Cliff Cook *Clifford Cook Senior Planning Information Manager* Cambridge Community Development Department 344 Broadway, Cambridge, MA. 02139 cid:image001.png@01CF4355.A65408C0 <https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2FCDDat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007288470&sdata=dwx2Z0l4Xhtdqp7i7xIGsXifvCyRbHhnkrsdgLEOjH8%3D&reserved=0> cid:image002.jpg@01CF4357.3478C720 <https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fcddat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007298469&sdata=O709duLEYROPOWi0s1nMXkCbA851VyG193pagzxd0JI%3D&reserved=0> cid:image010.jpg@01CF4357.3478C720 <https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Finstagram.com%2Fcddat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007298469&sdata=inffCsaXMuQjaam%2BuX%2BXDKvl8NP31gcruW9fLoQ5y7w%3D&reserved=0> *www.cambridgema.gov/CDD* <https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cambridgema.gov%2FCDD.aspx&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007308457&sdata=B9QpG%2BEzhj6dvq4ladXiA7MjacNUlibPLiZe3zdJL48%3D&reserved=0> ccook(a)cambridgema.gov <mailto:ccook@cambridgema.gov> M: 8:30-8:00   T-Th:  8:30-5:00  F: 8:30-Noon 617/349-4656 617/349-4669  FAX            617/349-4621  TTY *From:* Dara Goldberg (DCP) &lt;DGoldberg(a)planning.nyc.gov&gt; <mailto:DGoldberg@planning.nyc.gov> *Sent:* Monday, August 12, 2019 11:13 AM *To:* Cook, Cliff &lt;ccook(a)cambridgema.gov&gt; <mailto:ccook@cambridgema.gov> *Cc:* Werner, Bailey &lt;bwerner(a)cambridgema.gov&gt; <mailto:bwerner@cambridgema.gov>; Le Zhang (DCP) &lt;LZHANG(a)planning.nyc.gov&gt; <mailto:LZHANG@planning.nyc.gov> *Subject:* RE: 2012-16 CTPP Question about Counts Hello Cliff, The reason the flows are off is because the CTPP methodology for reporting geographies below the county level has been revised for this release (though AASHTO has yet to release documentation stating such). Missing records for smaller geographies are no longer imputed for Parts 2 and 3, therefore, tracts/places will no longer add to counties, nor will they have complete counts. This has resulted in a “truer” data set, albeit with missing records. Part 1 remains the same (i.e. “complete”), because it represents ACS data at place of residence. I’ve attached a memo summarizing the challenges with the new release. My colleague Le, who I have CC’d, put together the attached presentation explaining this issue, which we shared with our MPO members a couple of months ago. Le will be presenting this to the CTPP oversight board in Arkansas next week. Unfortunately, there was no documentation released accounting for the methodological change, and we believe transportation planners will rely on the current release to have a “complete” count as the prior releases had.  Therefore, this release is not comparable to the prior releases for smaller geographies. In the meantime, we have assumed a proportional allocation to account for missing records (though this of course introduces a bias into the results – also covered in Le’s presentation). Hopefully this helps! Happy to answer any questions you might have. Regards, DARA GOLDBERG SENIOR PLANNER | REGIONAL PLANNING DIVISION NYC DEPT. OF CITY PLANNING 120 BROADWAY, 31^st FLOOR• NEW YORK, NY 10271 212-720-3312 I _DGOLDBERG(a)planning.nyc.gov <mailto:DGOLDBERG@planning.nyc.gov>_ http://www.nyc.gov/planning <https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.nyc.gov%2Fplanning&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007308457&sdata=ry2SkbLTG4Wd78C76UeZAdrJdn4fIUHcttEuLWoNXYM%3D&reserved=0> *From:* ctpp-news [mailto:ctpp-news-bounces@chrispy.net] *On Behalf Of *Cook, Cliff *Sent:* Monday, August 12, 2019 10:58 AM *To:* ctpp-news(a)chrispy.net <mailto:ctpp-news@chrispy.net> *Cc:* Werner, Bailey &lt;bwerner(a)cambridgema.gov <mailto:bwerner@cambridgema.gov>> *Subject:* [CTPP] 2012-16 CTPP Question about Counts To All We are working to collect information about the resident  labor force in Cambridge, Massachusetts. We set the residence as the State-Place of Cambridge city, MA, and the workplace as POW State-Place of Cambridge city, MA. The numbers in the CTPP Flows tables are not adding up as expected. Table A304100 – Total workers (1) (Workers 16 years and over) provides an estimate of _27,725_ (MOE 847), whereas Table B303100 – Household income in the past 12 months (2016$) (9) (Workers 16 years and over in households) provides a total estimate of _37,300_ (MOE 2,054). Furthermore, when we add up the count of workers in each income bracket in Table B303100 they sum to _22,470_. I could understand if the total number of resident workers 16 and older in households was smaller than total workers over 16, but we cannot make sense of how the reverse could be true.  It also doesn’t explain why the sum of all categories is smaller than the listed total. Could data suppression account for this?  That would seem unlikely at the level of a city of our size. Could the results be due to data suppression at smaller geographic levels having a ripple effect on a larger geo?  I understand workers with an unclear or imprecise work address are excluded from the flow data.  Are these issues a result of that screening or is this a different type of issue? Interestingly, the numbers make sense as expected when we look at the Residence tables for the same geography. Table A102101 – Total workers (1) (Workers 16 years and over) provides an estimate of 61,925 (MOE 1,008) and Table A103100 – Total Workers in households (1) (Workers 16 years and over in households) estimates 54,195 (MOE 1,075). Any help on interpreting our resident labor force stats is appreciated. Cliff Cook *Clifford Cook Senior Planning Information Manager* Cambridge Community Development Department 344 Broadway, Cambridge, MA. 02139 cid:image001.png@01CF4355.A65408C0<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2FCDDat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007318452&sdata=qRT9U7LadxqEvSWCpNOeD8aiTe83hzm3QPtxTOyE2Cs%3D&reserved=0>cid:image002.jpg@01CF4357.3478C720<https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fcddat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007318452&sdata=FJv7VfNlxjD%2FS4gEDuvDWH19L29%2FeWxDKinZNuIcOu0%3D&reserved=0>cid:image010.jpg@01CF4357.3478C720<https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Finstagram.com%2Fcddat344&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007328448&sdata=V5AeQVWiOiCrAHD%2BfuRVGuBful7p67u6bt9qkJkKWWQ%3D&reserved=0> *www.cambridgema.gov/CDD*<https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cambridgema.gov%2FCDD.aspx&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007328448&sdata=DJfWOAAE58jTC7wwNn2%2F5yJ2sr8z1Ht6OJG9gFzw%2B0w%3D&reserved=0> ccook@cambridgema.gov<mailto:ccook@cambridgema.gov> M:  8:30-8:00   T-Th: 8:30-5:00  F:  8:30-Noon 617/349-4656 617/349-4669  FAX            617/349-4621  TTY _______________________________________________ ctpp-news mailing list ctpp-news(a)chrispy.net <mailto:ctpp-news@chrispy.net> https://www.chrispy.net/mailman/listinfo/ctpp-news <https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.chrispy.net%2Fmailman%2Flistinfo%2Fctpp-news&data=02%7C01%7Cccook%40cambridgema.gov%7C4b56c9d2e44944ba3f8a08d7218ba74a%7Cc06a8be784794d73b35193bc9ba8295c%7C0%7C0%7C637014756007338443&sdata=6s7hHsBpHkfkFD135tBn3APVRGiui0RxmuxUwLG0x6o%3D&reserved=0> -- Ed Christopher Transportation Planning Consultant 708-269-5237 -- Ed Christopher Transportation Planning Consultant 708-269-5237