How These Polls Were Conducted
Here are the key things to know about these polls:
• Interviewers spoke with 2,437 likely voters from Sept. 11 to 16, 2024.
• Times/Siena polls are conducted by telephone, using live interviewers, in both English and Spanish. Overall, about 96 percent of respondents were contacted on a cellphone for these polls.
• Voters are selected for the survey from a list of registered voters. The list contains information on the demographic characteristics of every registered voter, allowing us to make sure we reach the right number of voters of each party, race and region. For this poll, interviewers placed more than 240,000 calls to nearly 118,000 voters.
• To further ensure that the results reflect the entire voting population, not just those willing to take a poll, we give more weight to respondents from demographic groups that are underrepresented among survey respondents, like people without a college degree. You can see more information about the characteristics of our respondents and the weighted sample at the bottom of the page, under “Composition of the Sample.”
• The national poll’s margin of sampling error among likely voters is plus or minus 3 percentage points. The Pennsylvania poll’s margin of error among likely voters is plus or minus 3.8 percentage points. In theory, this means that the results should reflect the views of the overall population most of the time, though many other challenges create additional sources of error. When computing the difference between two values — such as a candidate’s lead in a race — the margin of error is twice as large.
If you want to read more about how and why The Times/Siena Poll is conducted, you can see answers to frequently asked questions and submit your own questions here.
Full Methodology
The New York Times/Siena College poll of 2,437 registered voters nationwide, including polls of Pennsylvania and Philadelphia conducted in partnership with the Philadelphia Inquirer, was conducted in English and Spanish on cellular and landline telephones from Sept. 11 to 16, 2024.
The margin of sampling error for the national poll is plus or minus 3 percentage points for the likely electorate. For the Pennsylvania poll, the margin of sampling error is plus or minus 3.8 percentage points for the likely electorate. Among registered voters, the margin of error is plus or minus 2.8 percentage points nationally and plus or minus 3.6 percentage points in Pennsylvania.
The Pennsylvania and Philadelphia polls were funded by a grant from the Lenfest Institute for Journalism. The polls, which were designed and conducted independently from the institute, were weighted so that in the end, respondents in Philadelphia represent the proper proportion of the poll of Pennsylvania. Similarly, the national poll was weighted so that respondents from Pennsylvania represent the proper proportion of the country.
Sample
The survey is a response rate-adjusted stratified sample of registered voters on the L2 voter file. The sample was selected by The New York Times in multiple steps to account for differential telephone coverage, nonresponse and significant variation in the productivity of telephone numbers by state.
First, records were selected by state. In Pennsylvania, records were selected separately for the poll of Philadelphia and of the rest of the state To adjust for noncoverage bias, the L2 voter file was stratified by statehouse district, party, race, gender, marital status, household size, turnout history, age and home ownership. The proportion of registrants with a telephone number and the mean expected response rate were calculated for each stratum. The mean expected response rate was based on a model of unit nonresponse in prior Times/Siena surveys. The initial selection weight was equal to the reciprocal of a stratum’s mean telephone coverage and modeled response rate. For respondents with multiple telephone numbers on the L2 file, the number with the highest modeled response rate was selected.
Second, state records were selected for the national sample. The number of records selected by state was based on a model of unit nonresponse in prior Times/Siena national surveys as a function of state, telephone number quality and other demographic and political characteristics. The state’s share of records was equal to the reciprocal of the mean response rate of the state’s records, divided by the national sum of the weights.
Fielding
The sample was stratified according to political party, race and region and fielded by the Siena College Research Institute, with additional field work by ReconMR, the Public Opinion Research Laboratory at the University of North Florida, the Institute of Policy and Opinion Research at Roanoke College and the Center for Public Opinion and Policy Research at Winthrop University in South Carolina. Interviewers asked for the person named on the voter file and ended the interview if the intended respondent was not available. Overall, across all three samples, 96 percent of respondents were reached on a cellular telephone.
The instrument was translated into Spanish by ReconMR. Bilingual interviewers began the interview in English and were instructed to follow the lead of the respondent in determining whether to conduct the survey in English or Spanish. Monolingual Spanish-speaking respondents who were initially contacted by English-speaking interviewers were recontacted by Spanish-speaking interviewers. Overall, across all three samples, 12 percent of interviews among self-reported Hispanics were conducted in Spanish, including 15 percent of weighted interviews.
An interview was determined to be complete for the purposes of inclusion in the ballot test question if the respondent did not drop out of the survey by the end of the two self-reported variables used in weighting — age and education — and answered at least one of the age, education or presidential election ballot test questions.
Weighting — registered voters
The survey was weighted by The Times using the R survey package in multiple steps.
First, the sample was adjusted for unequal probability of selection by stratum.
Second, each poll was weighted to match voter file-based parameters for the characteristics of registered voters.
The following targets were used:
• Party (party registration if available in the state, else classification based on participation in partisan primaries if available in the state, else classification based on a model of vote choice in prior Times/Siena polls) by a classification of how strongly partisan the respondent is based on a model of vote choice in prior Times/Siena polls
• Age (Self-reported age, or voter file age if the respondent refuses) by gender (L2)
• Race or ethnicity (L2 model)
• Education (four categories of self-reported education level, weighted to match NYT-based targets derived from Times/Siena polls, census data and the L2 voter file)
• White/non-white race by college or non-college educational attainment (L2 model of race weighted to match NYT-based targets for self-reported education)
• Marital status (L2 model)
• Home ownership (L2 model)
• Turnout history (NYT classifications based on L2 data)
• National region, if part of the national sample (NYT classifications by state)
• Method of voting in the 2020 elections, if part of the national sample (NYT classifications based on L2 data)
• Metropolitan status, if part of the national sample (2013 NCHS Urban-Rural Classification Scheme for Counties)
• Census tract educational attainment, if part of the national sample
• State region, if part of the Pennsylvania sample
• Record of requesting an absentee ballot for the 2024 general election, if part of the Pennsylvania or Philadelphia samples
Third, the sums of the weights of the polls were balanced so that respondents in Philadelphia represented the proper proportion of the Pennsylvania poll and respondents in Pennsylvania represented the proper proportion of the national poll.
Finally, the sample of respondents who completed all questions in the survey was weighted identically, as well as to the result for the general election horse race question (including leaners) on the full sample.
Weighting — likely electorate
The survey was weighted by The Times using the R survey package in multiple steps.
First, the samples were adjusted for unequal probability of selection by stratum.
Second, the first-stage weight was adjusted to account for the probability that a registrant would vote in the 2024 election, based on a model of turnout in the 2020 election.
Third, the sample was weighted to match targets for the composition of the likely electorate. The targets for the composition of the likely electorate were derived by aggregating the individual-level turnout estimates described in the previous step for registrants on the L2 voter file. The categories used in weighting were the same as those previously mentioned for registered voters.
Fourth, the initial likely electorate weight was adjusted to incorporate self-reported intention to vote. Four-fifths of the final probability that a registrant would vote in the 2024 election was based on their ex ante modeled turnout score and one-fifth based on their self-reported intentions, based on prior Times/Siena polls, including a penalty to account for the tendency of survey respondents to turn out at higher rates than nonrespondents. The final likely electorate weight was equal to the modeled electorate rake weight, multiplied by the final turnout probability and divided by the ex ante modeled turnout probability.
Fifth, the sums of the weights of the polls were balanced so that respondents in Philadelphia represented the proper proportion of the Pennsylvania poll and respondents in Pennsylvania represented the proper proportion of the national poll.
Finally, the sample of respondents who completed all questions in the survey was weighted identically, as well as to the result for the general election horse race question (including leaners) on the full sample.
The margin of error accounts for the survey’s design effect, a measure of the loss of statistical power due to survey design and weighting.
For the national poll, the design effect is 2.3 for the likely electorate and 2.05 for registered voters. The margin of error for the sample of respondents who completed the entire survey is plus or minus 3.3 percentage points for the likely electorate, including a design effect of 2.33, and plus or minus 3.2 percentage points for registered voters, including a design effect of 2.1.
For the Pennsylvania poll, the design effect for the full sample is 1.60 for the likely electorate and 1.51 for registered voters. The margin of error for the sample of respondents who completed the entire survey is plus or minus 4.3 percentage points for the likely electorate, including a design effect of 1.62, and plus or minus 4.1 percentage points among registered voters, including a design effect of 1.54.
Historically, The Times/Siena Poll’s error at the 95th percentile has been plus or minus 5.1 percentage points in surveys taken over the final three weeks before an election. Real-world error includes sources of error beyond sampling error, such as nonresponse bias, coverage error, late shifts among undecided voters and error in estimating the composition of the electorate.
