Memoir 1, Page 78, The 1856 Election

"I voted for Filmore in 1857, and never before nor since have I cast a political vote."

The 1856 election came down to three main candidates: James Buchanan of the Democratic Party, John C. Fremont of the new Republican Party, and Millard Fillmore representing the American (Know-Nothing) Party. Fillmore had been risen to President of the US following the death of Zachary Taylor two terms before, playing a crucial role in the passage of the Compromise of 1850. Despite his association with the Know-Nothings in the 1856 election, he spoke very little on immigration, focusing largely on the preservation of the Union in the face of impending civil war. James Buchanan ultimately won the election, with the results directly reflecting the rising sectionalism that Fillmore had feared.

"I was in a locality where a Republican scarcely dared to express himself by vote or word."

Unlike today where voting is done in secret and using the same ballot across the country, ballots in the mid-nineteenth century had to be provided by the candidates themselves. This was done by finding support in every state from not only registered voters but also publishers, usually newspapers, who would agree to print ballots with the candidate’s name. Lincoln was unsuccessful in finding this support in the majority of the Southern states, at least partially because few people wanted to declare public support for Lincoln while surrounded by outspoken Southerners. As a result, Lincoln’s name was not on the ballot in ten of the eventual eleven Confederate states. While some locations in Virginia did list his name, they were rare, giving Lincoln only 1.5% of the total vote in the state.