Lincoln’s Letter To Whom it May Concern
It was common knowledge in 1860 that the federal government held no authority whatsoever regarding social relations and institutions within the States, which were protected from federal encroachment by the Bill of Rights. When Southern agents refused to accept emancipation as a condition for reunion and peace, it was from the view that only the States where African slavery existed had the authority to emancipate. Northern States had accomplished emancipation by themselves and without unconstitutional federal interference or invasion, the South merely wanted the same. And only the States had the authority to prescribe the nature and acceptance of a voluntary union, not a US president.
Bernhard Thuersam, Chairman
North Carolina War Between the States Sesquicentennial Commission
"The Official Website of the North Carolina WBTS Sesquicentennial"
Lincoln’s Letter To Whom it May Concern:
“That summer

[of 1864] was one of the hottest that anyone could remember [in Washington]. Meanwhile, the springtime hopes of an early military breakthrough dissipated in the heat. Grant and Lee appeared to be in a bloody stalemate, as the death tolls mounted. In this stifling and highly charged context, a crisis developed that threw into doubt the wisdom of incorporating the thirteenth amendment into the Union [Republican] Party platform.
Rumors circulated that two Confederate agents at Niagara Falls were prepared to discuss peace and reunion, so long as the insistence on emancipation was dropped. Horace Greeley of the New York Tribune characteristically plunged into the fray: “I beg you, I implore you, to inaugurate or invite proposals for Peace forthwith,” he urged…[President Lincoln] on July 7. Under public pressure, Lincoln wrote a letter on July 18, addressed “to whom it may concern,” stating the terms on which he would accept a settlement. Those terms were “the restoration of the Union” but also “the abandonment of slavery.”
Although, in reality, as Lincoln well knew, the Confederate agents were not empowered to accept peace without disunion…[the agents sent] a report of the conference to the Associated Press. They accused Lincoln of deliberately sabotaging the negotiations by prescribing conditions he knew to be unacceptable.
 [Northern] Democrats maintained that, by insisting on emancipation as a condition of reunion, the administration was prolonging the war. The feeling grew that Lincoln could be beaten in November by any candidate who stood firmly for the Union as it had been, without the revolutionary purpose of emancipation. A Democratic pamphlet that took the form of a spoof biography of “Old Abe” reveals the nature of the political capital that the president’s opponents made from the issue. “He wrote a letter to whom it may concern,” wrote the anonymous author sarcastically, “offering to make peace if the rebels would give up their theories, their property, their lives, and do just as he wanted them to for ever after. What more could be desired?”
Henry J. Raymond wrote privately to the President telling him frankly that he faced [election] defeat.  Public dissatisfaction, he warned, was due to “the impression in some minds, the fear and suspicion in others, that we are not to have peace in any event under this administration until slavery is abandoned.”  To a party leader in Pennsylvania, Raymond noted that the “Niagara letter has been very adroitly and successfully used to [Mr. Lincoln’s] detriment” and despaired that “we have not a ghost of a chance in November” because of the suspicion…that…[he] does not seek peace, that he is fighting not for the Union but for the abolition of slavery.”
 [Raymond] wrote despairingly to the President on August 8, urging him once again to offer peace on only one condition – reunion. [Newspapers friendly to Lincoln] eagerly [defended] Lincoln from the charge that he was waging war to abolish slavery.”
(No Party Now, Politics in the Civil War North, Adam I.P. Smith, Oxford University Press, 2006, pp. 112-113)