The "Lincoln" quote
On April 4, 1861, eight days before the incident at Fort Sumter, Lincoln had a conference with Col. John B. Baldwin, a unionist representative to the Virginia secession convention; and on February 10, 1866, Baldwin testified before the Reconstruction Committee of the US Congress in reference to that meeting. The entire testimony together with statements and testimonies of others and letters related to those testimonies may be found in the Special Collections Department, University of Virginia Library, Charlottesville, Virginia. It consists of more than 19,000 words, 4,333 words of testimony by Baldwin alone.
Here is an excerpt from that testimony after Baldwin has suggested to Lincoln that he can defuse tensions by withdrawing the troops from Fort Sumter:
"Well," said he [Lincoln], "what about the revenue? What would I do about the collection of duties?"
Said I [Baldwin], "Sir, how much do you expect to collect in a year?"
Said he [Lincoln], "Fifty or sixty millions."
"Why sir," said I [Baldwin], "four times sixty is two hundred and forty. Say $250,000,000 would be the revenue of your term of the presidency; what is that but a drop in the bucket compared with the cost of such a war as we are threatened with? Let it all go, if necessary; but I do not believe that it will be necessary, because I believe that you can settle it on the basis I suggest."
He [Lincoln] said something or other about feeding the troops at Sumter. I told him that would not do. Said I, "You know perfectly well that the people of Charleston have been feeding them already. That is not what they are at. They are asserting a right. They will feed the troops and fight them while they are feeding them. They are after the assertion of a right. Now, the only way that you can manage them is to withdraw from them the means of making a blow until time for reflection, time for influence which can be brought to bear, can be gained, and settle the matter. If you do not take this course, if there is a gun fired at Sumter—I do not care on which side it is fired—the thing is gone."
"Oh," said he [Lincoln], "sir, that is impossible."
Said I [Baldwin], "Sir, if there is a gun fired at Sumter, as sure as there is a God in heaven the thing is gone. Virginia herself, strong as the Union majority is now, will be out in forty-eight hours."
I have on occasion read that Lincoln in some other context said in reference to the Southern states leaving the Union something like "Then who will pay for the government?" But as yet I have found no primary (original) source for it. I will appreciate it very much if anyone can provide that source.