Here are some parts of a charming letter (X:96) by Pliny the Younger.
At the time of writing, around 112 CE, Pliny was the governor of a Roman province in a region of modern-day Turkey. He is asking for advice from the Roman Emperor Trajan about how strict he should be in punishing his subjects for being Christians:
It is my custom, Sir, to refer to you in all cases where I do not feel sure, for who can better direct my doubts or inform my ignorance? I have never been present at any legal examination of the Christians, and I do not know, therefore, what are the usual penalties passed upon them, or the limits of those penalties, or how searching an inquiry should be made. I have hesitated a great deal in considering whether any distinctions should be drawn according to the ages of the accused; whether the weak should be punished as severely as the more robust; whether if they renounce their faith they should be pardoned, or whether the man who has once been a Christian should gain nothing by recanting; whether the name itself, even though otherwise innocent of crime, should be punished, or only the crimes that gather round it.
As the quote makes clear, Pliny is not sure what the punishments for Christians should be, but later in the letter, it seems that the death penalty was a common option:
Many persons of all ages, and of both sexes alike, are being brought into peril of their lives by their accusers.
One issue that he ran into, was that, on fairly careful investigation, the activities of Christians seemed oddly benign:
[Recanting Christians] declared that the sum of their guilt or their error only amounted to this, that on a stated day they had been accustomed to meet before daybreak and to recite a hymn among themselves to Christ, as though he were a god, and that so far from binding themselves by oath to commit any crime, their oath was to abstain from theft, robbery, adultery, and from breach of faith, and not to deny trust money placed in their keeping when called upon to deliver it. When this ceremony was concluded, it had been their custom to depart and meet again to take food, but it was of no special character and quite harmless, and they had ceased this practice after the edict in which, in accordance with your orders, I had forbidden all secret societies.
He even went to the trouble of torturing a couple of female church leaders, also with disappointing results:
I thought it the more necessary, therefore, to find out what truth there was in these statements by submitting two women, who were called deaconesses, to the torture, but I found nothing but a debased superstition carried to great lengths.
Nevertheless, he does note that his measures against the Christians have been successful in the improvement of society:
For the contagion of this superstition has spread not only through the free cities, but into the villages and the rural districts, and yet it seems to me that it can be checked and set right. It is beyond doubt that the temples, which have been almost deserted, are beginning again to be thronged with worshippers, that the sacred rites which have for a long time been allowed to lapse are now being renewed, and that the food for the sacrificial victims is once more finding a sale, whereas, up to recently, a buyer was hardly to be found. From this it is easy to infer what vast numbers of people might be reclaimed, if only they were given an opportunity of repentance.
Trajan’s reply is model of managerial wisdom and restraint. The Christians should be punished, but only if they refuse to recant.
You have adopted the proper course, my dear Pliny … The Christians are not to be hunted out; if they are brought before you and the offence is proved, they are to be punished, but with this reservation - that if any one denies that he is a Christian and makes it clear that he is not, by offering prayers to our deities, then he is to be pardoned because of his recantation, however suspicious his past conduct may have been.