The papacy as an institution evolved over time, contrary to what Roman Catholics believe. Historical facts can really be a pain in the ass for Roman apologists, who would love to think that there was a pope in the 1st century or 2nd century AD. That there were no popes involved at Nicea in 325AD should have given them pause. The first real pope, as an office that began to stand out above the other bishops, was Pope Gregory the Great, in the 6th century AD. In one of this letters to the Eastern bishops, he wrote:
For, as your venerable Holiness knows, this name of Universality was offered by the holy synod of Chalcedon to the pontiff of the Apostolic See which by the providence of God I serve. But no one of my predecessors has ever consented to use this so profane a title; since, forsooth, if one Patriarch is called Universal, the name of Patriarch in the case of the rest is derogated. But far be this, far be it from the mind of a Christian, that any one should wish to seize for himself that whereby he might seem in the least degree to lessen the honor of his brethren. While, then, we are unwilling to receive this honor when offered to us, think how disgraceful it is for any one to have wished to usurp it to himself perforce.
Wherefore let not your Holiness in your epistles ever call any one Universal, lest you detract from the honor due to yourself in offering to another what is not due. (Book V, EPISTLE XLIII: TO EULOGIUS AND ANASTASIUS, BISHOPS; Source)
The Bishop of Rome slowly evolved over time into the Pope, and even at the time of Gregory the Great, the pope was still considered primus inter pares (first among equals), not yet superior as the universal bishop, with the title Pontifex Maximus. Of course, the papacy slowly grew in power until Vatican I where he usurps for himself infallibility when he decides to define dogma ex cathedra.