The history of this church dates back to 1702 when the Huguenot, former galley slave, and prisoner Elias Neau petitioned England for missionaries to instruct "Negroes and Indians in New York.” In 1704 the Society for the propagation of the Gospel appointed Neau "catechist to the outcast" and he was charged with opening the "School for Negroes."
Many non-Africans believed that Africans had no souls and perished as beasts. All worship was segregated, and most religious establishments saw no need to educate the Africans.
By the early 1800s, the "Free African Church of St. Philip" was formed by those who refused to continue to accept the racism they faced in worship but who wished to worship in the "Anglican Way."
n 1809, St. Philip's Church was organized and led by Peter Williams, Jr., an abolitionist who had been the first black deacon in the Diocese of New York. The church was originally located on Centre Street, but that building was destroyed by fire in 1822, then rebuilt in brick. In 1826, Peter Williams, Jr., was ordained to the priesthood and became the first rector of St. Philip's Church. He remained in the position until his death in 1840.
After Williams' death, the congregation's spiritual needs were tended by priests-in-charge, including Samuel Vreeland Berry and John Peterson. Fr. Berry grew up in St. Philip's and was ordained in 1849. Fr. Peterson was a teacher by profession, a life-long Episcopalian and served at St. Philip's as a vestryman and warden. He was ordained in 1854 and became an assistant minister. It was not until 1872 that the second rector was called.