Sherwin was born on September 27, 1842, in Baltimore, Vermont, and stayed in school until he was 15 years of age. His first job was in a local general store. In 1860, an uncle invited Henry to Cleveland, Ohio. He soon found employment with Freeman & Kellogg Co., a Cleveland dry goods store. Starting out as a clerk, two years later Sherwin had been promoted to bookkeeper. The young bookkeeper next moved to a wholesale grocery company by the name of Geo. Sprague & Co, where he became a partner, but soon left, apparently because he disliked the selling of liquor.
By 1866, Sherwin had saved $2,000. He was offered employment with a bank, a partnership with a wholesale drug company, and a partnership in a wholesale paint business. Choosing the later, Sherwin became the newest partner in the Truman Dunham & Co. He stated years later, that Truman Dunham offered the least but "was the one which in my youthful strength and ambition gave promise of a future greater than the others". He also admitted that at the time he "did not realize what it would lead to". By 1869, Sherwin discovered that his partners were more interested in the manufacture of linseed oil than in the production of paint.
In February 1870, they dissolved the existing partnership and Sherwin, with an associate by the name of Osborn (who had been bookkeeper at Dunham & Co.) and Edward Williams, a Civil War veteran who was a Phi Beta Kappa graduate of Western Reserve College, each invested $15,000 for equal shares in what became "Sherwin Williams & Co". Later that year they recruited and hired Sereno Peck Fenn as cashier and bookkeeper. The company prospered and in its first year reported sales of $422,390.97. In 2009, the company reported a decline in sales from the previous year, but ended up with $4.21 billion.
Sherwin was the President of Sherwin–Williams from 1870 to 1909, and chairman of the board until his death on June 26, 1916. He was buried in Lakeview Cemetery, Cleveland, Ohio.