The judges said there had been a "process violation" by the prosecution, but admitted their ruling was unusual.
Mr Cosby, 83, served more than two years of a three to 10-year sentence at a state prison near Philadelphia.
In 2018 he was found guilty of drugging and molesting ex-basketball player Andrea Constand.
Mr Cosby is best known for starring in the 1980s TV series The Cosby Show and was once known as "America's Dad".
Dozens of women have publicly accused Mr Cosby of sexual assault, but he was only tried criminally for the incident against Ms Constand. His conviction in 2018 was widely seen as a landmark moment in the #MeToo movement.
In a verdict issued on Wednesday, Pennsylvania's highest court found there was a "process violation" because Mr Cosby's lawyers had made an agreement with a previous state prosecutor that he would not be charged in the case.
The former actor appeared frail as he slowly walked to waiting media outside his home, shortly after being released from prison.
He did not say anything, instead leaning on his team of lawyers and spokesman Andrew Wyatt to answer questions.
"On this hot day - this is a hot verdict for us," Mr Wyatt said.
"Mr Cosby has always used his celebrity and his name to uplift women... How could a man who is being watched by the FBI every day be raping and drugging women… especially a black man?" he added.
Off camera, fans of Mr Cosby could be heard yelling their support throughout the media conference.
Mr Cosby was found guilty on three counts of felony indecent assault against Ms Constand.
Decades his junior, she met him in 2002 when working at Temple University in Philadelphia and described the comedian as a mentor figure. She later testified at trial how she became "frozen" after Cosby drugged and molested her at his home in 2004.
Ms Constand first came forward to police about the assault in 2005, but former state prosecutor Bruce Castor did not press criminal charges. She then sued the comedian for sexual battery and defamation, reaching a settlement with a confidentially agreement in 2006.
In 2014 and 2015, dozens of women came forward with similar allegations of drugging and assault by Mr Cosby. Local authorities knew that statute of limitation rules meant they could not pursue the majority of these accusations - but they reopened the case involving Ms Constand and eventually charged him just days before the 12-year limit on her allegations was set to expire.
A judge declared a mistrial in his first trial in 2017 after the jury failed to reach a verdict.
The testimony of other accusers was then allowed during a second trial, which helped prosecutors paint a pattern of predatory behaviour by Mr Cosby.
The judges' reasoning is complex - their written finding is 79-pages long.
They cite a written agreement reached by Mr Castor, the former prosecutor, stating that he would not charge Mr Cosby if he testified in Ms Constand's civil lawsuit.
In Wednesday's ruling, the Supreme Court judges also found that testimony from accusers unrelated to the case had tainted the trial.
"There is only one remedy that can completely restore Cosby to the status quo ante. He must be discharged, and any future prosecution on these particular charges must be barred", the finding released on Wednesday reads.
"We do not dispute that this remedy is both severe and rare. But it is warranted here."
A spokesman for Mr Cosby, Andrew Wyatt, thanked the court in a statement.
"This is the justice Mr Cosby has been fighting for. They saw the light," Mr Wyatt said. "He was given a deal and he had immunity. He should have never been charged."
Actor Phylicia Rashad, who played his wife on the Cosby Show, wrote on Twitter: "A terrible wrong is being righted - a miscarriage of justice is corrected!" in response to the news.
Prosecutor Kevin Steele said in a statement that Mr Cosby "was found guilty by a jury and now goes free on a procedural issue that is irrelevant to the facts of the crime".
"My hope is that this decision will not dampen the reporting of sexual assaults by victims… We still believe that no one is above the law - including those who are rich, famous and powerful."
Lawyer Gloria Allred, who represented more than 30 accusers of Mr Cosby, described the ruling as "devastating" but said the decision did not vindicate the comedian's conduct.
"My heart especially goes out to those who bravely testified in both of his criminal cases," her statement added.