In the past 12 years, the U.S. has spent more than $1.4 billion funding abstinence programs in Africa. They’re part of a larger program — called the President’s Emergency Plan for AIDS Relief — aimed at stopping the spread of HIV around the world.

Many health officials consider PEPFAR a succes. It is credited with giving lifesaving HIV drugs to more than 5 million people and preventing nearly 1 million babies from getting HIV from their mothers.

But a study, published Monday in Health Affairs, finds the abstinence programs have been a failure.

When President George W. Bush proposed PEPFAR in 2003, it was an unprecedented plan. The program would give billions of dollars to test and treat people for HIV in Africa. No one had ever given this much money to fight a single disease.

Congress funded the program with bipartisan support. But one part of the plan was controversial: A third of the money going toward HIV prevention was earmarked for programs teaching abstinence before marriage and faithfulness. This included sex education classes in schools and public health announcements on billboards and the radio.

Some critics worried the abstinence programs would use aid to impose American values on Africans, says John Dietrich, an assistant professor of political science at Bryant University.

The earmark was added to please some Republicans, Dietrich says, “who wanted to make sure the money wouldn’t be spent on anything that might be seen as promoting teenage sex or promiscuity.”

At the time, there was little evidence to suggest abstinence programs work. Randomized-control trials in the U.S. had shown that abstinence education programs didn’t prevent teenage pregnancies or decrease high-risk sexual behavior.

But there hadn’t been similar studies in Africa. And one program in Uganda — called Abstinence, Be Faithful and Condomize — was thought to have helped the country slow down the spread of HIV early on in the epidemic.

So Eran Bendavid, an infectious disease doctor, and his team at Stanford University wanted to see whether abstinence programs helped stop HIV in other African countries. They analyzed surveys given to nearly a half-million people in 22 countries. The surveys asked personal questions, such as “How many sexual partners do you have? At what age did you first have sex?”

Fourteen of the countries in the study had received funding from PEFPAR between 2004 and 2013. Eight of the countries hadn’t.

The results were clear: PEPFAR funding wasn’t associated with changes in young people’s choices about sex. Bendavid and his team could find no detectable differences in the rates of teenage pregnancies, average number of sexual partners and age at first sexual intercourse in countries that had received PEFPAR money compared with those that hadn’t.

It takes more than billboards or radio messages to change people’s behavior, Bendavid says. “I think the decisions about sexual behavior and preferences are much deeper,” he says. “They’re much more deeply rooted.”

Funding for abstinence programs peaked in 2008, when the U.S. spent about $250 million. That year, President Obama removed the requirement that PEPFAR fund abstinence programs. Since then, funding has steadily declined. In 2013, the U.S. government spent about $40 million a year on these programs in Africa.

We asked the head of PEPFAR if this new study would lead to an even bigger decrease. Ambassador Deborah Birx declined to be interviewed. In a statement, the agency said:

“Since its inception, PEPFAR’s mission has been to prevent as many new HIV infections and save as many lives as possible toward controlling the HIV/AIDS epidemic. To fulfill this mission, PEPFAR has continually evolved its approach to, and investments in, HIV prevention based on the latest scientific evidence and lessons learned from applying this evidence in programs. Current prevention science demonstrates that a combination package of evidence-based behavioral, biomedical, and structural prevention interventions, tailored to the populations and geographic areas with the greatest burden, is most effective in addressing the epidemic.”