Bold promises to empower women economically will come to nothing if the structural barriers that prevent women from getting decent jobs are not removed, civil society groups have warned.

A high-level panel backed by the World Bank, UN Women, the International Monetary Fund and the International Labour Organisation (ILO) is producing an action plan to improve women’s economic opportunities over the next 15 years.

The panel, launched during the World Economic Forum in January, held its first consultative meeting at the UN on Wednesday during the annual Commission on the Status of Women. It includes business leaders, politicians, academics and civil society, and is chaired by Luis Guillermo Solis, the president of Costa Rica, and Simona Scarpaleggia, CEO of Ikea Switzerland.

Campaigners said they were disappointed not to have heard more from panellists about women’s unpaid work, exploitative contracts that keep women poor, and dangerous working conditions

Chiara Capraro, gender policy adviser at Christian Aid, said empowering women is not just about getting more female CEOs.

“The ILO estimates that, by 2020, 80% of people across the world will be affected by austerity policies. When services are cut, women have to pick up the slack and that affects their ability to make a living and access decent work,” she said.

Many women work in special economic zones, or for companies that don’t pay tax, or a living wage, and are denied access to unions, she added. “It’s really essential to see these dimensions reflected [in the panel],” she said.

Rachel Noble, women’s rights policy adviser at ActionAid, said: “It’s absolutely critical they pay attention to the fundamental structural issues blocking women from participating in the economy.

“There’s not nearly enough recognition of the fact that this is about women’s human rights, not just about creating economic growth.

“The biggest role companies can play is ensuring women have access to decent work, paying a living wage, giving them secure contracts, and ensuring their right to form unions and engage in collective bargaining.”

Ann Kargbo, head of Rehabilitation and Development Agency, a Sierra Leonean NGO working on gender equality in Bo county, added that women’s economic empowerment goes hand in hand with gender equality and will only be fully realised when women can take control of what they earn.

“We need to know who owns the resources, who controls the resources and who makes decisions on the utilisation of resources.”

Allison Julien, from the National Domestic Workers Alliance, told the panel that the right to organise “has a profound impact on women’s ability to move from poverty and negotiate for a living wage and have access to social protection”.

“Unpaid care is the major structural barrier for women’s economic empowerment. Domestic work makes all other work possible. Without someone taking care of your home and children, people wouldn’t be able to take their jobs.”

According to the McKinsey Global Institute, if women in every country were to play an equal role to men in labour markets, up to $28tn could be added to the global annual GDP by 2025.

But women spend more than twice as much time as men doing unpaid care and domestic work, while earning on average 24% less than men for doing the same work. Women in poorer areas are more likely to work in informal unprotected jobs.

The high-level panel will continue its consultations over the coming months before publishing the first of two reports in September. The report promises to include practical ways to improve women’s lives and “promote women’s leadership in driving economic growth”.

“I assure you, this panel is going to be bold, aim high and keep it focused,” said Solis. “We can’t afford to wait. It’s clear we need to move forward very fast and do everything that needs to be done to give more autonomy to women. We have to inspire.”

Justine Greening, the UK’s international development secretary, who sits on the panel but was not in New York for Wednesday’s meeting, said in a statement that the panel marked “a major step forward in the battle to close the gender gap in economies around the world. We now need to work hard to make sure it delivers real change for the world’s poorest women.”