The African Union thrives or fails according to how successful it is in preventing and resolving conflict. Over the past 14 years, since its foundational meeting in Durban, South Africa, the AU has constructed an impressive peace and security architecture. It has presided over an impressive decline in armed conflicts and military coups.

Last year, the commissioner for peace and security asked us to review peace efforts in Africa with a view to informing AU policies. Our report is published on Thursday. The next six months, as the AU chooses a new chair for its commission, are a chance to examine the state of peace and security in Africa – and, more important, the direction in which the AU can steer the continent.

The AU has built a solid foundation and adopted far-reaching principles. Africa routinely rejects unconstitutional changes in government, upholds the principle of “non-indifference” – meaning countries are required to intervene when mass atrocities are committed against civilians – and supports more inclusive peace processes. The AU has set up the peace and security council, high-level panels for conflict resolution, and the African standby force for supporting peace operations.

The AU’s successes have come about through political and diplomatic approaches. While the AU also mounts military operations, these are expensive and, while sometimes necessary, aren’t consistently successful.

Worryingly, Africa’s peace and security record has shifted in the wrong direction over the past five years. Old patterns of conflict and authoritarianism have recurred and new threats have emerged.

The biggest task for the new commission chair is how to put Africa back on its track towards fewer armed conflicts. Experience shows that this is best done through prevention and mediation.

Contested regime transitions – and particularly attempts by those in power to hang on despite constitutional term limits – are a major driver of armed conflict and popular protest. While there is nothing sacred about a two-term limit for presidents, the principles of constitutional democracy are a core African value and need to be protected.

Inter-state rivalries are another important and under-acknowledged trigger for conflict. The AU has devoted much attention to resolving them and creating a common approach among “frontline states”. This is never simple, and one challenge is getting the right division of labour between the AU and regional economic communities, which have the advantage of being nearby, but are not always impartial.

Significant too is violent extremism and the spillover of crises from the Arab world. The Sahara-Sahel region, and increasingly west Africa and the Horn of Africa, are affected by the turmoil in the Middle East, including both the spread of militant extremism and rivalries among Middle East nations. Africa needs a new mechanism for political partnership in the shared spaces of north Africa, the Mediterranean and the Red Sea, so that African voices are heard and the agenda is not set entirely by others.

The AU has massively expanded its peace support operations, which now rival the UN’s in size and ambition. Yet the AU’s peacekeeping capabilities need to be fit for purpose. We propose two different roles for the African standby force: one should be a light footprint deployment aimed at preventing conflict, and the other a fully combat-capable peace enforcement operation. These demand very different command and control systems – the first could be run by the AU commission, but countries that contribute troops will insist on keeping their national command systems.

At a recent summit, the AU’s high representative for the African peace fund, Donald Kaberuka, presented his report. It proposes that if African countries can cover 25% of the cost of AU peace operations, the UN will cover the remaining 75% from its member states’ regular contributions for peacekeeping. That’s an important proposal: AU peace operations must be put on a firmer financial footing. The AU needs to assure that African states pay their portion of the bill as there can be no political ownership without financial responsibility. But it would be futile to spend money on military missions while shortchanging the AU’s political instruments – prevention and mediation should be the top funding priorities.

This is a tough agenda and demands a vigorous commission chair, someone prepared to use the AU’s tools to their limits and bend them to grapple with new challenges. The AU has six months of transition to its new leadership: it should use this time to set its strategic priorities for peace and security, and ensure that its next chair can deliver.

• Mulugeta Gebrehiwot is director of the African peace missions programme at the World Peace Foundation and Alex de Waal is the foundation’s executive director