APRA fails its super performance test

One of the most challenging problems plaguing superannuation choice is how to evaluate historical performance outcomes against a fund’s stated objectives, which has been cast into relief by the regulator’s inaugural return test for 80 MySuper products.

Unfortunately, a new report by actuarial researcher Chant West finds that the Australian Prudential Regulation Authority’s approach “fails to measure the most important thing, which is whether the fund is pursuing an investment strategy (ie, asset allocation) that’s appropriate for its members”.

This column has previously argued that ranking super funds based on raw returns gives members no insights on the risks that were assumed to achieve those results. And yet this is how super fund league tables are normally compiled with zero reference to the variability or volatility of the returns and/or the maximum draw-downs that were incurred.

Given the spectre of consumer choice, these rankings encourage funds to best peers by maximising risk and return irrespective of their targets, which possibly explains the heavy equity-bias in Aussie pension fund portfolios vis-à-vis the rest of the world.

And yet super funds have very explicit return objectives, which are normally expressed as a margin above inflation (eg, CPI plus 3 per cent). A super fund’s theoretical objective should be to maximise the probability of delivering returns at or above the target while minimising the risk of loss. (Sometimes these targets are further calibrated based on the age-profile of the saver.)

With this in mind, Chant West General Manager Ian Fryer says APRA’s methodology for assessing performance is flawed and likely to result in situations where “funds can deliver great outcomes for their members but still fail the test…[while] other funds may deliver inferior outcomes, compared with some of those that fail the test, but still pass”. He notes that failing the test has serious consequences, including significant outflows that could force the super fund to change its investment strategy to mitigate liquidity risks in a way that hurts the remaining members.

APRA’s test judges a fund’s past 7-year performance by comparing the returns of each of its individual asset-classes to a benchmark selected by APRA. If the fund has underperformed the combined benchmarks by more than 0.5 per cent annually (after adjusting for taxes and fees), it is deemed by APRA to have “failed”. It must then write to its members advising them of this failure, and encourage them to look for another fund by consulting the ATO’s YourSuper comparison tool.

The first shortcoming is that the test tells us absolutely nothing about whether the super fund has met or exceeded its stated return objectives and delivered on its obligations to members. Success is defined by whether the fund beats the asset-class benchmarks selected by APRA rather than the goals in the fund's product disclosure statements. You could, therefore, have a scenario where a fund fails its own CPI plus x per cent objective but does brilliantly on APRA’s test simply because it outperformed the asset-classes it chose.

This brings us to a second problem: asset-allocation. The test tells members nothing about the asset-allocation decisions of funds (eg, the cash, bonds, equities, property splits) despite the fact that most researchers and chief investment officers would claim that this is their single-biggest source of value-add.

Imagine a situation where over the next 7 years the world struggles with high inflation, rising interest rates, and a massive downward rerating of growth asset-classes. A conservative super fund might save members huge amounts of money, and substantially outperform its peers, by going materially overweight cash, zero interest rate duration bonds, and liquid defensive alternatives. Within each asset-class, this fund might have also invested in lower-risk exposures that underperform the overall asset-class benchmark, but again with much lower probabilities of loss.

The net result could be a fund that beats its performance goals, smashes rivals, and which demonstrates tremendous asset-allocation skill, and yet it still fails APRA’s test simply because it has chosen defensive solutions within each asset-class. “By taking those crucial asset allocation decisions out of the picture and simply focusing on implementation, APRA’s test ignores the most potent source of added value,” Fryer says.

Here Chant West highlights the choice of fixed-income benchmark. Whereas there is hundreds of billions of dollars of zero duration, floating-rate fixed-income outstanding in Australia, APRA has selected a benchmark—the AusBond Composite Bond Index—that only comprises fixed-rate bonds. Aside from overlooking a big part of the bond market, it also forces super funds to take huge amounts of interest rate risk at arguably the worst point possible.

“The benchmark indices that APRA uses for fixed interest have no ‘risk awareness’,” Chant West’s Fryer says. “So they’re longer duration than most funds’ actual exposure,” he continues. “The 7-year return to June 2021 for short duration bonds was 2.4 per cent annually compared with 4.1 per cent for the broader bond market. This discrepancy has hurt a lot of funds in the 2021 MySuper performance test and, unless yields rise significantly, will hurt many more funds’ conservative options in the 2022 choice performance test.”

Ironically, the tables have turned over the last 12 months: whereas the fixed-rate Composite Bond Index has lost 5.3 per cent, the AusBond Floating-rate Note Index outperformed by 5.8 per cent with a 0.5 per cent return.

“We applauded APRA and Treasury for changing the benchmarks earlier this year so they now cater better for unlisted assets,” Fryer says. “An urgent review is needed now to achieve a similar improvement in the treatment of defensive assets.”

Fryer cites other examples, including “defensive alternatives, which some funds turned to as more attractive than conventional cash and bonds, yet were classified by APRA as 50 per cent growth, which was an unattainable and inappropriate benchmark target for these assets”.

Compounding these flaws, Chant West says requiring failing super funds to suggest members use the ATO’s YourSuper comparison tool to select a new solution “has serious shortcomings”. Their first concern is that “the tool shows funds’ raw performance only with no indication as to the level of risk”. “It also shows a range of returns for lifecycle products, with no indication of where in that range the enquiring member would sit,” Fryer says. Another worry is that it has out-of-date investment fees for some funds, which “impacts the accuracy of the comparison”.

Chant West further argue that the use of the last 7-years, and only the last 7-years, to decide the fate of super funds is biased because of the unique low-rates-for-long paradigm that has favoured equities and long duration bonds. “Ironically the current test, which looks back over a period of strong markets, has rewarded risk and penalised some funds that have attempted to moderate risk for their members,” Fryer says.

“Whatever the explanations, as researchers we believe there were funds that failed the test that deserved to pass and, conversely, funds that passed the test that probably deserved to fail.”

What are the solutions? Chant West believes the test needs much more granularity in the choice of benchmarks, such as allowing super funds to choose between floating-rate and fixed-rate debt in the same way they distinguish between Aussie equities and global equities. In this vein, the choice of benchmarks for composite multi-asset-class products and lower risk defensive alternatives could be risk-adjusted to provide for more accurate performance evaluation.

Rather than de facto killing funds that fail a flawed test, which they might have passed had they be properly assessed, Chant West proposes APRA adopt a “strike” system. “It would be far preferable, in our view, for APRA to engage in a rigorous review process with those funds that fail the test once – a ‘first strike’ – and only to name them once it becomes clear that their shortcomings are unlikely to be rectified, and only when there’s a clear plan in place for the fund to exit the system in an orderly manner,” Fryer says.

This is similar to the approach APRA takes when stress-testing banks. Unlike other regulators that publish the individual bank stress-test results, APRA does not disclose this information. It works privately with banks to address their capitalisation concerns precisely because it wants to avoid publicly destabilising these institutions.

In the case of super funds, APRA should still publish the results, but not immediately judge them to be a pass or fail. Only once it has exhaustively interrogated the findings with a full review, should it then publish a decision, which would ideally be accompanied by a full remediation plan agreed with the fund in question.