1. Introduction

With the first hedge fund said to be dating back to 1949, hedge funds have been around for quite some time. Academic research into hedge funds, however, only took off towards the end of the 1990s when sufficient data became available. Since then, and inspired by the strong growth of the hedge fund industry worldwide, a respectable number of research papers and articles have provided insight in many different aspects of hedge funds. One question largely remains unanswered though. Do hedge funds provide their investors with superior returns? In other words, do hedge funds provide their investors with returns, which they could not have obtained otherwise?

According to the hedge fund industry itself, the answer to the above question is of course affirmative, although with the recent disappointing performance of hedge funds, this point is put forward less often and less forcefully than it used to. Nowadays, most emphasis is on the diversification properties of hedge funds. Various academic studies have attempted to shed light on the issue of hedge fund return superiority as well. Most of these apply traditional performance measures, such as the Sharpe ratio or factor model based alphas, to hedge fund returns obtained from one or more of the main hedge fund databases. The conclusion is typically that hedge fund returns are indeed superior. From other studies, however, it is now well understood that raw hedge fund return data may suffer from various biases, which, when not corrected for, will produce artificially high Sharpe ratios and alphas. In addition, hedge fund returns are typically not normally distributed and may derive from exposure to very unusual risk factors. This makes traditional performance measures unsuitable for hedge funds, as deviations from normality as well as every risk factor that is incorrectly specified or left out altogether, will tend to show up as alpha, thereby suggesting superior performance where there actually may be none.

In theory, once the relevant risk factors have been identified, factor model based performance evaluation of hedge fund returns should work well. In practice, however, we don't know enough about hedge fund return generation to be certain that all the relevant risk factors are included and correctly specified. As a result, factor models typically explain only 25-30% of the variation in individual hedge fund returns, which compares very unfavourably with the 90-95% that is typical for mutual funds.

Although the procedure works better for portfolios of hedge funds, funds of funds and hedge fund indices, where most of the idiosyncratic risk is diversified away, the low determination coefficients of these models make it impossible to arrive at a firm conclusion with respect to the superiority of hedge fund returns.

It is quite surprising that so many people, on the buy-side as well as in academia, are so eager to believe that the, sometimes huge, alphas reported for hedge funds are truly there. Anyone who is well calibrated to the world we live in and the global capital markets in particular, knows how difficult it is to consistently beat the market, ie, systematically obtain a better return than what would be fair given the risks taken. Over time, hundreds, if not thousands, of studies have confirmed this. Is it therefore likely that suddenly we are facing a whole new breed of super-managers; not one or two, but literally thousands of them? Of course not! And if anything, the rise of the hedge fund industry has made markets more efficient, not less.

Although by far the most popular, factor models are not the only way to evaluate hedge fund performance. Based on previous work by Amin and Kat (2003), Kat and Palaro (2005), or KP for short, recently developed a technique that allows the derivation of dynamic trading strategies, trading cash, stocks, bonds, etc, which generate returns with predefined statistical properties. The technique is not only capable of replicating (the statistical properties of) fund of funds returns, but works equally well for individual hedge fund returns. Since the KP replicating strategies are explicitly constructed to replicate the complete risk and dependence profile of a fund, the average return on these strategies can be used as a performance measure. When the average fund return is significantly higher than the average return on the replication strategy, the fund is the most efficient alternative and vice versa.

The KP replication technique is similar to that used in Amin and Kat (2003). The important difference, however, is that the latter only replicate the marginal distribution of the fund return, while KP also replicate its dependence structure with an investor's existing portfolio. This is a very significant step forward as most investors nowadays are attracted to hedge funds because of their relatively weak relationship with traditional asset classes, ie, their diversification potential. Only replicating the marginal distribution without giving any consideration to the dependence structure between the fund and the investor's existing portfolio would therefore be insufficient.

From a performance evaluation perspective, replication of a fund's dependence pattern with other asset classes is a necessity. According to theory as well as casual empirical observation, expected return and systematic co-variance, co-skewness and co-kurtosis are directly related. In other words, it is not so much the marginal distribution, but its dependence structure with other assets that determines an asset's expected return. An asset, which is highly correlated with stocks and bonds, offers investors very little in terms of diversification potential. As a consequence, there will be little demand for this asset. Its price will be low and its expected return therefore relatively high. On the other hand, an asset that offers substantial diversification potential will be in high demand. Its price will be high and its expected return relatively low. Although hedge funds are not priced by market forces in the same way as primitive assets are, they do operate in the latter markets. It therefore seems plausible that a similar phenomenon is present in hedge fund returns as well .

2. The KP Efficiency Measure

Applying the KP replication technique to hedge funds, the goal is to create a dynamic trading strategy, which generates returns with the same statistical properties as a given hedge fund or fund of funds, ie, returns that are drawings from the same distribution as the distribution from which the actual fund returns are drawn. The basic idea behind the procedure is straightforward. From the theory of dynamic trading it is well known that in the standard theoretical model with complete markets any payoff function can be hedged perfectly. This observation forms the foundation of arbitrage-based option pricing theory. If it is possible to find a payoff function which, given the distribution of the underlying assets, implies the same distribution as the one from which the fund returns are drawn, then the accompanying dynamic trading strategy will generate (returns that are drawings from) that distribution.

1. Monthly return data are collected on the fund to be evaluated, the representative investor's portfolio, and a so-called reserve asset. The latter is the main source of uncertainty in the replication strategy. As we want to know whether the returns that investors obtain from hedge funds are superior, fund returns should be net of all fees.
   
   
2. From the available return data, the bivariate distribution of the fund return and the representative investor's portfolio return is inferred (KP refer to this as the 'desired distribution'). The same is done for the bivariate distribution of the investor's portfolio return and the return on the reserve asset (the 'building block distribution'). In line with KP, we allow for 54 different joint distributions, choosing between them using the Akaike Information Criterion (AIC) .
   
   
3. Assuming an initial investment in the fund of 100, we determine the cheapest payoff function, which is able to turn the building block distribution into the desired distribution. This payoff function is known as the 'desired payoff function' and lies at the basis of the KP replication strategies.
   
   
4. The desired payoff function is priced using the multivariate option pricing model of Boyle and Lin (1997), which explicitly allows for transaction costs. For the pricing of the payoff function, we estimate the required volatility and correlation inputs over the period covered by the track record of the fund being evaluated. We use the average 1-month interest rate over the same period for the interest rate input. We will refer to the price thus obtained as 'the KP efficiency measure'.
   
   
5. Finally, we compare the KP efficiency measure with the 100 initially invested in the fund. If the efficiency measure is 100 as well, then the replication strategy and the fund are equivalent. If the efficiency measure is less (more) than 100, the strategy is cheaper (more expensive) than the fund and the fund therefore inefficient (efficient).

In the evaluations, we do not use hedge funds' raw returns. The reason is that, as shown in Brooks and Kat (2002) and Lo et al. (2004) for example, monthly hedge fund returns may exhibit high levels of autocorrelation. This primarily results from the fact that many hedge funds invest in illiquid securities, which are hard to mark to market. When confronted with this problem, hedge fund administrators will either use the last reported transaction price or a conservative estimate of the current market price. This creates artificial lags in the evolution of hedge funds' net asset values, ie, artificial smoothing of the reported returns. As a result, estimates of volatility, for example, will be biased downwards.

One possible method to correct for this bias is found in the real estate finance literature. Due to smoothing in appraisals and infrequent valuations of properties, the returns of direct property investment indices suffer from similar problems as hedge fund returns. The approach employed in this literature has been to "unsmooth" the observed returns to create a new set of returns which are more volatile and whose characteristics are believed to more accurately capture the characteristics of the underlying property values. Nowadays, there are several unsmoothing methodologies available. In this study we use the method originally proposed by Geltner (1991).


3. An Example

To clarify the above, let's look at a worked-out example. XYZ is a well-known fund of hedge funds, which started in 1985. Given XYZ's monthly, net-of-fee returns since 1985, the first step is to model the joint distribution of XYZ and the investor's portfolio, as well as the joint distribution of the investor's portfolio and the reserve asset. Before we can do so we need to decide what exactly the investor's portfolio and the reserve asset are, as well as unsmooth the raw fund return data.

Let's assume that the representative investor's portfolio consists of 50% S&P 500 and 50% long-dated US Treasury bonds. Let's also assume that all exposure management is done in the futures markets. So instead of investing in the cash market, we will hold fully collateralised (nearby) futures contracts. We use nearby Eurodollar futures as the reserve asset. Futures have several advantages over cash, in particular high liquidity and low transaction costs, which is extremely important given the dynamic nature of the KP replication strategies.

Table 1 shows the marginal risk characteristics of the raw and unsmoothed XYZ returns. From the table, we see that XYZ's raw returns exhibit negative skewness and positive autocorrelation. Application of the unsmoothing procedure eliminates the autocorrelation and produces returns with the same degree of skewness, but with a substantially higher volatility (annualised 14.7% vs 12.8% for the raw returns).

We are now ready to infer the desired and the building block distribution. Using the same methodology as KP, we find that the best fit (according to the AIC) is provided by the following set of marginals and copulas :

XYZ: Student-t (µ = 0.0101, s = 0.0406, df = 4.0544)
Portfolio: Normal (µ = 0.0101, s = 0.0282)
Reserve: Johnson (? = 0.0031, ? = 0.0046, ? = -0.60, d = 1.599)
Copula (XYZ, portfolio): Normal (? = 0.754)
Copula (portfolio, reserve): Gumbel (a = 1.3349)

Given the above distributions, we can derive the desired payoff function following the methodology developed in KP. The result is depicted in Figure 1 and shows that the desired payoff is an increasing function of both the investor's portfolio and the reserve asset, implying that the replication strategy will take long positions in both assets. Subsequently, we price this payoff function using the Boyle and Lin (1997) model, assuming transaction costs in the futures markets are 1bp one-way. This produces a value for the KP efficiency measure of 99.53, meaning that, seen over the whole life of the fund, XYZ's returns are not as miraculous as many investors may have thought. Trading S&P 500, T-bond and Eurodollar futures, investors could have generated the same risk profile as XYZ and obtained a higher average return at the same time.

Figure 1: Desired Payoff Function for Replication XYZ Returns


Figure 2: Scatter Plot Investor's Portfolio Returns vs XYZ Returns (left) and
Replicated Returns (right)

To see how well the derived payoff function succeeds in replicating the desired distribution, Figure 2 shows a scatter plot of the investor's portfolio return versus the XYZ return (left) as well as a plot of the portfolio return versus the replicated return (right). The two plots are very similar, suggesting that the replication has indeed been successful. We see that the replication strategy is unable to replicate the three large losses that XYZ reported during the sample period. This is not surprising as these are clearly outliers, which simply cannot be captured by a parametric model like ours.

A further indication of the accuracy of the replication strategy comes from comparing the mean, standard deviation, skewness and kurtosis of XYZ's returns with those of the replicated returns. The latter statistics can be found in Table 2, together with the correlation and Kendall's Tau with the investor's portfolio. Since the XYZ returns exhibit a few negative outliers, apart from the standard skewness and kurtosis measures, we also report a more robust skewness and kurtosis measure . To test whether the marginal distribution of the replicated returns and the joint distribution of the replicated returns and the investor's portfolio are significantly different from the original distributions, we use the univariate and bivariate Kolmogorov-Smirnov (K-S) tests .

Comparing the entries in Table 2, it is clear that the statistical properties of XYZ's returns have been quite successfully replicated. The replication strategy has not only replicated the marginal distribution of XYZ's returns but also its relationship with the investor's portfolio. The same conclusion follows from both the K-S tests.

4. Distributional Analysis

A crucial stage in the evaluation procedure is the proper modelling of the distributional characteristics of the fund, the investor's portfolio and the reserve asset. This means that, although not explicitly designed to do so, the evaluations provide a wealth of information on the distributional properties of fund of funds returns. Table 3 summarises how often (out of a total of 485 funds) a given marginal or copula was used in the evaluations for modelling the fund return marginal and the joint distribution of the fund and the investor's portfolio return.

Table 3 confirms that, despite an often substantial degree of diversification in the larger funds, the majority of fund of funds returns are far from normally distributed. Out of 485 funds, 340 funds' marginal return is better modelled by a Student-t or Johnson distribution than a normal distribution. In addition, for only 88 of the 485 funds is the relationship with the investor's portfolio (consisting of 50% S&P 500 and 50% T-bonds) best modelled by the normal copula. This emphasises once more how important it is to evaluate hedge fund and fund of funds performance using a method, which does not rely on the assumption of normally distributed returns.

5. Evaluation Results

Having introduced the evaluation procedure, we now present the evaluation results. Our total sample consists of 485 funds of funds with a minimum of four years of history available. All data were obtained from TASS as of November 2004. Note that this implies that the again disappointing results of 2005 were not taken into account in the evaluation . Funds denominated in another currency than USD were converted to USD, ie, the perspective taken is that of a USD-based investor. Table 4 provides some information on the starting and end dates of the track records of the funds in our sample.

Table 4 shows that, reflecting the increasing popularity of hedge funds in the second half of the 1990s, the majority of funds started after 1994. Most hedge fund databases, including TASS, first started collecting data around 1994. As a result, our sample contains no funds that stopped reporting before that date. We also see that out of 485, 208 funds stopped reporting before October 2004. This confirms that, although lower than for individual hedge funds, the attrition rate in funds of funds is still quite high.

Table 5 provides details on the length of the available fund of funds track records. Out of the 485 funds in the sample, only 118 have more than ten years of history. This again reflects the fact that most funds of funds are still relatively young and attrition can be significant.

As in the example in section 3, in the evaluations we assume that the representative investor's portfolio consists of 50% S&P 500 and 50% long-dated US Treasury bonds, with all exposure management done through fully collateralised (nearby) futures contracts . Since it is one of the most traded futures contracts in the world, we use nearby Eurodollar futures (trading on the CME) as the reserve asset. Transaction costs on all futures contracts are assumed to be 1bp one-way. For the pricing of the payoff functions, we use 1-month USD Libor as the relevant interest rate, while estimating the required volatilities and correlations over the period covered by the track record of the fund that is being evaluated. The interest rate data was obtained from Datastream, while the futures data was obtained from Commodity Systems Inc (CSI).

Figure 3: Scatter Plot Fund vs Replicated Standard Deviation

Figure 4: Scatter Plot Fund vs Replicated Skewness

Figure 5: Scatter Plot Fund vs Replicated Correlation with Investor's Portfolio

To get an idea of the typical accuracy of the replication procedure, Figure 3-5 show scatter plots of the fund standard deviation (Fig. 3), standard skewness (Fig. 4) and correlation with the investor's portfolio (Fig. 5) versus the replicated values for all 485 funds. As is clear from these graphs, on average the replication of these parameters is unbiased and quite accurate. Not surprisingly, the replication of skewness can be difficult at times as fund returns may contain one or more outliers, which will have a major impact on the standard skewness statistic, but which cannot be replicated. We encountered this problem before in the example in section 3.

Figures 3-5 also provide additional information on the risk-return profile of the funds in our sample. From Figure 4 for example, we see that for most funds of funds estimated skewness lies somewhere between -1 and +1. Likewise, from Figure 5 we see that the majority of funds are positively correlated a portfolio of 50% stocks and 50% bonds. Most correlation coefficients lie between 0 and 0.6, indicating that many funds of funds' returns are a lot less 'market neutral' than the term 'absolute returns'suggests .

Figure 6: Histogram KP Efficiency Measure 485 Funds of Funds

Figure 6 shows a histogram of the values of the KP efficiency measure obtained for the 485 funds of funds in our sample. From the graph we see that the majority of funds produce a value for the efficiency measure that is below 100. The average value for the KP efficiency measure over all 485 funds is 99.36. We tested the statistical significance of the above efficiency measure results by calculating bootstrapped confidence intervals. We distinguished between three cases and obtained the following results:

• The confidence interval is entirely lower than 100 - 389 funds
• The confidence interval contains 100 - 41 funds
• The confidence interval is entirely higher than 100 - 55 funds

This confirms the results in the histogram in Figure 6. The majority of funds of hedge funds have not provided their investors with returns, which they could not have generated themselves in the futures market.

We sorted the funds in our sample in various ways depending on starting date and/or end date to see if there was any indication of old funds doing better than new funds, live funds doing better than funds that closed down, etc. However, none of these subdivisions produced significant results.

Figure 7: Percentage of Funds that Stopped Reporting Before October 2004
as Function KP Efficiency Measure

The histogram in Figure 6 is negatively skewed, implying that some of the funds in our sample have shown extremely bad performance relative to what could have been achieved trading S&P 500, T-bond and Eurodollar futures. Since lack of performance is one of the main reasons for funds to close down, Figure 7 shows the percentage of funds that stopped reporting to the database before October 2004, as a function of their KP measure. From the graph we see that there is a strong relationship. Out of the 93 funds with a KP measure below 99, no less than 71 (76%) stopped reporting. Out of the 70 funds with a KP measure higher than 100, only 16 (23%) did so. A similar relationship is observed in the average KP measures of live and dead funds. The average KP measure over the 208 dead funds is 98.97, while over the 277 funds still alive the average is 99.65.

6. Conclusion

In this paper we have used the hedge fund return replication technique recently introduced in Kat and Palaro (2005) to evaluate the net-of-fee performance of 485 funds of hedge funds. The results indicate that the majority of funds of funds have not provided their investors with returns, which they could not have generated themselves by trading S&P 500, T-bond and Eurodollar futures. Purely in terms of returns therefore, most funds of funds have failed to add value.

Compared with the various hedge fund performance evaluation studies that have been carried out over the last couple of years, our results are quite unusual. Often, the conclusion from hedge fund performance studies is that funds of funds generate superior returns, not inferior. This emphasises how tricky factor model based performance evaluation really is. As long as one can't be sure that all relevant risk factors are accounted for, it is impossible to know whether unexplained returns are indeed true alpha or just unexplained because one or more risk factors were left out or specified incorrectly. Our methodology is a lot more robust, as it relies on only one simple principle: "if it can be replicated, it can't be superior". Of course, we need to make assumptions as well, but these are a lot less crucial for the final outcome of the evaluation than the kind of assumptions required to make factor model based alphas work.

Should investors rush out to buy into the funds with the highest KP measures? Although tempting, the answer is no. By definition, extreme events only occur infrequently. With more than 75% of the funds in our sample having a track record of less than 120 months, it is therefore hard to identify the presence of any catastrophic (but compensated) risks from the available data. A fund may be taking the most terrible risks, but if so far it has been lucky, the premium collected for taking on those risks will show from the fund's track record, but the risk itself won't. A high KP measure should therefore first and foremost be interpreted as an indication that further due diligence is required. One can only speak of truly superior performance if such follow-up research shows that the manager in question has generated the observed excess return without taking any extreme risks.

Finally, it has to be noted that although in terms of the returns delivered to investors, funds of funds do not seem to add value, this does not mean there is no economic reason for funds of funds to exist. Most private and smaller institutional investors do not have the skills and/or resources required to perform the necessary due diligence that comes with hedge fund investment. In addition, given typical minimum investment requirements, small private investors will often lack sufficient funds to build up a well-diversified hedge fund portfolio. They therefore have no choice. If they want hedge funds, they will have to go through a fund of funds.

Large institutions do have a choice. Most of them, however, still prefer to go the fund of funds route. This is quite surprising given the amount of fees that could be saved by skipping the middlemen. Apart from believing that fund of funds managers add enough value to justify their fees (which research has shown to be unlikely), part of the reason that many large institutions go for funds of funds lies in the fact that the interests of institutional asset managers are typically not correctly lined up with the interests of those whose money they manage. As a result, job protection becomes an important consideration. By investing in a fund of funds, instead of picking hedge funds themselves, institutions avoid having to take responsibility for the bottom-line fund selection. In the end, all they can be held responsible for is the decision to invest in hedge funds and the selection of the fund of funds that they invested in; risks, which can easily be hedged by not making a move until others do, hiring a big name consultant and a big name fund manager, as most institutions do.

The authors like to thank CSI and Rudi Cabral for allowing them to access the CSI futures database.

References

Akaike, H., Information Theory and an Extension of the Maximum Likelihood Principle, in: B. Petrov and F. Csaki (eds.), Second International Symposium on Information Theory, Academiae Kiado, Budapest, 1973, pp. 267-281.

Amin, G. and H. Kat (2003). Hedge Fund Performance 1990-2000: Do the Money Machines Really Add Value? Journal of Financial and Quantitative Analysis, Vol. 38, pp. 1-24.

Boyle, P. and X. Lin (1997). Valuation of Options on Several Risky Assets When There are Transaction Costs, in: P. Boyle, G. Pennacchi and P. Ritchken (eds.), Advances in Futures and Options Research, Vol. 9, Jai Press, pp. 111-127.

Brooks, C. and H. Kat (2002). The Statistical Properties of Hedge Fund Index Returns and Their Implications for Investors. Journal of Alternative Investments, Fall, pp. 26-44.

Crow, E. and M. Siddiqui (1967). Robust Estimation of Location. Journal of the American Statistical Association, Vol. 62, pp. 353-389.

Fasano, G. and A. Franceschini (1987). Monthly Notices of the Royal Astronomical Society, Vol. 225, pp. 155-170.

Geltner, D. (1991). Smoothing in Appraisal-Based Returns. Journal of Real Estate Finance and Economics, Vol. 4, pp. 327-345.

Hinkley, D. (1975). On Power Transformations to Symmetry. Biometrika, Vol. 62, pp. 101-111.

Kat, H. and J. Miffre (2005). Hedge Fund Performance: The Role of Non-Normality Risks and Conditional Asset Allocation, Working Paper, Alternative Investment Research Centre, Cass Business School, City University London, (downloadable from www.cass.city.ac.uk/airc).

Kat, H. and H. Palaro (2005). Who Needs Hedge Funds? A Copula-Based Approach to Hedge Fund Return Replication, Working paper, Alternative Investment Research Centre, Cass Business School, City University London, (downloadable from www.cass.city.ac.uk/airc).

Lo, A., M. Getmansky and I. Makarov (2004). An Econometric Analysis of Serial Correlation and Illiquidity in Hedge Fund Returns. Journal of Financial Economics, Vol. 74, pp. 529-609.

Footnotes

¹ This is confirmed by the results in Kat and Miffre (2005).

² See Akaike (1973) for details.

³ Distributions and copulas as defined in Kat and Palaro (2005).

⁴ See Hinkley (1975) and Crow and Siddiqui (1967) for details.

⁵ See Fasano and Franceschini (1987) for details.

⁶ The CISDM Fund of Funds Index recorded a return of 4.72% over the first 11 months of 2005. Over the same period, the Barclay/Global HedgeSource Fund of Funds Index registered 4.81%.

⁷ More in particular, we traded S&P 500 futures on the CME and T-bond futures on the CBOT. Both contracts are in the top 10 of most traded futures contracts in the US.

⁸ In this context it is important to note that at least for some of the more complex distributions encountered (see Table 3), the correlation coefficient will not be a particularly good measure of dependence and may underestimate the true level of dependence.