New entrants into the field of systematic investing must look for ways to differentiate themselves. Impressive backtested performance is one way to stand out. But are these backtests robust? Are they likely to withstand the challenges of real-world implementation? Historical returns can be noisy, and even small changes to how an experiment is run can produce very different outcomes. While it is easy to find exciting results in a backtest, we believe the right thing to do is to evaluate these findings using a comprehensive research framework and determine whether the discoveries are truly new and can benefit investors going forward.
SETTING THE STAKES
Let’s say we have two simulated strategies, A and B, that focus on US small cap value stocks and are rebalanced annually. Strategy A returned 15.6% per year from January 1973 to June 2019. A slight methodology tweak to strategy B, however, boosted its performance to 16.7%. The easy thing to do with this observation is to conclude that B is superior. After all, a pickup of 110 basis points per year (1.1%) is no small feat.
So, what was the special enhancement for B relative to A? Simply changing the rebalancing month from May to January. As we see from Exhibit 1, there was substantial variation in the returns based on the choice of rebalancing month for otherwise identical small cap value strategies. However, one should not interpret this 110-basis point spread as an expected value-add. In fact, this is a cautionary tale about the potential for noise in empirical research.
What’s the right thing to do? Be wary of the potential for such noise and seek to mitigate its influence on the inferences drawn from the empirical findings. In this case, the right approach could be to use annual staggered rebalancing that takes an average of all the bars in Exhibit 1, thus reducing the potential to game the backtest by picking the rebalancing month.
Simulated data for research purposes only. This does not reflect actual performance of a live or proposed strategy. Results of an actual account may vary significantly. Simulated returns are hypothetical, are subject to numerous limitations and do not reflect costs or fees associated with an actual investment. Source: Dimensional, using CRSP and Compustat data. Simulations include small cap value firms excluding lower profitability firms. Small cap is defined as approximately the bottom 8% of the market capitalization. Value represents the bottom 35% of the market capitalization within small caps based on price-to-book ratio. In addition, the simulations exclude firms with high year-on-year asset growth, up to 5% of eligible small cap universe by market cap and consider momentum. No sector constraints are applied. Each simulation is rebalanced annually in a different month. See Appendix “Important Disclosures” for important information on simulated performance.
IN SEARCH OF ANOMALIES
When it comes to financial research, few things are more exciting than finding a new anomaly, or a variable that appears to drive differences in average returns. Let’s take stock volatility as an example. As shown in Exhibit 2, using Fama/French data on quintiles formed by sorting US stocks on past volatility (standard deviation), average returns for the low volatility quintile have been similar to those of the Fama/French Total US Market Research Index since the 1960s, despite the former having lower-than-market volatility. The easy thing to do? Declare that we have found a new anomaly.
US Market represented by the Fama/French Total US Market Research Index. Low Volatility Stocks represented by the lowest quintile of stocks from a univariate sort on return variance (a measure of how returns vary from the average). Variance sorted portfolio data available from Ken French’s Data Library: http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html.
Not so fast. We believe the right way to analyze a new pattern in the historical return data is in the context of known drivers of expected returns. Specifically, is this new observation additive to our understanding of asset pricing? When evaluated against the Fama/French Five-Factor Model in Exhibit 3, the answer appears to be no. The small intercept and t-stat suggest the returns of low volatility stocks are well explained by their exposure to known return drivers. No anomaly here, just a reminder that empirical analysis should not be conducted in a vacuum.
TWEAKING THE MOUSETRAP
Exceptionally negative recent value premiums have prompted many investors to scramble for ways to improve their process of pursuing the premium. That has led some down the road of tinkering with the metric used to define value versus growth. One example is adjusting book values for intangible assets, such as patents, copyrights, brands, and reputation.
This adjustment is alluring on the surface. Over the 10-year period ending December 2018, using a price-to-book ratio adjusted for internally developed intangibles would have substantially narrowed the annualized return difference between value and growth but not entirely eradicated value’s underperformance.1 The easy thing to do? Adopt the fix immediately. The right thing to do? Take a step back and ask: “Are internal intangibles a new phenomenon? And, if not new, have they grown in importance over time?” The answer to the first question is no. History buffs will know that the US started issuing patents back in 1790 and registering trademarks in 1870. So intangibles have been part of the economic landscape and capital markets for a long time.
To answer the second question, you have to estimate internally developed intangibles because they are generally expensed on the income statement under the US accounting principles, rather than capitalized on the balance sheet. Our recent paper on intangibles did that, finding that internally developed intangibles have been a steady fraction of company assets for a long time. Exhibit 4 shows, for the US Market, that they represented about 30% of company assets back in the 1980s and they represent about 30% of company assets today.
Our deep dive into intangibles also highlighted the noise involved in the estimation of internally developed intangibles. Procedures for doing so involve accumulating prior expenditures on research and design (R&D) and selling, general, and administration (SGA), relying on assumptions for how much of each component to include and over what time horizon to amortize. Couple these dependencies with data limitations, particularly for R&D, and there is ample susceptibility to noise with this adjustment.
Perhaps due to this noise and the higher uncertainty around the valuation of internally developed intangibles (which, unlike externally acquired ones, do not go through a market assessment), we find that adjusting for internally developed intangibles does not yield consistently higher value and profitability premiums.
As mentioned earlier, the US value premium over the past decade benefited from the adjustment. But further inspection of this result shows it is mainly driven by different sector exposures—turns out, the past decade was a good one for technology stocks, and technology had a higher weight in a value strategy formed on adjusted price-to-book. And, despite an improvement with the adjustment, the value premium was still negative.
Looking over the longer term and assessing value and profitability premiums together, the impact from intangibles adjustments has been minimal. Specifically, we see that the value premium gets a little larger while the profitability premium gets a little smaller. The net effect is near zero. There is no compelling evidence an investor can more effectively pursue higher expected returns by adjusting the value and profitability metrics for internally developed intangibles.
In summary, the easy thing to do in response to the recent dismal performance of the value premium is to immediately adopt an intangibles adjustment. But we believe the right thing to do is to evaluate thoughtfully and fully the impact of such an adjustment. We need to consider the high level of uncertainty around the value of internally developed intangibles (for example, only 8% of drugs that start the research phase end up in the marketplace2 ) and the noise that this adjustment will inject into the implementation process. We also need to consider that the empirical results from this adjustment are not compelling. Moreover, mixing and matching accounting variables to find the optimum backtest risks falling into the trap of overfitting the data3 —in other words, finding adjustments that make the past returns look great but might have no impact on future returns. The right thing to do is to look beyond an alpha or a Sharpe ratio and assess the rationale behind using each individual variable, how alternative variable specifications may differ, and what consequences this might have for implementation.
TIMING ISN’T EVERYTHING
It is probably not news to value investors that premiums can be negative even for long periods of time. There is obviously a major incentive to find ways to predict and avoid disappointing performance. Accordingly, substantial research has been conducted on timing markets and premiums.
In great news for investors who own a time machine, Dimensional’s research on the subject has even identified a successful timing strategy for the value premium. A mean reversion approach in Italian stocks that switches in and out of value based on the trailing average of the value premium has outperformed a buy-and-hold value approach by 7.5 percentage points per year. For those of us living in the present, enthusiasm over this result continuing in the future is tempered by the fact that this specific timing strategy is the best one out of 680 strategies tested4 and actually underperformed buy-and-hold strategies outside of Italian stocks by more than two percentage points per year. As illustrated by the excess returns for all 680 strategies in Exhibit 5, underperformance was by far the most likely outcome for these approaches to timing premiums.
The odds are high that—if you are willing to pore through the data long enough—you will eventually find a timing strategy that worked in the past. There are many inputs to a timing strategy: the premium being pursued, the indicator used for timing (e.g., valuation ratios or past performance), and the threshold for switching (in both directions), to name a few. The vast number of potential combinations of these inputs is ripe for data mining. Of course, the successful outcomes are the ones that will garner attention, but investors should acknowledge these as pyrite5 : attractive at first glance but not robust to further testing.
Realized premiums can be negative even for long stretches of time. The easy thing to do is to vary exposure to premiums in accordance with the output of a model that appears successful in a backtest. But rigorous research casts doubt on this approach effectively reducing the downside risks of the equity, size, value, or profitability premiums. Moreover, this approach can often yield meaningful trading and tax costs. Therefore, we believe the right thing to do is to deliver what you say you will deliver through the continuous pursuit of the reliable equity premiums. In addition to reducing investor surprises, this helps ensure one captures the premiums when they appear.
FINDING WISDOM
In the interest of writing a column rather than a textbook, we have touched on only a fraction of the empirical research on expected returns. If all these decades of poking and prodding the data have proved anything, it’s that you can’t prove something with data alone. The endless combinations available to researchers and the noise inherent in stock return data imply historical return comparisons should be evaluated with a heaping spoon of salt.
This is not to dismiss the importance of empirical research. Performance simulations can help us understand the behavior of returns and inform our expectations for future returns. However, investors need to avoid the risk of extrapolating historical patterns that may have occurred by chance. How can investors wade through the noise? Comprehensively evaluating the research may be a tall order, just as it’s probably not realistic to expect a patient to perform his or her own surgery. But a good place to start is asking questions about empirical techniques. The more moving parts in the backtest, the more susceptible the inferences are to noise. An investor’s assessment of the likelihood that simulated investment performance translates into real-world portfolios may depend upon whether a manager can demonstrate an approach to research that mitigates these sensitivities.
From a manager standpoint, it’s important to have a framework that guides research and hypotheses, even before looking at the data. Research must be conducted in a way that yields robust inferences. This includes running robust experiments that attempt to alleviate random sources of noise and outcomes excessively dependent upon a particular set of circumstances. It also means aligning the research with how its insights will be deployed within the investment process so that inferences from the research are more accurate. Deeply understanding empirical financial research requires extensive expertise developed through time. But this is a key part of applying judgment in the investment process: it is easy to find something that looks great in hindsight, but the right thing to do is to find ways to add value going forward.
1. Rizova, Savina and Namiko Saito (2020), “Intangibles and Expected Stock Returns”, available upon request. 2. Ernst R. Berndt, Adrian H.B. Gottschalk, and Matthew W. Strobeck, “Opportunities for Improving the Drug Development Process: Results From a Survey of Industry and the FDA,” Innovation Policy and the Economy 6 (2006): 91–121. 3. Robert Novy-Marx, “Backtesting Strategies Based on Multiple Signals,” NBER Working Paper No. w21329 (2015), available at: papers.ssrn.com/sol3/papers.cfm?abstract_id=2629935. 4. Wei Dei, “Premium Timing with Valuation Ratios” (white paper, Dimensional Fund Advisors, 2016), available here. 5. Pyrite, or iron sulfide, is more commonly known as “fool’s gold.” 6. Ryan H. Peters and Lucian A. Taylor, “Intangible Capital and the Investment-q Relation,” Journal of Financial Economics 123, no. 2 (2017): 251–272.
GLOSSARY
Regression analysis: A statistical technique for estimating the strength of the relation between one variable and one or more other variables.
Price-to-book ratio: The ratio of a firm’s market value to its book value, where market value is computed as price multiplied by shares outstanding and book value is the value of stockholder equity as reported on a company’s balance sheet.
Alpha: The rate of return on an investment in excess of a benchmark or return predicted by a financial model. A higher alpha value implies greater outperformance.
Premium: A return difference between two assets or portfolios. Size premium: the return difference between small market capitalization stocks and large market capitalization stocks. Value premium: the return difference between stocks with low relative prices (value) and stocks with high relative prices (growth). Profitability premium: the return difference between stocks of companies with high profitability over those with low profitability.
Profitability: A company’s operating income before depreciation and amortization minus interest expense scaled by book equity.
Sharpe ratio: A ratio of return per unit of volatility.
Five-Factor Model: An empirical asset pricing model developed by Fama and French (2015) to explain variation in stock returns.
APPENDIX
Premium timing simulations
Filters were applied to data retroactively and with the benefit of hindsight. Returns are not representative of indices or actual strategies and do not reflect costs and fees associated with an actual investment. Actual returns may be lower. The excess return of a trading rule is equal to the difference in average return between the rule and the long side of the premium that the rule is applied to. Data sources: CRSP and Compustat data for US firms listed on the NYSE, AMEX, or NASDAQ, and the historical book equity data in Kenneth French’s data library (http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html). The trading rules include nonparametric trading rules comparing valuation spreads to their historical distribution, linear trading rules based on the linear models predicting premiums based on valuation spreads, and logit trading rules based on the probability of a future premium being positive. Results will vary with each use and over time. For more information, please refer to Dimensional’s white paper “Premium Timing with Valuation Ratios,” available upon request.
Fama/French Indices
Fama/French Total US Market Research Index: July 1926–present: Fama/French Total US Market Research Factor + One-Month US Treasury Bills. Source: Kenneth French website.
Results shown during periods prior to each index’s index inception date do not represent actual returns of the respective index. Other periods selected may have different results, including losses. Backtested index performance is hypothetical and is provided for informational purposes only to indicate historical performance had the index been calculated over the relevant time periods. Backtested performance results assume the reinvestment of dividends and capital gains.
Intangibles Research
Source: Dimensional, using CRSP and Compustat data. The eligible universe includes all firms in the US excluding REITs, tracking stocks, and investment companies.
Following Peters and Taylor (2017),6 internally developed intangible capital for each firm at a point in time is estimated by accumulating the historical spending on research and development (R&D) and a fraction of selling, general, and administrative (SG&A) expenses while amortizing it at constant rates.
Important Disclosures
Simulated strategy returns based model/back-tested performance. These are not live strategies managed by Dimensional Fund Advisors LP or any of its affiliates. The performance was achieved with the retroactive application of a model designed with the benefit of hindsight; it does not represent actual investment performance. Backtested model performance is hypothetical (it does not reflect trading in actual accounts) and is provided for informational purposes only. The securities held in the model may differ significantly from those held in client accounts. Model performance may not reflect the impact that economic and market factors might have had on the advisor’s decision making if the advisor were actually managing client money. These strategies were not available for investment in the time periods depicted. Actual management of these types of simulated strategies may result in lower returns than the backtested results achieved with the benefit of hindsight. Past performance (including hypothetical past performance) does not guarantee future or actual results. The simulated performance shown is “gross performance,” which includes the reinvestment of dividends but does not reflect the deduction of investment advisory fees and other expenses. A client’s investment returns will be reduced by the advisory fees or other expenses it may incur. For example, if a 1% annual advisory fee were deducted quarterly and a client’s annual return were 10% (based on quarterly returns of approximately 2.41% each) before the deduction of advisory fees, the deduction of advisory fees would result in an annual return of approximately 8.91% due, in part, to the compound effect of such fees.
Eugene Fama and Ken French are members of the Board of Directors of the general partner of, and provide consulting services to, Dimensional Fund Advisors LP.
Robert Novy-Marx provides consulting services to Dimensional Fund Advisors LP.
The following persons listed as references are employees of Dimensional Investment LLC, a subsidiary of Dimensional Fund Advisors LP: Wei Dai, Savina Rizova, and Namiko Saito.
The information in this document is provided in good faith without any warranty and is intended for the recipient’s background information only. It does not constitute investment advice, recommendation, or an offer of any services or products for sale and is not intended to provide a sufficient basis on which to make an investment decision. It is the responsibility of any persons wishing to make a purchase to inform themselves of and observe all applicable laws and regulations. Unauthorized copying, reproducing, duplicating, or transmitting of this document are strictly prohibited. Dimensional accepts no responsibility for loss arising from the use of the information contained herein.
“Dimensional” refers to the Dimensional separate but affiliated entities generally, rather than to one particular entity. These entities are Dimensional Fund Advisors LP, Dimensional Fund Advisors Ltd., Dimensional Ireland Limited, DFA Australia Limited, Dimensional Fund Advisors Canada ULC, Dimensional Fund Advisors Pte. Ltd, Dimensional Japan Ltd., and Dimensional Hong Kong Limited. Dimensional Hong Kong Limited is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
UNITED STATES: Dimensional Fund Advisors LP is an investment advisor registered with the Securities and Exchange Commission.
CANADA: These materials have been prepared by Dimensional Fund Advisors Canada ULC. Commissions, trailing commissions, management fees, and expenses all may be associated with mutual fund investments. Unless otherwise noted, any indicated total rates of return reflect the historical annual compounded total returns, including changes in share or unit value and reinvestment of all dividends or other distributions, and do not take into account sales, redemption, distribution, or optional charges or income taxes payable by any security holder that would have reduced returns. Please read the prospectus before investing. Mutual funds are not guaranteed, their values change frequently, and past performance may not be repeated.
AUSTRALIA and NEW ZEALAND: This material is issued by DFA Australia Limited (AFS License No. 238093, ABN 46 065 937 671). This material is provided for information only. No account has been taken of the objectives, financial situation or needs of any particular person. Accordingly, to the extent this material constitutes general financial product advice, investors should, before acting on the advice, consider the appropriateness of the advice, having regard to the investor’s objectives, financial situation and needs. Any opinions expressed in this material reflect our judgement at the date of publication and are subject to change.
JAPAN Provided for institutional investors only. This document is deemed to be issued by Dimensional Japan Ltd., which is regulated by the Financial Services Agency of Japan and is registered as a Financial Instruments Firm conducting Investment Management Business and Investment Advisory and Agency Business. This material is solely for informational purposes only and shall not constitute an offer to sell or the solicitation to buy securities or enter into investment advisory contracts. The material in this article and any content contained herein may not be reproduced, copied, modified, transferred, disclosed, or used in any way not expressly permitted by Dimensional Japan Ltd. in writing. All expressions of opinion are subject to change without notice.
Dimensional Japan Ltd. Director of Kanto Local Finance Bureau (FIBO) No. 2683 Membership: Japan Investment Advisers Association
FOR PROFESSIONAL INVESTORS IN HONG KONG. This material is deemed to be issued by Dimensional Hong Kong Limited (CE No. BJE760) (“Dimensional Hong Kong”), which is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
This material should only be provided to “professional investors” (as defined in the Securities and Futures Ordinance [Chapter 571 of the Laws of Hong Kong] and its subsidiary legislation) and is not for use with the public. This material is not directed to any person in any jurisdiction where (by reason of that person’s nationality, residence, or otherwise) the publication or availability of this material are prohibited or which would subject Dimensional Hong Kong (including its affiliates) or any of Dimensional Hong Kong’s products or services to any registration, licensing, or other such legal requirements within such jurisdiction or country. When provided to prospective investors, this material forms part of, and must be provided together with, applicable fund offering materials. This material must not be provided to prospective investors on a standalone basis. Before acting on any information in this materiel, you should consider whether it is suitable for your particular circumstances and, if appropriate, seek professional advice.
Unauthorized copying, reproducing, duplicating, or transmitting of this material are prohibited. This material and the distribution of this material are not intended to constitute and do not constitute an offer or an invitation to offer to the Hong Kong public to acquire, dispose of, subscribe for, or underwrite any securities, structured products, or related financial products or instruments nor investment advice thereto. Any opinions and views expressed herein are subject to change. Neither Dimensional Hong Kong nor its affiliates shall be responsible or held responsible for any content prepared by financial advisors. Financial advisors in Hong Kong shall not actively market the services of Dimensional Hong Kong or its affiliates to the Hong Kong public.
SINGAPORE This material is deemed to be issued by Dimensional Fund Advisors Pte. Ltd., which is regulated by the Monetary Authority of Singapore and holds a capital markets services license for fund management.
This advertisement has not been reviewed by the Monetary Authority of Singapore. This information should not be considered investment advice or an offer of any security for sale. All information is given in good faith without any warranty and is not intended to provide professional, investment, or any other type of advice or recommendation and does not take into account the particular investment objectives, financial situation, or needs of individual recipients. Before acting on any information in this materiel, you should consider whether it is suitable for your particular circumstances and, if appropriate, seek professional advice. Dimensional Fund Advisors Pte. Ltd. does not accept any responsibility and cannot be held liable for any person’s use of or reliance on the information and opinions contained herein. Neither Dimensional Fund Advisors Pte. Ltd. nor its affiliates shall be responsible or held responsible for any content prepared by financial advisors.
The Right Thing, Not the Easy Thing
New entrants into the field of systematic investing must look for ways to differentiate themselves. Impressive backtested performance is one way to stand out. But are these backtests robust? Are they likely to withstand the challenges of real-world implementation? Historical returns can be noisy, and even small changes to how an experiment is run can produce very different outcomes. While it is easy to find exciting results in a backtest, we believe the right thing to do is to evaluate these findings using a comprehensive research framework and determine whether the discoveries are truly new and can benefit investors going forward.
SETTING THE STAKES
Let’s say we have two simulated strategies, A and B, that focus on US small cap value stocks and are rebalanced annually. Strategy A returned 15.6% per year from January 1973 to June 2019. A slight methodology tweak to strategy B, however, boosted its performance to 16.7%. The easy thing to do with this observation is to conclude that B is superior. After all, a pickup of 110 basis points per year (1.1%) is no small feat.
So, what was the special enhancement for B relative to A? Simply changing the rebalancing month from May to January. As we see from Exhibit 1, there was substantial variation in the returns based on the choice of rebalancing month for otherwise identical small cap value strategies. However, one should not interpret this 110-basis point spread as an expected value-add. In fact, this is a cautionary tale about the potential for noise in empirical research.
What’s the right thing to do? Be wary of the potential for such noise and seek to mitigate its influence on the inferences drawn from the empirical findings. In this case, the right approach could be to use annual staggered rebalancing that takes an average of all the bars in Exhibit 1, thus reducing the potential to game the backtest by picking the rebalancing month.
Simulated data for research purposes only. This does not reflect actual performance of a live or proposed strategy. Results of an actual account may vary significantly. Simulated returns are hypothetical, are subject to numerous limitations and do not reflect costs or fees associated with an actual investment. Source: Dimensional, using CRSP and Compustat data. Simulations include small cap value firms excluding lower profitability firms. Small cap is defined as approximately the bottom 8% of the market capitalization. Value represents the bottom 35% of the market capitalization within small caps based on price-to-book ratio. In addition, the simulations exclude firms with high year-on-year asset growth, up to 5% of eligible small cap universe by market cap and consider momentum. No sector constraints are applied. Each simulation is rebalanced annually in a different month. See Appendix “Important Disclosures” for important information on simulated performance.
IN SEARCH OF ANOMALIES
When it comes to financial research, few things are more exciting than finding a new anomaly, or a variable that appears to drive differences in average returns. Let’s take stock volatility as an example. As shown in Exhibit 2, using Fama/French data on quintiles formed by sorting US stocks on past volatility (standard deviation), average returns for the low volatility quintile have been similar to those of the Fama/French Total US Market Research Index since the 1960s, despite the former having lower-than-market volatility. The easy thing to do? Declare that we have found a new anomaly.
US Market represented by the Fama/French Total US Market Research Index. Low Volatility Stocks represented by the lowest quintile of stocks from a univariate sort on return variance (a measure of how returns vary from the average). Variance sorted portfolio data available from Ken French’s Data Library: http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html.
Not so fast. We believe the right way to analyze a new pattern in the historical return data is in the context of known drivers of expected returns. Specifically, is this new observation additive to our understanding of asset pricing? When evaluated against the Fama/French Five-Factor Model in Exhibit 3, the answer appears to be no. The small intercept and t-stat suggest the returns of low volatility stocks are well explained by their exposure to known return drivers. No anomaly here, just a reminder that empirical analysis should not be conducted in a vacuum.
TWEAKING THE MOUSETRAP
Exceptionally negative recent value premiums have prompted many investors to scramble for ways to improve their process of pursuing the premium. That has led some down the road of tinkering with the metric used to define value versus growth. One example is adjusting book values for intangible assets, such as patents, copyrights, brands, and reputation.
This adjustment is alluring on the surface. Over the 10-year period ending December 2018, using a price-to-book ratio adjusted for internally developed intangibles would have substantially narrowed the annualized return difference between value and growth but not entirely eradicated value’s underperformance.1 The easy thing to do? Adopt the fix immediately. The right thing to do? Take a step back and ask: “Are internal intangibles a new phenomenon? And, if not new, have they grown in importance over time?” The answer to the first question is no. History buffs will know that the US started issuing patents back in 1790 and registering trademarks in 1870. So intangibles have been part of the economic landscape and capital markets for a long time.
To answer the second question, you have to estimate internally developed intangibles because they are generally expensed on the income statement under the US accounting principles, rather than capitalized on the balance sheet. Our recent paper on intangibles did that, finding that internally developed intangibles have been a steady fraction of company assets for a long time. Exhibit 4 shows, for the US Market, that they represented about 30% of company assets back in the 1980s and they represent about 30% of company assets today.
Our deep dive into intangibles also highlighted the noise involved in the estimation of internally developed intangibles. Procedures for doing so involve accumulating prior expenditures on research and design (R&D) and selling, general, and administration (SGA), relying on assumptions for how much of each component to include and over what time horizon to amortize. Couple these dependencies with data limitations, particularly for R&D, and there is ample susceptibility to noise with this adjustment.
Perhaps due to this noise and the higher uncertainty around the valuation of internally developed intangibles (which, unlike externally acquired ones, do not go through a market assessment), we find that adjusting for internally developed intangibles does not yield consistently higher value and profitability premiums.
As mentioned earlier, the US value premium over the past decade benefited from the adjustment. But further inspection of this result shows it is mainly driven by different sector exposures—turns out, the past decade was a good one for technology stocks, and technology had a higher weight in a value strategy formed on adjusted price-to-book. And, despite an improvement with the adjustment, the value premium was still negative.
Looking over the longer term and assessing value and profitability premiums together, the impact from intangibles adjustments has been minimal. Specifically, we see that the value premium gets a little larger while the profitability premium gets a little smaller. The net effect is near zero. There is no compelling evidence an investor can more effectively pursue higher expected returns by adjusting the value and profitability metrics for internally developed intangibles.
In summary, the easy thing to do in response to the recent dismal performance of the value premium is to immediately adopt an intangibles adjustment. But we believe the right thing to do is to evaluate thoughtfully and fully the impact of such an adjustment. We need to consider the high level of uncertainty around the value of internally developed intangibles (for example, only 8% of drugs that start the research phase end up in the marketplace2 ) and the noise that this adjustment will inject into the implementation process. We also need to consider that the empirical results from this adjustment are not compelling. Moreover, mixing and matching accounting variables to find the optimum backtest risks falling into the trap of overfitting the data3 —in other words, finding adjustments that make the past returns look great but might have no impact on future returns. The right thing to do is to look beyond an alpha or a Sharpe ratio and assess the rationale behind using each individual variable, how alternative variable specifications may differ, and what consequences this might have for implementation.
TIMING ISN’T EVERYTHING
It is probably not news to value investors that premiums can be negative even for long periods of time. There is obviously a major incentive to find ways to predict and avoid disappointing performance. Accordingly, substantial research has been conducted on timing markets and premiums.
In great news for investors who own a time machine, Dimensional’s research on the subject has even identified a successful timing strategy for the value premium. A mean reversion approach in Italian stocks that switches in and out of value based on the trailing average of the value premium has outperformed a buy-and-hold value approach by 7.5 percentage points per year. For those of us living in the present, enthusiasm over this result continuing in the future is tempered by the fact that this specific timing strategy is the best one out of 680 strategies tested4 and actually underperformed buy-and-hold strategies outside of Italian stocks by more than two percentage points per year. As illustrated by the excess returns for all 680 strategies in Exhibit 5, underperformance was by far the most likely outcome for these approaches to timing premiums.
The odds are high that—if you are willing to pore through the data long enough—you will eventually find a timing strategy that worked in the past. There are many inputs to a timing strategy: the premium being pursued, the indicator used for timing (e.g., valuation ratios or past performance), and the threshold for switching (in both directions), to name a few. The vast number of potential combinations of these inputs is ripe for data mining. Of course, the successful outcomes are the ones that will garner attention, but investors should acknowledge these as pyrite5 : attractive at first glance but not robust to further testing.
Realized premiums can be negative even for long stretches of time. The easy thing to do is to vary exposure to premiums in accordance with the output of a model that appears successful in a backtest. But rigorous research casts doubt on this approach effectively reducing the downside risks of the equity, size, value, or profitability premiums. Moreover, this approach can often yield meaningful trading and tax costs. Therefore, we believe the right thing to do is to deliver what you say you will deliver through the continuous pursuit of the reliable equity premiums. In addition to reducing investor surprises, this helps ensure one captures the premiums when they appear.
FINDING WISDOM
In the interest of writing a column rather than a textbook, we have touched on only a fraction of the empirical research on expected returns. If all these decades of poking and prodding the data have proved anything, it’s that you can’t prove something with data alone. The endless combinations available to researchers and the noise inherent in stock return data imply historical return comparisons should be evaluated with a heaping spoon of salt.
This is not to dismiss the importance of empirical research. Performance simulations can help us understand the behavior of returns and inform our expectations for future returns. However, investors need to avoid the risk of extrapolating historical patterns that may have occurred by chance. How can investors wade through the noise? Comprehensively evaluating the research may be a tall order, just as it’s probably not realistic to expect a patient to perform his or her own surgery. But a good place to start is asking questions about empirical techniques. The more moving parts in the backtest, the more susceptible the inferences are to noise. An investor’s assessment of the likelihood that simulated investment performance translates into real-world portfolios may depend upon whether a manager can demonstrate an approach to research that mitigates these sensitivities.
From a manager standpoint, it’s important to have a framework that guides research and hypotheses, even before looking at the data. Research must be conducted in a way that yields robust inferences. This includes running robust experiments that attempt to alleviate random sources of noise and outcomes excessively dependent upon a particular set of circumstances. It also means aligning the research with how its insights will be deployed within the investment process so that inferences from the research are more accurate. Deeply understanding empirical financial research requires extensive expertise developed through time. But this is a key part of applying judgment in the investment process: it is easy to find something that looks great in hindsight, but the right thing to do is to find ways to add value going forward.
LEARN MORE
An Exceptional Value Premium
Untangling Intangibles
Why Worry About Survivorship Bias?
1. Rizova, Savina and Namiko Saito (2020), “Intangibles and Expected Stock Returns”, available upon request.
2. Ernst R. Berndt, Adrian H.B. Gottschalk, and Matthew W. Strobeck, “Opportunities for Improving the Drug Development Process: Results From a Survey of Industry and the FDA,” Innovation Policy and the Economy 6 (2006): 91–121.
3. Robert Novy-Marx, “Backtesting Strategies Based on Multiple Signals,” NBER Working Paper No. w21329 (2015), available at: papers.ssrn.com/sol3/papers.cfm?abstract_id=2629935.
4. Wei Dei, “Premium Timing with Valuation Ratios” (white paper, Dimensional Fund Advisors, 2016), available here.
5. Pyrite, or iron sulfide, is more commonly known as “fool’s gold.”
6. Ryan H. Peters and Lucian A. Taylor, “Intangible Capital and the Investment-q Relation,” Journal of Financial Economics 123, no. 2 (2017): 251–272.
GLOSSARY
Regression analysis: A statistical technique for estimating the strength of the relation between one variable and one or more other variables.
Price-to-book ratio: The ratio of a firm’s market value to its book value, where market value is computed as price multiplied by shares outstanding and book value is the value of stockholder equity as reported on a company’s balance sheet.
Alpha: The rate of return on an investment in excess of a benchmark or return predicted by a financial model. A higher alpha value implies greater outperformance.
Premium: A return difference between two assets or portfolios. Size premium: the return difference between small market capitalization stocks and large market capitalization stocks. Value premium: the return difference between stocks with low relative prices (value) and stocks with high relative prices (growth). Profitability premium: the return difference between stocks of companies with high profitability over those with low profitability.
Profitability: A company’s operating income before depreciation and amortization minus interest expense scaled by book equity.
Sharpe ratio: A ratio of return per unit of volatility.
Five-Factor Model: An empirical asset pricing model developed by Fama and French (2015) to explain variation in stock returns.
APPENDIX
Premium timing simulations
Filters were applied to data retroactively and with the benefit of hindsight. Returns are not representative of indices or actual strategies and do not reflect costs and fees associated with an actual investment. Actual returns may be lower. The excess return of a trading rule is equal to the difference in average return between the rule and the long side of the premium that the rule is applied to. Data sources: CRSP and Compustat data for US firms listed on the NYSE, AMEX, or NASDAQ, and the historical book equity data in Kenneth French’s data library (http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html). The trading rules include nonparametric trading rules comparing valuation spreads to their historical distribution, linear trading rules based on the linear models predicting premiums based on valuation spreads, and logit trading rules based on the probability of a future premium being positive. Results will vary with each use and over time. For more information, please refer to Dimensional’s white paper “Premium Timing with Valuation Ratios,” available upon request.
Fama/French Indices
Fama/French Total US Market Research Index: July 1926–present: Fama/French Total US Market Research Factor + One-Month US Treasury Bills. Source: Kenneth French website.
Results shown during periods prior to each index’s index inception date do not represent actual returns of the respective index. Other periods selected may have different results, including losses. Backtested index performance is hypothetical and is provided for informational purposes only to indicate historical performance had the index been calculated over the relevant time periods. Backtested performance results assume the reinvestment of dividends and capital gains.
Intangibles Research
Source: Dimensional, using CRSP and Compustat data. The eligible universe includes all firms in the US excluding REITs, tracking stocks, and investment companies.
Following Peters and Taylor (2017),6 internally developed intangible capital for each firm at a point in time is estimated by accumulating the historical spending on research and development (R&D) and a fraction of selling, general, and administrative (SG&A) expenses while amortizing it at constant rates.
Important Disclosures
Simulated strategy returns based model/back-tested performance. These are not live strategies managed by Dimensional Fund Advisors LP or any of its affiliates. The performance was achieved with the retroactive application of a model designed with the benefit of hindsight; it does not represent actual investment performance. Backtested model performance is hypothetical (it does not reflect trading in actual accounts) and is provided for informational purposes only. The securities held in the model may differ significantly from those held in client accounts. Model performance may not reflect the impact that economic and market factors might have had on the advisor’s decision making if the advisor were actually managing client money. These strategies were not available for investment in the time periods depicted. Actual management of these types of simulated strategies may result in lower returns than the backtested results achieved with the benefit of hindsight. Past performance (including hypothetical past performance) does not guarantee future or actual results. The simulated performance shown is “gross performance,” which includes the reinvestment of dividends but does not reflect the deduction of investment advisory fees and other expenses. A client’s investment returns will be reduced by the advisory fees or other expenses it may incur. For example, if a 1% annual advisory fee were deducted quarterly and a client’s annual return were 10% (based on quarterly returns of approximately 2.41% each) before the deduction of advisory fees, the deduction of advisory fees would result in an annual return of approximately 8.91% due, in part, to the compound effect of such fees.
Eugene Fama and Ken French are members of the Board of Directors of the general partner of, and provide consulting services to, Dimensional Fund Advisors LP.
Robert Novy-Marx provides consulting services to Dimensional Fund Advisors LP.
The following persons listed as references are employees of Dimensional Investment LLC, a subsidiary of Dimensional Fund Advisors LP: Wei Dai, Savina Rizova, and Namiko Saito.
The information in this document is provided in good faith without any warranty and is intended for the recipient’s background information only. It does not constitute investment advice, recommendation, or an offer of any services or products for sale and is not intended to provide a sufficient basis on which to make an investment decision. It is the responsibility of any persons wishing to make a purchase to inform themselves of and observe all applicable laws and regulations. Unauthorized copying, reproducing, duplicating, or transmitting of this document are strictly prohibited. Dimensional accepts no responsibility for loss arising from the use of the information contained herein.
“Dimensional” refers to the Dimensional separate but affiliated entities generally, rather than to one particular entity. These entities are Dimensional Fund Advisors LP, Dimensional Fund Advisors Ltd., Dimensional Ireland Limited, DFA Australia Limited, Dimensional Fund Advisors Canada ULC, Dimensional Fund Advisors Pte. Ltd, Dimensional Japan Ltd., and Dimensional Hong Kong Limited. Dimensional Hong Kong Limited is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
UNITED STATES: Dimensional Fund Advisors LP is an investment advisor registered with the Securities and Exchange Commission.
CANADA: These materials have been prepared by Dimensional Fund Advisors Canada ULC. Commissions, trailing commissions, management fees, and expenses all may be associated with mutual fund investments. Unless otherwise noted, any indicated total rates of return reflect the historical annual compounded total returns, including changes in share or unit value and reinvestment of all dividends or other distributions, and do not take into account sales, redemption, distribution, or optional charges or income taxes payable by any security holder that would have reduced returns. Please read the prospectus before investing. Mutual funds are not guaranteed, their values change frequently, and past performance may not be repeated.
AUSTRALIA and NEW ZEALAND: This material is issued by DFA Australia Limited (AFS License No. 238093, ABN 46 065 937 671). This material is provided for information only. No account has been taken of the objectives, financial situation or needs of any particular person. Accordingly, to the extent this material constitutes general financial product advice, investors should, before acting on the advice, consider the appropriateness of the advice, having regard to the investor’s objectives, financial situation and needs. Any opinions expressed in this material reflect our judgement at the date of publication and are subject to change.
JAPAN
Provided for institutional investors only. This document is deemed to be issued by Dimensional Japan Ltd., which is regulated by the Financial Services Agency of Japan and is registered as a Financial Instruments Firm conducting Investment Management Business and Investment Advisory and Agency Business. This material is solely for informational purposes only and shall not constitute an offer to sell or the solicitation to buy securities or enter into investment advisory contracts. The material in this article and any content contained herein may not be reproduced, copied, modified, transferred, disclosed, or used in any way not expressly permitted by Dimensional Japan Ltd. in writing. All expressions of opinion are subject to change without notice.
Dimensional Japan Ltd.
Director of Kanto Local Finance Bureau (FIBO) No. 2683
Membership: Japan Investment Advisers Association
FOR PROFESSIONAL INVESTORS IN HONG KONG.
This material is deemed to be issued by Dimensional Hong Kong Limited (CE No. BJE760) (“Dimensional Hong Kong”), which is licensed by the Securities and Futures Commission to conduct Type 1 (dealing in securities) regulated activities only and does not provide asset management services.
This material should only be provided to “professional investors” (as defined in the Securities and Futures Ordinance [Chapter 571 of the Laws of Hong Kong] and its subsidiary legislation) and is not for use with the public. This material is not directed to any person in any jurisdiction where (by reason of that person’s nationality, residence, or otherwise) the publication or availability of this material are prohibited or which would subject Dimensional Hong Kong (including its affiliates) or any of Dimensional Hong Kong’s products or services to any registration, licensing, or other such legal requirements within such jurisdiction or country. When provided to prospective investors, this material forms part of, and must be provided together with, applicable fund offering materials. This material must not be provided to prospective investors on a standalone basis. Before acting on any information in this materiel, you should consider whether it is suitable for your particular circumstances and, if appropriate, seek professional advice.
Unauthorized copying, reproducing, duplicating, or transmitting of this material are prohibited. This material and the distribution of this material are not intended to constitute and do not constitute an offer or an invitation to offer to the Hong Kong public to acquire, dispose of, subscribe for, or underwrite any securities, structured products, or related financial products or instruments nor investment advice thereto. Any opinions and views expressed herein are subject to change. Neither Dimensional Hong Kong nor its affiliates shall be responsible or held responsible for any content prepared by financial advisors. Financial advisors in Hong Kong shall not actively market the services of Dimensional Hong Kong or its affiliates to the Hong Kong public.
SINGAPORE
This material is deemed to be issued by Dimensional Fund Advisors Pte. Ltd., which is regulated by the Monetary Authority of Singapore and holds a capital markets services license for fund management.
This advertisement has not been reviewed by the Monetary Authority of Singapore. This information should not be considered investment advice or an offer of any security for sale. All information is given in good faith without any warranty and is not intended to provide professional, investment, or any other type of advice or recommendation and does not take into account the particular investment objectives, financial situation, or needs of individual recipients. Before acting on any information in this materiel, you should consider whether it is suitable for your particular circumstances and, if appropriate, seek professional advice. Dimensional Fund Advisors Pte. Ltd. does not accept any responsibility and cannot be held liable for any person’s use of or reliance on the information and opinions contained herein. Neither Dimensional Fund Advisors Pte. Ltd. nor its affiliates shall be responsible or held responsible for any content prepared by financial advisors.
12/11/2020
https://www.mydimensional.com/the-right-thing-not-the-easy-thing
Categories
Categories