Andreas Pick

Academic writing tips for econometrics theses

Sun, 08 May 2022 15:32:00 +0000

In this blog post, I address some frequently made mistakes in the writing of econometrics theses. Naturally, there is more to academic writing than can fit into a single blog post, so I will focus on the issues that seem to go wrong most often. Note, also that the comments reflect my own preferences. When writing in other circumstances, take from this what you will. Finally, this blog post is about the academic side of writing not about English prose. I am semi-qualified to talk about the former and absolutely unqualified to talk about the latter.

A general point is that you need to write for the reader and not yourself. Do not focus on your discovery process but on the research results that you want to convey to the reader. These results are the main message of your writing and everything else needs to serve the purpose of getting them across as clearly as possible.

The introduction

This is arguably the most important section of a scientific paper and of your thesis. This is where you make the reader enthusiastic about your research and entice them to continue reading. Academic readers (such as your supervisor) have many papers to read, so any paper will quickly have to prove its worth to keep them reading. Do not waste time with „warming up“ the reader, instead cut straight to the chase.

Unless you have a very obscure topic that needs an introduction, you do not need to provide any background information before announcing what your research is about. It can feel brutal to start the thesis without giving a general introduction but the thesis will read better for it.

A good structure for the first few paragraphs of the intro is as follows:

What exactly is the research question you are dealing with. An example: „This thesis analyses the forecast accuracy of boosted regression trees for Dutch inflation.“ Under no circumstances start with a general statement such as „Inflation is an important variable for central banks“ — the reader knows this already and you have wasted their time making them read this.
Next, let the reader know how you are approaching the topic. For example: „The empirical analysis compares forecasts for Dutch CPI inflation from 1992 to 2021. The forecasts are constructed from boosted regression trees and are compared to those from benchmark models, such as AR models and the random walk.“ Then, provide some more details of your approach to the problem. Be reasonably precise without killing the reader with minute details.
Next, explain your headline findings, that is, what are the most important take-aways. For example: „I find that the random forest has a root mean square forecast error that is 11% below that of the best AR model.“ More headline results could follow, such as the performance of the other methods, important results in sub-periods, and so forth.

These first few paragraphs should explain your research and your research only. Once you are done — and this can take three or more paragraphs — the reader should be aware of what the general research question is, how you go about answering it, and what you find.

General introductions are only necessary if the topic is very obscure, for example, due to the regulation of a particular industry or the requirements of a particular company where you write your thesis. Any general introduction that provides a background should nonetheless be specific to your research and not exceed a short paragraph. Always avoid general statements, such as „GDP is an important economic variable“ from which the reader learns nothing.

Literature

It is important that you place your own research in the literature. This serves a number of purposes. First, it is unlikely that you have not based your research on that of others and you need to indicate what you have been building on. This is important as even the suspicion of plagiarism is damaging. Second, given that most readers will be aware of the existing literature, you need to emphasise what is new in your research. This is easier when discussing what has been done before. Any difference will then be your own work and you can then easily point out those differences. Third, academic readers can be impatient and will want to know early on that it is worthwhile reading your work. Pointing out the differences to the previous literature can help emphasise the importance of your work.

From the above, it should be clear that your review of the literature needs to be tightly linked to your own research, which you explained at the beginning of your thesis. For this reason, it is a good idea to have the discussion of the literature in the introduction without a separating section header, that is, do not have a separate section called „Literature“ but discuss the literature in the introduction.

Another reason to avoid a separate literature section is that section headings seem to act in a similar way to doors. When looking for something and walking into another room, people tend to forget what they came looking for (see evidence described here). Similarly, section headings appear to disconnect the mind of the writer from the previous section, which results in literature reviews that are only vaguely connected to the description of the research in the previous section. This makes the literature review much less useful.

Finally, for the reader, a literature section blocks the path from the exciting promise of great research in the introduction to the actual research. The purpose of writing a research paper or a thesis is to transmit a great idea from the creator to the reader. Anything that blocks this transmission, such as a section on the literature that only vaguely relates to that idea, is a problem. The computer scientist Simon Peyton Jones explains this eloquently.

Comprehensive writing

Writing is hard work and takes time. Good writing explains as much as possible in as few words as possible. Make it an exercise to go over your thesis and ask yourself if you can express what you have written in fewer words. (For example, the previous sentence could drop “you can express what you have written” for “it can be expressed”. It could have been better that way.) You will be surprised how many words you can save. This does not mean you should drop content but that the same content can be transmitted with fewer words. It is possible to be too stingy with words but I have yet to encounter a thesis where this was the case.

What needs to be explained in the thesis

Sometimes students struggle with the questions which concepts to explain. There are two issues to consider. First, how well known is a concept. Generally, if it was in a course that everybody in your degree followed, you can assume it is known. Second, how central is the concept to your thesis. If it is central you need to explain it. Say, you write about an extension to the AR model, then you need to have the equation of the basic AR model in the thesis even if this was the subject of a second year course.

The language of econometrics

I supervise many theses that use machine learning methods and there seems to be a temptation to use machine learning slang in econometrics theses. This is risky. Not because machine learning slang is worse than econometrics slang but because you will end up using different words for the same concept. What is the difference between features and regressors, weights and parameters? Being able to translate a machine learning model into the language of econometrics has the added benefit of demonstrating that you understand what you do.

Layout

As a rule, stick to the defaults that LaTeX uses. Two exceptions: use 11pt as your supervisor may not have perfect eyesight any more, and use a4paper as we are in Europe where A4 is the standard. Under absolutely no circumstances fiddle with the margins or the line spacing. Just don‘t.

Some details

Avoid footnotes.
Do not use punctuation after display equations.
In the References section, do not put “The” in front of journal names. That is, “Review of Economic Studies” and not “The Review of Economic Studies”.

Most importantly, enjoy the thesis process, including the writing part. It is hard work but it can be a lot of fun.

Do structural breaks matter when forecasting?

Sun, 21 May 2017 15:32:00 +0000

tl;dr Should you worry about structural breaks when forecasting? Probably not as much as you think. And if you do, you can test for the importance of a structural break for your forecast using a new test.

Structural breaks have been pinpointed as one of the culprits for forecast failures in the past. In a well-known article, Stock and Watson (1996, JBES) point out that a wide range of U.S. macroeconomic time series suffer from structural breaks. However, they also found that modeling the breaks does not appear to lead to substantial improvements in forecast accuracy as measured by the mean square forecast error. They conclude that “statistically significant parameter variation might be only marginally significant from a forecasting perspective”. However, the latter has never been investigated formally even if the existence of breaks kept forecasters worried.

A new paper with Tom Boot remedies the situation. We develop a new test to assess whether the forecast from a model that takes a potential structural break in the estimation sample into account is more precise than that from a model that ignores breaks. Along the way, we establish that a structural break that should be modeled to improve forecast accuracy is very large. The graph below, which is from the paper with Tom, shows the minimum size of a break at which modeling it makes forecasts more precise. The size depends on the time of the break, which, as a proportion of the sample size, is on the x-axis. For example, 0.2 means that the break occurs after the first 20% of the observations. On the y-axis is the magnitude of the break of the forecast model standardized by the variance of the model. The graph therefore suggests that a break that is early in the sample will need to be three standard deviations of the distribution of the forecast model before one would want to model it. This break magnitude decreases as the break appears later in the sample but stays above one standard deviation.

How likely is it to encounter such a break? In the paper, we use the 130 time series in the FRED-MD data base and check for structural breaks in rolling windows of 10 years length. The graph below (also from the paper with Tom) shows the frequency of detecting a break for each series (sorted into eight categories and within each category by number of breaks) over the different rolling windows using the test of Andrews (1993, Econometrica), which is an in-sample test of parameter variation, as dashed lines and our test as solid lines. It is clear that Stock and Watson’s hunch is correct: Andrews’ test of parameter variation finds many more breaks across all series than our test of significance from a forecast perspective.

The eagle-eyed may have spotted additional dashed lines below the solid lines. These represents a variant of our test. But it is clear that, compared to Andrews’ test, the difference between our two variants is minimal.

Dutch bank lending is growing just fine

Thu, 11 May 2017 10:30:00 +0000

Disclaimer: This post (like all others) represents my opinion alone and should not be attributed to De Nederlandsche Bank, the Eurosystem or, indeed, anyone else.

tl;dr An article in the Financieele Dagblad claims that Dutch bank lending growth does not seem to be in line with economic growth in the Netherlands. This observation is based on a stock-flow mismatch. If one corrects this mistake, the conclusion gets reversed.

In a recent article in the Financieele Dagblad, Mathijs Bouman asks “If it is going so well with the Dutch economy why is lending not growing, yet?” (All quotes are my translations from the Dutch.) His evidence is the graph below (replicated from the same data used by Bouman, which is available at DNB’s website), which plots the growth rate of bank credit to Dutch firms over time.

Mr Bouwman says: “While in most European countries the credit flow is already growing healthily, it is still shrinking in the Netherlands.” In the graph above, the current growth rate is at about -3 per cent.

The problem with this statement is that the graph does not plot the correct variable. In the graph above is the growth rate of the stock of credit. That is, the growth rate of all the lending that has been done in the past that hasn’t been repaid yet. However, if firms need to borrow, the investment that they undertake will be financed by new borrowing and not by credit these firms took out years ago. In other words, investment depends on the flow of credit, and as a result, the growth rate of investment will be related to the growth of the flow of credit.

It turns out that the DNB spreadsheet is transparent in this. The first column is headlined Standen, that is, the stock of credit, and the growth rates in the DNB spreadsheet are calculated from this variable. Helpfully, however, the column next to Standen is Stromen, that is, flows. If one uses the flow of data to calculate the credit impulse, which is our prefered measure of the growth of credit, one obtains the graph below.

This graph reflects recent economic developments: In the years leading up to the financial crisis, the Netherlands was experiencing a credit driven boom. This is followed by the financial crisis, recovery around 2011-12, another milder downturn, and then the current recovery.

Unfortunately, using the growth of credit stock in place of the credit impulse appears to be a very common mistake. It underlies the idea of credit cycles, that I wrote about in a previous blog entry, and it is the source of the literature on Phoenix miracles that I addressed with Mike Biggs and Thomas Mayer in this paper. Milton Friedman asked in 1992 if old fallacies ever died. Maybe they do live forever.

Financial cycles

Thu, 15 Sep 2016 10:30:00 +0000

Disclaimer: This post (like all others) represents my opinion alone and should not be attributed to De Nederlandsche Bank, the Eurosystem or, indeed, anyone else.

tl;dr Financial cycles, the way they are commonly defined, do not exist separately from the business cycle. Evidence for separate credit cycles is artificially constructed via a stock-flow mismatch.

An increasing number of papers are appearing on the topic of financial cycles, including the papers presented at a DNB workshop I attended recently. In the papers of this workshop, financial cycles were defined as cycles in outstanding credit to the non-financial sector. The assumption is that credit cycles are distinct from business cycle as defined via GDP and that these different cycles are interesting to understand from a policy point of view.

As an example of what this might look like consider the graph below that plots consumption and investment (C+I) growth together with credit growth for the US. I use consumption and investment growth as the credit plotted in the graph (and used in the papers of the workshop) is to the private, non-financial sector only.

Notice that in the early and late 1970s and the mid 1980s credit growth does not seem to be synchronized with C+I growth. The most recent event where this divergence appeared was the recovery from the financial crisis in 2009. C+I growth recovers relatively swiftly but credit growth has the trough only a few quarters later. This led Calvo and Loo-Kung to call the US recovery a “Phoenix miracle” in a VOX EU column.

However, there is a major problem with the graph above: consumption and investment are flow variables and credit is a stock variable; comparing them is a stock-flow mismatch. This means that in the above graph the expenditures in a given period are compared to the credit that firms and households have taken out in the past (and not repaid yet). However, it is fairly intuitive that expenditures in a given period are most closely related to borrowing of that period. Together with Mike Biggs and Thomas Mayer, I discussed this problem in the context of credit-less recoveries (Biggs et al., 2010); a more concise explanation of the issue is available in our VOX EU column.

In place of the stock of credit, compare C+I to the flow of credit, that is, to the net new borrowing in each period (below I write how we approximate this). In the graph below are C+I growth and the growth of the flow of credit, which Biggs et al. (2010) call the credit impulse.

It is clear that all the anomalies that appeared in the previous graph are now gone. The recovery from the financial crisis in 2009 are equally fast for C+I as for the flow of credit.

An important insight from the credit impulse is that economies can recover while they are deleveraging. The logic is simple: Assume your income is fixed at 100, and that you have to repay 10 units of currency in one period and 5 units in the next. In both periods, you are deleveraging but your consumption in the second period will be 5 units larger than in the first: consumption growth will be positive. This occurs while credit is still shrinking (you are repaying 5) but the flow of credit has increased from -10 to -5.

How, then, did so many economist fall for a stock-flow mismatch?

My best guess, based on the presentations in the workshop, is the following. Assume that firms borrow to invest, pay interest r on existing credit, and investment is a fixed proportion of GDP. Then

(1) C(t) = (1+r) C(t-1) + I = (1+r) C(t-1) + g*Y

where C is the stock of credit, I is investment, Y is GDP, r is the interest rate, g is a constant, and t denotes discrete time. (And despite the notation, this is in discrete time – clearly, I need to get mathjax up and running.)

There is an equilibrium relationship between GDP and credit, which is

(2) C = b*Y

where b is a constant that is comprised of the deeper parameters of the economy. Hence, in equilibrium, the stock of credit and the flow of GDP are related. Drehmann and Juselius (2016, Bank of Finland WP) derive a similar equilibrium relationship (in a more elaborate model). They then assume that this equilibrium holds at all times except for a disturbance term.

However, it is clear that

(3) C(t) = b*Y(t) + e(t)

behaves fundamentally different from the dynamics in equation (1) from which the equilibrium was derived. As Brainard and Tobin (1968, AER, p.105) wrote:

“No one seriously believes that either the economy as a whole or its financial subsector is continuously in an equilibrium. Equations like those of the model described above do not hold every moment in time. […] There are, of course, identities–e.g. balance sheet or income identities–that apply out of equilibrium as well.”

For the description of financial crises and recoveries it should be clear that the equilibrium is not a good description of the economy but that the credit identity will hold in each period. Rewriting equation (1) yields

(4) C(t) - (1-r) C(t-1) = g*Y

that is, domestic, private demand is related to the net flow of credit. A complication is that the interest rates that are used for private credit is generally not observed. It could be approximated by a 10 year corporate bond rate or a similar interest rate. In the graph above, I approximated the net flow by the difference of credit as I found in previous work that this yields a reasonably good approximation as long as interest rates are not too high.

In conclusion, I hope that it has become clear that credit cycles that are distinct from the business cycle do not exist, that they should not be artificially constructed via stock-flow mismatches, and that analyzing stock-flow mismatch relations cannot lead to valid insights.

Empirics of financial contagion

Tue, 30 Jun 2015 20:40:51 +0000

Recently, I attended the SYRTO conferences in Amsterdam, which contained two days worth of presentations on financial contagion and systemic risk. The conference demonstrated how much progress has been made on these topics but also highlighted the challenges that the empirical literature on financial contagion still faces.

Financial contagion

Contagion, the spread of a financial crisis from one country or market to another, is a causal concept. This implies a number of econometric challenges that I discussed in a paper with Hashem Pesaran in 2007. Many papers in this literature start with the statement that no unique (empirical) definition of financial contagion exists. Yet, this ignores the fact that politicians have a clear definition in mind when they talk about financial contagion: a crisis in one country, e.g. Ireland, causes a crisis in another, e.g. Italy.

The question of causality is important for economic policy. If there is a causal effect from a sovereign debt crisis in Ireland to a sovereign debt crisis in Italy, then a bail-out of Ireland could stop the domino effect. If Ireland and Italy were affected by a common factor, then a bail-out of one country will have no effect on the second country. During the European sovereign debt crisis, politicians justified bail-outs with arguments of financial contagion (e.g. Josef Pröll’s quote reported in the Irish Times). Yet, as far as I am aware, there is no firm evidence in the literature for or against the existence of contagion. And this may not come as a surprise, given that establishing causality in financial data is generally no easy task.

Pearson’s rho vs. Kendall’s tau

A short post-script to my discussion of the paper by Arakelian et al. at the SYRTO conference. The paper uses Kendall’s tau to measure contagion because it assumes non-elliptic distributions. While this ignores the issue of causality described above, it might address the shortcoming of Pearson’s rho, which has been discussed by Forbes and Rigobon in their 2002 JF paper. In short, Forbes and Rigobon show that changes in correlations can be just as much caused by increases in volatility as by contagion.

Unfortunately, the same is true for Kendall’s tau. In a miniature Monte Carlo experiment, I generated data from a linear model with normally distributed errors: first with a change in the regression coefficient, which represents financial contagion, and, second, with a change in the volatility. The plots below contain estimates of Pearson’s rho and Kendall’s tau from rolling windows. In the first plot, the coefficient linking one variable (think financial market) to the other changes at period 40, which you can think of as contagion. In the second plot, the coefficient remains the same so that there is no contagion but the volatility changes. As expected, Pearson’s rho looks essentially identical in the two plots, so it cannot distinguish between the two — but neither can Kendall’s tau.

Financial networks

Another approach with a slightly different aim may offer some hope. This approach analyses the network that financial institutions span. These papers, including that presented by Marco van der Leij at the SYRTO conference, do not attempt to establish contagion in the sense outlined above. Rather they investigate the relevance of individual institutions within the financial network. If such an analysis could be extended to multiple countries then we might be able to address the question of contagion.

Forecasting with Markov switching models

Tue, 05 May 2015 10:39:50 +0000

Forecasts from Markov switching models have, by and large, disappointed. In a recent paper with Tom Boot, we investigated this issue.

Our starting point is the observation that, conditional on the states, standard Markov switching forecasts do not use all data. This cannot be optimal. Instead, our approach uses all data and weights them such that our forecast loss function is optimised. In this paper, we use the mean square forecast error (MSFE) and derive weights such that the MSFE is minimised.

This work is based on previous results for break point models, where the weights are based on the DGP and estimates of parameters are plugged in to make them operational. In that case, using a plug-in version of the weights lead to forecasts that have an MSFE similar to standard forecasts. Markov switching models allow us to address this point.

The results are quite informative.

Incorporating uncertainty around the states in the derivation of the optimal weights, improves forecasts dramatically – and this is even true asymptotically.
The common finding that Markov switching models have nice in-sample properties but do poorly out-of-sample is largely based on the selection of the forecast evaluation period. In the paper, we show that, in a symmetric three state Markow switching model, the linear model (i.e. without Markov switching) will be the optimal model for forecasting in the middle regime. In empirical forecast evaluations of Markov switching models, the typical forecasting period has been dominated by the great moderation. This is precisely the middle regime where the optimal forecast from the Markov switching model is the same as the forecast from the linear model.

The paper is on my research page or download the latest pdf file directly.

Website redesign

Mon, 04 May 2015 16:45:50 +0000

Here we go: new web site. It’s now largely written in markdown and rendered with Jekyll. The page is on GitHub. These changes allow me to include a blog with informative posts such as this one. So, dear reader, brace yourself.