使用Google趋势跟踪实时经济活动.docx

《使用Google趋势跟踪实时经济活动.docx》由会员分享，可在线阅读，更多相关《使用Google趋势跟踪实时经济活动.docx（59页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、Table of contentsTracking activity in real time with Google Trends61. Introduction and Summary62. The COVID-19 crisis called for the use of high-frequency indicators73. Exploiting the full potential of Google Trends94. A neural panel model of GDP growth114.1. From quarterly GDP growth to a weekly tr

2、acker: a bridge model of GDP growth124.2. A non-linear algorithm124.3. To pool or not to pool: a panel nowcasting model for 46 countries145. How well can the OECD Weekly Tracker nowcast the economy?156. Model insights: a dive into the black box206.1. Shapley values: explaining machine learning with

3、game theory206.2. A dive into the model inner workings216.3. From Shapley values to sectoral insights237. The OECD Weekly Tracker237.1. The COVID-19 crisis: a week-by-week analysis247.2. Latest insights from the Weekly Tracker: a stalling recovery below 2019 levels307.3. Consumption volume remains s

4、ubdued while its composition has shifted31References35Annex A. Data pre-processing and data issues40Additional details44Annex B. Additional results47TablesTable 1. Standard indicators were outpaced by the crisis8Forecast performance18FiguresQueries in Google Trends: beyond keyword searches11Figure 1

5、. Nowcasting GDP growth with Google trends (M-1 forecast)15Trackers predictions for Q2 202017Figure 2. The OECD Weekly Tracker and Google Mobility19Most important variables and their contributions to predictions22Figure 3. Partial dependence plots22The neural network can be thought of as an alternat

6、ive to using dynamic factors or principal components as it reduces the dimensionality to a number of intermediate components in the middle layer before making a prediction. The multi-layer structure helps avoid overfitting. As opposed to PCA, it allows for capturing non-linear relationships. Variabl

7、es are pre-processed using normalisation. The main caveat of neural network is their black-box nature, which is addressed using machine learning interpretability techniques in section 6.24. While a vast research has focused on the inclusion of the GT indicator as an explanatory variable in conventio

8、nal autoregressive models, papers have used factor models of multiple GT categories (Vosen and Schmidt, 2011(31); Balakrishnan and Dixit, 2013(32). Other papers used linear shrinkage methods such as Ridge (Ferrara and Simoni, 201M) or Spike-and-Slab (Scott and Varian, 2014(33). Fewer papers have use

9、d non-linear methods: Burdeau and Kinzler (2017(34) experimented with Support Vector Machines (SVMs) and boosting and reported better results from non-linear approaches.25. Neural networks have had attracted little attention from macroeconomists compared to tree-based methods such as Random Forests,

10、 mostly because of the small size of macroeconomic data. By providing variables comparable across countries for a large number of countries and at a high frequency, Google Trends creates opportunities for using a wider array of algorithms and econometric methods, be it for prediction or policy analy

11、sis.Box 1, Training the neural networkAdditional details on the training on the neural network algorithm.Architecture and technical details. The neural network algorithm used in this paper is a standard multi-layer perceptron implemented with most of the default parameters in Python statistical soft

12、ware scikit-learn. It includes two hidden layers of respectively 100 and 20 neurons. Each neuron uses a rehJ activation function. The activation function takes a weighted sum of input signals (the variables values) and yields the linear combination of inputs provided it is higher than a given thresh

13、old. The weights and threshold are optimised using stochastic gradient descent.Standard Scaler. It has become an industry standard to scale the variable values prior to fitting the algorithm. Early experiments proved that Quantile Scalers performed badly especially when it comes to predicting extrem

14、e values in times of crises. Standard Scalers do not treat extreme values as outliers and thus allow better performance around downturns.Ensemble. Neural networks are notoriously sensitive to the initial random parameters. The choice of random parameters proved to have a strong effect on the results

15、. In order to curb the effect of that randomness, the tracker uses an ensemble of five neural networks initialised with random parameters, whose predictions are averaged over.Hyper-parameter optimisation. The use of gridsearch for hyperparameter optimisation was purposely avoided. Even when perfomed

16、 on a training set prior to the forecast simulations, gridsearch can lead to overfitting the validation set1: users may experiment with many parameter grids and simulation settings leading to biased simulation results. In order to prevent that issue, parameter optimisation was performed through a si

17、mple trial-and-error process aiming at finding a good level of fit with reasonable performance rather than at maximising goodness-of-fit. An additional guarantee against overfitting the validation set is provided by the generality of the model over a large number of countries, which reduces the like

18、liness of bias in performance measurement caused by ad hoc hyper-parameter selection.25.3. To pool or not to pool: a panel nowcasting model for 46 countriesThe panel nature of the data raises the question of whether to pool countries together or run country-wise models. Country-specific models seem

19、more intuitive as various levels of internet penetration, habits, culture, demography and institutions could explain possibly large differences in country-specific elasticities, but would involve many more variables (248) than observations (61). Conversely, pooling countries together increases the s

20、ample size and thus estimation accuracy The alternative between the two options can thus be thought of as bias-variance trade-off: introducing some bias by using average elasticities rather than country-specific ones allows to substantially reduce the variance of the estimator and increases overall

21、predictive accuracy.26. The panel nature of the data raises the question of whether to pool countries together or run country-wise models. Country-specific models seem more intuitive as various levels of internet penetration, habits, culture, demography and institutions could explain possibly large

22、differences in country-specific elasticities, but would involve many more variables (248) than observations (61). Conversely, pooling countries together increases the sample size and thus estimation accuracy The alternative between the two options can thus be thought of as bias-variance trade-off: i

23、ntroducing some bias by using average elasticities rather than country-specific ones allows to substantially reduce the variance of the estimator and increases overall predictive accuracy.27. This paper uses a neural panel model, which exploits a large sample of observations from 46 countries while

24、capturing cross-country heterogeneity. Neural networks are able to handle heterogeneity in the data as long as country dummies are included. A neural network whose architecture incudes an intermediate layer with enough neurons (in our case, 100) can flexibly model each possible interaction between G

25、oogle Trends variables and country dummies. Each neuron takes as input signals from Google Trends variables and country dummies, and returns a non-linear function of the weighted sum of these inputs. As a result, the model can capture country-specific elasticities. As a consequence, neural networks

26、with few but large layers may be preferred.28. The next section provides an out-of-sample assessment of the quarterly model. Predictions from quarterly SVIs annual growth rates are compared against official GDP year-on-year growth rates at each quarter. This exercise will provide a notion of the mod

27、el performance, while the weekly predictions cannot be evaluated as there is no official record of weekly GDP.Box 2. Sparse or dense?A useful consideration prior to choosing a model type is whether the data generating process is sparse or dense. Sparse processes involves only a few variables relevan

28、t for predicting the outcome while most others are noise. Dense processes relates to cases where a large number of variables will have small but significant impact on the y variable. Sparse models will thus try and select the most important variables, as would Random Forests or Lasso. Dense models s

29、uch as Dynamic Factor Models or Ridge will include a larger number of variables while constraining the total number of parameters in order to prevent overfitting.The dense-or-sparse question is pivotal in economic predictive modelling. In a recent paper, Giannone, Lenza and Prim iceri (2018(35) shed

30、 light on the illusion of sparsity1 in macro, whereby a small number of variables are key while a large number of variables may remain significant. The absence of a clearcut answer would explain the long-standing debate between proponents of DFMs against Vector Auto- Regressive models (VARs).Experim

31、ents with Elastic Net (Zou and Hastie, 2005(36) in a spirit close to (Giannone, Lenza and Primiceri, 2018(35) showed dense patterns in the Google Trends data. This finding suggest that neural networks are better suited. In addition, Random Forests and XGBoost regressors (which are both sparse algori

32、thms), both with standard scikit-learn parameters, underperformed the neural network used in this paper.5. How well can the OECD Weekly Tracker nowcast the economy?29. The predictive performance of the model underlying the Weekly Tracker is assessed using pseudo-real time simulations on the quarterl

33、y series. Pseudo-real time simulations emulate the conditions a forecaster would have faced at each time period, by looping on time and using only past or present data (except for the fact that revised series are used, not vintages). The simulations suggest that the Google Trends Tracker provides re

34、levant leading information on GDP growth, economic crises and business cycles in almost all 46 countries in the sample. With one exception, it accurately signals the COVID-19 downturn in all countries.30. The dependent variable to be explained is GDP growth at M-1, which is GDP growth one month befo

35、re its official release. Simulations are an out-of-sample exercise and emulate a forecast made at the end of the last month of the current quarter. A Q2 GDP growth forecast will thus use Google Trends data up until June and the algorithm will be trained on Google Trends and GDP data up to Q1. At eac

36、h iteration, the algorithm training parameters are optimised using early stopping and a 10% hold-out sample from the training set.31. The quarterly model of annual GDP growth based on Google Trends performs well in out-of-sample nowcast simulations (Table 2). On average across 46 countries, the quar

37、terly model based on Google Trends has a Root Mean Squared Error (RMSE) that is 17% lower than an autoregressive model that just uses lags of year-on-year GDP growth. For the G7 countries, the improvement in the RMSE relative to the use of an autoregressive model is even larger, at 26%. It captures

38、a sizeable share of business cycle variations, including around the Global Financial Crisis (when the available data for training the algorithm was much smaller) and the euro area sovereign debt crisis (Figure 2, additional results in Annex C). Its RMSE is on average 8% lower than an autoregressive

39、model in 2008-10 and 41% lower in 2020. The timing of the downturn and subsequent rebound is well captured by the model, although the full magnitude of the negative shock in 2020 Q2 is typically under-estimated, given its unprecedented scale. The mean average error in predicting year-on-year GDP gro

40、wth in the first (second) quarter was 2.42 (3.86) percentage points, compared with actual falls in GDP for the median country of 0.12% (10.4%). The Tracker thus provides a useful tool for real-time narrative analysis on a weekly basis, although it does not on average outperform models based on more

41、standard variables, once these are eventually released.Figure 2. Nowcasting GDP growth with Google trends (M-1 forecast)Nowcasting GDP in growth rate compared to the same quarter of the previous year, seasonally adjusted, from 2006 Q3 to 2020 Q2.Panel A. United StatesFigure 2. Nowcasting GDP growth

42、with Google trends (M-1 forecast) (contd.)Panel B. United KingdomPanel C. Spain200920112013201520172019Panel D. Italy200920112013201520172019Panel E. Germany200920112013201520172019Figure 2. Nowcasting GDP growth with Google trends (M-1 forecast) (contd.)Panel F. France-20%-20%2009201120132015201720

43、19Note: The quarterly model is applied to 3-month moving averages of Google Trends series and yields monthly estimates that can be compared to quarterly GDP growth for February (Q1), May (Q2), August (Q3) and November (Q4). Shaded areas in 2011 and 2016 are years when the tracker is unavailable due

44、to structural breaks in Google Trends data preventing the calculation of year-on-year growth rates in search intensities. Simulations are based on the latest GDP data, not the real-time vintages. For each quarter, the forecast is made 5 days after the end of the month, so 3-7 weeks before the GDP is

45、 published.Source: Google Trends; OECD Quarterly National Accounts; and OECD calculations.33. Figure 3 provides a closer focus on the COVID-19 crisis. The Weekly Tracker captures on average 60% of the fall observed in the second quarter. The tracker captures the sign of GDP growth accurately for all

46、 countries but Korea. The country ranking is relatively accurate. The extreme nature of the COVID- 19 shock makes it challenging to predict the resulting growth dynamics based on history. This good overall performance reflects the ability of Google Trends to capture in a qualitative way the rapid ch

47、ange of economic situation during the second quarter and to translate this into a meaningful number.Figure 3. Trackers predictions for Q2 2020Source: Google Trends, OECD Quarterly National Accounts and OECD calculations.Table 2. Forecast performance2006-20202006-20202008-20102020Unadjusted Standardi

48、sed Relative to Unadjusted Standardised Relative Unadjusted Standardised RelativePoland US Japan Belgium South Africa Slovak Rep. Hungary Netherlands Lithuania Chile Mexico Bulgaria Germany Norway Canada Denmark Spain Austria Brazil Slovenia UK PortugalPoland US Japan Belgium South Africa Slovak Rep

49、. Hungary Netherlands Lithuania Chile Mexico Bulgaria Germany Norway Canada Denmark Spain Austria Brazil Slovenia UK Portugal1637975876052 111132.261.353.362.052.261.801.771.201.521.782.191.672.142.422.310.920.620.700.590.930.700.600.560.600.600.600.640.580.670.690.550.590.620.620.632.172.260.650.67AR(4)0.390.490.560.570.611.920.61