“Problems in evaluating accuracy and consistency of macroeconomic forecasts”

The paper focuses on the analysis of the quality of macroeconomic forecasts, as well as on general issues of the functioning of the system of macroeconomic forecasting in the Republic of Kazakhstan. It provides a comparative analysis of errors of initial and revised macroeconomic forecasts of the real sector of economy indicators. To assess forecast accuracy, the authors use the world’s most widespread measures, such as the mean error and the mean absolute percentage error, as well as official statistical data and records. The overall forecast consistency and congruence of assumptions on the dynamics of indicators of the real sector and fiscal policy are estimated using the basic identities of the System of National Accounts. The paper also considers institutional aspects of the system of macroeconomic forecasting in the Republic of Kazakhstan. The authors conclude that revised forecasts, with the exception of monetary policy indicators, exhibit a smaller error, which can indicate a need for greater coordination of public authorities in the process of preparing forecasts, developing the system of independent assessment of their quality and improving their transparency.


INTRODUCTION
Macroeconomic forecasting and planning are the key functions of public administration.In the context of global economic instability, there is a growing need for accurate forecasts of macroeconomic indicators for making informed decisions on performing socio-economic policy of the state.At that, accuracy and consistency of macroeconomic forecasts are probably of particular importance for emerging economies that are significantly influenced by external factors and macroeconomic shocks.The situation can be exacerbated by the fact that the system of macroeconomic forecasting in developing countries is usually poorly built and has many shortcomings.One of these weaknesses is the lack of proper coordination of the activities of public authorities while producing macroeconomic forecasts.All this can lead to adoption of inaccurate parameters of the government budget and other benchmarks when implementing government socio-economic policy.It is noteworthy that the system of economic forecasting refers to the unity of methodology, organization and development of forecasts ensuring their consistency, continuity and incessancy (Nevskaya, 2015).
The system of macroeconomic forecasting in the Republic of Kazakhstan is at an early stage of its development and is being constantly modernized.After gaining independence, since the mid-1990s macroeconomic forecasts in the Republic of Kazakhstan were developed as part of five-year indicative plans for socio-economic develop-ment, which in 2004 were renamed medium-term plans.In 2009, a new system of state planning was adopted, which abolished the practice of developing medium-term plans for socio-economic development.The new system provided for the development of five-year forecasts of socio-economic development of the Republic of Kazakhstan.There were also changes in the tools and methods for developing macroeconomic forecasts.
The paper analyzes and compares the accuracy of forecasts of the major macroeconomic indicators in the real sector of economy, monetary policy and fiscal policy for the period 2009-2016.At that, the indicators of forecast assessment recognized in the global practice are used.To study the consistency of forecasts of various sectors' indicators, the basic equations and dependencies of the System of National Accounts (SNA), which represents an integrated base of macroeconomic statistics, are applied.
The conducted analysis helps to identify errors and inconsistencies in forecasts of macroeconomic indicators in the Republic of Kazakhstan and outlines possible reasons behind their emergence.In addition, the article analyzes institutional issues of the system of macroeconomic forecasting in the Republic of Kazakhstan, as well as addresses the issues of coordination of the activities of public authorities when preparing macroeconomic forecasts.

LITERATURE REVIEW
Macroeconomic forecasts are used worldwide as the basis for drawing up budgets, analyzing economic policy and taking decisions.This requires regular evaluation of the quality of forecasts, which has become a popular practice in many countries.It is generally recognized that forecasting is not an easy task.At that, the system of forecasting should also constantly improve.Analyzing errors made while forecasting is an important part of upgrading the forecasting system.For instance, discovering that an econometric model or expert estimates regularly produce inflated forecasts can be a sign that the approach used is missing some structural changes in economy (see, for example, Hawkins, 2005).
The overall evaluation of macroeconomic forecasts in theory and practice includes three main directions: analysis of accuracy, bias and efficiency (see, for example, Bratu, 2012).In many cases, there are legislative provisions for analyzing the accuracy of macroeconomic forecasts, which often serve as the basis for the formation of a country's budget.For example, the legislative requirement for the states in the US on the availability of a balanced budget by the end of the year has become an incentive for numerous studies in the sphere of forecasting fiscal policy indicators in the US.The Maastricht Agreement and the Stability and Growth Pact have significantly increased the scientific interest in fiscal forecasting and monitoring in Europe (see, for example, Leal, Perez, Tujula, & Vidal, 2007).Furthermore, closer coordination of economic policies in the European Union and related changes in legislation have led to more stringent requirements for projections that act as the basis for planning public finance.It is common for many countries to assess the quality of macroeconomic forecasts by delegating this task to national bodies auditing the execution of the government budget.For instance, the National Audit Office of Finland assesses the fairness of macroeconomic projections, the results of which are utilized to compile a report and sent to the Ministry of Finance.Copies of the report are also sent to the Audit Committee of the Finnish Parliament and to the financial inspector in the Government of Finland.The report analyzes macroeconomic forecasts of the Ministry of Finance (Finland) with the aim of revealing how accurately they characterize the economic situation today and in the planned period.In addition, it evaluates the openness and transparency of the forecasting processes and the forecasting methods employed, as well as gives an assessment of independence and impartiality of macroeconomic and fiscal projections provided by the Ministry of Finance (see, for example, NAOF, 2016).In Spain, there is an independent body as well that assesses the macroeconomic forecast, which serves as the basis for budget adoption (see, for example, AIReF, 2017).
The practice of evaluating the central banks' macroeconomic forecasts is also quite widespread.For this purpose, estimates of independent experts or organizations are typically used.For instance, the Bank of England involves the Independent Evaluation Office to assess the bank's macroeconomic forecasts (see, for example, Bank of England, 2015).
The analysis is also regularly carried out with respect to macroeconomic forecasts published by international financial organizations, such as the International Monetary Fund (IMF), the Organization for Economic Co-operation and Development (OECD), and the European Bank for Reconstruction and Development (EBRD).The standard approach of such an analysis is to study forecasts bias, their effectiveness in terms of transferring all available information, as well as their accuracy in comparison with the projections provided by other institutions and formal models (see, for example, Krkoska & Teksoz, 2005).It should be noted that overall calculation of indicators contains only one possible standard error.According to the findings of other research studies, such errors can be reasonably categorized into three groups: alpha, beta, and gamma (Kuzmin, 2016).
There is extensive literature available on measuring the accuracy of forecasts and their quality in general.There also exist several different methods that can help in assessing accuracy of projections.As Daníelsson points out (Daníelsson, 2008), one of the methods is to apply some forecast accuracy metrics, such as the mean absolute error (МАЕ) or the root mean square error (RMSE).The smaller these indicators, the better the quality of forecasts.Second method is to compare some forecast error indicators with similar indicators when utilizing a very simple forecasting method.Third method compares prediction error indicators with indicators of shocks faced by economy in the forecast period.Simionescu (2015)  With so many accuracy measures used by different researches, it is not easy to choose a superior forecasting method or technique.Clearly, there is a need for an algorithm of decision making for selecting the best prediction models by consid-ering several accuracy measures simultaneously.Mehdiyev et al. (2016) proposed a multi-criteria decision analysis approach for solving this problem.
Such measures as the mean error and the mean absolute percentage error make it possible to assess forecast accuracy of certain macroeconomic indicators, e.g.gross domestic product (GDP) or inflation.At the same time, it is often necessary to evaluate the overall consistency and congruence of all macroeconomic indicators of the real sector, the external sector, public finance and monetary statistics.Easterly (1989) proposes a system for checking the congruence between macroeconomic analysis and forecast.This system is founded on checking budget constraints for every institutional sector of economy.Using the cases of Columbia and Zambia, the author demonstrates the application of the system.The UN Manual on National Accounts (Use of macro accounts, 2002) offers numerous examples of utilizing national accounts for modeling, forecasting and ensuring the congruence of structures engaged in saving, investing and financing.Andryakov et al. (2006) address the issues of harmonization of fiscal policy indicators with overall macroeconomic forecasts and forecasts of the balance of payments.The authors develop a financial balance scheme that takes into account the basic identities of the SNA and the parameters of the macroeconomic forecast.The significance of manipulating basic identities and equations of the SNA in order to check the harmonization of the overall macroeconomic forecast is emphasized in (Banco de España, 2006).Stănică (2013) underlines the importance of using the financial programming models (especially for transition economies), since they allow checking the congruence of macroeconomic forecast for four main sectors of economy.The author presents a simple model for making macroeconomic forecast that is consistent with the adopted indicators of fiscal policy.This method for checking the congruence of a macroeconomic forecast is of special interest for countries publishing detailed forecasts of fiscal policy indicators.
The role of institutions in forming the macroeconomic environment and building a reliable sys-tem of macroeconomic forecasting should also be highlighted.For example, Kiptoo (2011) argues that the intensity and quality of key players' coordination are fairly important, since various aspects of macroeconomic policy are formed and implemented by various public authorities having different motives, limitations and their individual goals.Some institutions often use their internal institutional capacities for analysis and forecasting without considering the views of other important institutions that must be effectively involved in the processes of macroeconomic policy formation.Barker (2012) deals with the issues of coordinating various institutions when conducting macroeconomic policy.The author studies various options for institutional organization of macroeconomic management.Bogaert et al. (2006) examines independence and interaction of institutions involved in macroeconomic forecasting.The authors conclude that forecasts should be developed by an independent national institution.Bratu (2012) stresses the importance of evaluation of macroeconomic forecasts, because it is necessary to inform the public about the way in which government institutions predicted the economic perspectives of a country.
Transparency of macroeconomic forecasts, especially for developing countries, is also of high importance.For example, Mazhar (2014) conducts an empirical study of correlation between forecast transparency and macroeconomic volatility.The findings prove that forecast transparency can exert a stabilizing effect on economy only in connection with other forms of institutional transparency.

METHODOLOGY
In the present paper, the authors apply a simple approach, which suggests using the two most common indicators for assessing forecast accuracy, i.where i g is the actual growth rate for the variable in the period , i * i g is the forecast growth rate for the variable in the period i (both indicators are measured in percentage points), n is the number of observations.The mean error measures the forecast bias.A positive number indicates that, on average, the forecast value tends to be higher than the actual value (overestimation), while a negative number demonstrates that the forecast was usually below the actual value (underestimation).
The mean absolute percentage error makes it possible to evaluate forecast accuracy, since it measures the average distance between the forecast and actual values.For both of the above-mentioned indicators, a smaller final number corresponds to better forecast quality.In addition, the root mean square error, which makes larger errors more important, is often utilized as a forecast accuracy indicator.Most studies for the United States and the United Kingdom have shown that the conclusions are insensitive to the choice of measure (see, for example, Ewing et al., 2005).
In the paper, the authors also use simple identities and equations of the SNA to check budget constraints for public administration and other sectors using forecast indicators of fiscal policy and other macroeconomic indicators.Specifically, a well-known macroeconomic identity of the equality of the external balance and the balance between domestic savings and investments is applied: where S is savings, I is gross fixed capital forma- tion, CAB is current account balance.

RESULTS
Currently, macroeconomic forecasts in the Republic of Kazakhstan are developed on the basis of the documents of the state planning system adopted in 2009 (Edict of the President of the Republic of Kazakhstan of June 18, 2009 No. 827).In particular, the document "The Forecast of the Socio-Economic Development of the Republic of Kazakhstan" annually approves forecasts of macroeconomic indicators for a five-year period on a rolling basis.Along with forecasting macroeconomic indicators for a five-year period, this document includes a forecast of budget parameters for a three-year period.The rules for preparing the forecast of socio-economic development (Approved by the resolution of the Government of the Republic of Kazakhstan of August 2009 No. 1251) sanctioned a specific list of indicators for the forecast of socio-economic development.This list comprises 46 indicators of the real sector, the external sector, monetary policy and social sphere.Moreover, it also encompasses 38 indicators of budget parameters for a three-year period.The rules also dictate the interaction procedure of government authorities while producing a forecast of socio-economic development.
The development of the forecast is conducted in two stages.At the first stage, during the first half of the year, public authorities submit their industry forecasts to a central authorized planning institution that releases the initial version of scenario forecasts of the main macroeconomic indicators.This version is directed to the National Bank to prepare forecast indicators of monetary policy and the balance of payments.At the second stage, macroeconomic indicators are revised and, as a rule, the Government approves the initial macroeconomic forecast of the main socio-economic indicators in August of the year preceding the forecast one.In March of the following year, the forecast indicators of the current year are usually revised.Let us look at how errors in forecasts of growth of macroeconomic indicators changed after the revision.To do this, the initial and revised forecasts for 2009-2016 are compared with the actual reported data.For instance, the initial projections for 2009 were approved in August 2008 and revised in March 2009.Until 2009, macroeconomic forecasts were included in indicative plans for socio-economic development, and the method for producing them was significantly different from that exercised today.For this reason, the authors chose to analyze the period 2009-2016.Table 1 shows the values of er-rors in percentage points for the initial and revised forecasts of the real sector of economy.The mean absolute percentage error for all indicators of the real sector in the revised forecast was lower, and for some indicators to a quite significant extent.This is the case of nominal and real GDP, industry, agriculture, communications and trade.Thus, accuracy of the revised projections turned out to be higher and the revision had a positive effect on the forecasts.At the same time, the mean errors in the forecast of agriculture, construction, transport, communications and trade increased after the revision, especially for agriculture and construction.The forecasts for these sectors were biased towards underestimation of actual values.Hence, when revising the forecasts for these industries, they were incorrectly revised downwards, which resulted in an increase in the mean error.
Figure 1 illustrates the initial and revised forecasts of real GDP growth, as well as the actual reported data for 2009-2016.It is obvious that the initial forecasts failed to predict a slowdown in economic growth rates in 2009, 2012 and 2014.There was also an error in forecasting a sharp rise in the economy in 2010.The revised forecasts were generally much closer to the actual values of real GDP growth.Note: * Percentage points, а) mean error; b) mean absolute percentage error; c) average growth rate during the period.
Table 2 shows errors in the forecasts of monetary policy indicators.The National Bank of the Republic of Kazakhstan provides the authorized forecasting institution with these indicators.Interestingly, in the course of revising, the mean error of the forecast increased in all indicators except inflation.Similarly, the values of the mean absolute percentage error of the revised forecast also raised in all indicators except the indicator of credits to the economy.Thus, in general, revising initial data did not improve forecast accuracy of monetary policy indicators, but on the contrary, worsened them.Note: * Percentage points, а) mean error; b) mean absolute percentage error; c) average growth rate during the period.
Table 3 demonstrates errors in the forecasts of the growth in revenues and expenditures of the government budget.The mean absolute percentage error of the revised forecast of revenues is significantly lower than the error indicated by the initial forecast.But still, the magnitude of this error remains considerable even after the revision.The average value of the absolute percentage error of the forecast exceeds the average value of the annual actual growth in revenues, which signifies that there are significant reserves of improving the accuracy of forecasting this indicator.The positive increase in the mean error of the revised forecast of revenues indicates that the forecast overestimated the amount of revenues received.As for the budget expenditures, both forecast errors decreased significantly in the course of the revision.This is mainly because budget expenditures are subject to much stricter control if compared to revenues.This is facilitated by the availability of a guaranteed transfer from the National Fund of the Republic of Kazakhstan, which in recent years has been financing about 20-30% of the total government budget expenditures of the country.
Table 4 shows the values of correlation coefficients of the mean errors in the forecasts of several macroeconomic indicators for the period 2002-2016.
It is noticeable that the errors in the forecasts of the global oil prices display a strong positive correlation with the errors in the forecasts of real and nominal GDP, manufacturing, and budget revenues and expenditures.Errors in oil prices forecasts exert a direct effect on errors in forecasting the GDP deflator, and accordingly, the nominal GDP growth.Since global oil prices influence the volume of oil production in Kazakhstan, errors in the forecasts of the global oil price are positively correlated with errors in the forecasts of growth in the industrial production and GDP as a whole.Budget revenues of Kazakhstan demonstrate a positive dependence on changes in oil prices, which explains the close correlation between these indicators.Growing budget revenues driven by an increase in oil prices also cause a rise in budget expenditures.It is also noteworthy that there is a high positive correlation coefficient (0.86) between the errors in money supply growth (M3) and inflation, which can be justified from the theoretical perspective.
To check the balance of the macroeconomic forecast, the authors address the forecast of socio-economic development of the Republic of Kazakhstan for 2018-2022 approved at the meeting of the Government of the Republic of Kazakhstan on August 29, 2017.For simplicity's sake, let us consider some forecast parameters of the most important indicators for 2018 given in Table 5.

Table 4. Correlation coefficients of forecast errors for 2002-2016
Real GDP (1)  The short list of the indicators shown in Table 5 is sufficient to draw certain conclusions about the balance of the forecast.Real GDP growth in 2018 is expected to be 3.1% due to, among other things, an increase in final consumption by 3.2% and gross accumulation by 4.7%.It is projected that gross saving in the economy will be insufficient and the deficit will be financed by foreign borrowing.Negative current account balance in 2018 is forecast to reach 8.7 billion U.S. dollars, or approximately 3 trillion tenge.The government budget deficit in 2018 is planned to be 0.6 trillion tenge, and saving in the National Fund will decrease by the same amount.Thus, the public administration sector is assumed to act as a net borrower.
According to the forecast, gross accumulation in Kazakhstan in 2018 will exceed gross saving by 3 trillion tenge.If one knows the projected rate of real gross accumulation growth and uses the GDP deflator, it is possible to obtain the forecast value of gross accumulation in nominal terms in 2018 which accounts for 12,9 trillion tenge.Using data on gross accumulation, the balance of external transactions and the mean value of statistical discrepancies, the authors obtain the projected gross saving in the economy in 2018 that amounts to 9,9 trillion tenge.This is the sum of national saving that is necessary to achieve the approved forecast parameters of the economy for 2018.Now, using the analysis of the previous years and the parameters of the adopted forecast, let us consider the feasibility of the projected balance of net lending/borrowing in the economy and institutional sectors for 2018.ed period.In this regard, there arises a question about the feasibility of such assumptions when preparing a macroeconomic forecast.

DISCUSSION
The findings of the analysis conducted demonstrate that the revision of the macroeconomic forecasts has led to a decrease in the magnitude of forecast errors for the real economy and fiscal policy indicators.At the same time, errors in the forecasts of monetary policy indicators did not decrease, but on the contrary, increased after the revision.This fact signifies that there are problems with coordinating the preparation of forecasts, for which the RK Government and the National Bank are responsible.According to the existing practice, the National Bank provides the Government with its forecasts of monetary statistics and the balance of payments indicators.At that, experts of the Ministry of Economy and the National Bank do not hold joint detailed discussions of the forecasts.
There is no formalized working group in charge.It is quite possible that when forecasting its indicators, the National Bank applies its models that are different from those of the Ministry of Economy.
Assumptions used in the forecasts can also differ.
Preparing a macroeconomic forecast requires the work of various institutions, such as the Ministry of National Economy, the National Bank, the Ministry of Finance and other public authorities, to be coordinated.At the moment, the situation is that each institution produces its own forecast, but the degree to which all forecasts are corrected and checked for consistency is inadequate.For example, the current account of the balance of payments is forecast by the National Bank and simply included in the general macroeconomic forecast.The same is true for fiscal policy.However, it is also necessary to check the parameters of fiscal policy and the external sector for compatibility with the indicators of the real sector of economy in the framework of the SNA accounting structure.Unfortunately, as shown by the analysis, this work has not yet been carried out properly.The forecast parameters of fiscal policy, the real sector and the balance of payments for 2018-2022 suggest such magnitudes of savings and investments that are achievable only if economic agents change their behavior significantly, which seems improbable.There are numerous similar examples of checking the forecast for consistency.The analysis performed serves only as an illustration; it shows the questions that may arise when examining the consistency of the macroeconomic forecast and looks at the role of the SNA in this process.According to Andryakov et al. (2006), in order to carry out systematic work on ensuring the consistency of the macroeconomic forecast, it is expedient to develop a projected financial balance in Kazakhstan.Moreover, it is necessary to further analyze forecast errors using other methods and indicators and to administer various statistical tests, which is in some cases impossible due to a short time series of indicators.
The analysis also revealed that one of the sources of reducing errors in forecasting macroeconomic indicators in Kazakhstan is enhancing the accuracy of forecasts of changes in the world's oil price.However, when planning budget expenditures, a cautious approach usually prevails, which often does not allow forecasting a significant increase in oil prices, since this can entail an unjustified increase in government budget expenditures.
Furthermore, the authors believe that from the institutional perspective the system of macroeconomic forecasting in the Republic of Kazakhstan is currently characterized by a number of weaknesses that influence the quality of the forecasting work.
First, the crucial element of the national forecasting system is institutions that analyze macroeconomic projections in terms of their fairness, reliability, coherence and independence.Private analytical and research centers existing in the country do not express any interest in analyzing official macroeconomic forecasts on a system basis.In many countries, the National Audit Chambers prepare a report containing the analysis of macroeconomic forecasts, which are the basis for budget formation.In the Republic of Kazakhstan, one of the functions of the Accounts Committee for Control over Execution of the Republican Budget is to audit the efficiency of planning and execution of the Republican budget in accordance with the principles of the budget system of the Republic of Kazakhstan and to compile a report on execution of the Republican budget for the fiscal year.However, the reports of the Accounts Committee do not present a detailed analysis of errors in macroeconomic forecasts, but focuses on the effectiveness of budget implementation.This is possibly due to the fact that according to the documents of the current System of State Planning in the Republic of Kazakhstan, monitoring, evaluation and control of the Forecast of socio-economic development are not implemented (Edict of the President of the Republic of Kazakhstan of June 18, 2009 No. 827, item 3.4.23).This provision undoubtedly impedes the development of the system for analyzing the fairness of macroeconomic projections to boost their quality.
Second, public attention to macroeconomic forecasts would be more scrupulous, if public authorities fully observe the principles of openness and transparency of such forecasts.For example, on the website of the central authorized body for forecasting there is currently no information about methods and tools for forecasting and describing the macroeconomic models applied in the work.This does not encourage the development of the system for assessing macroeconomic forecasts and improving their quality.

CONCLUSION
The performed analysis of accuracy and consistency of macroeconomic projections with the use of the world's most widespread methodologies enabled us to identify a number of problems in organization of the forecasting work in the Republic of Kazakhstan.In general, revised projection are more accurate.At the same time, forecast errors in indicators of monetary policy, when revising, have increased.The possible reason behind it is insufficient coordination between the Ministry of the National Economy, the Ministry of Finance and the National Bank of the Republic of Kazakhstan while preparing macroeconomic projections.
The analysis of the balance of the macroeconomic forecast showed that there were logical errors in the forecasts and assumptions.This may be due to the absence of a single logical model taking into account the assumptions of fiscal policy, the external sector, financial and real sectors of economy, as well as to the lack of joint work of the interested public authorities.In addition, when producing macroeconomic forecasts, it is required to utilize all the SNA accounts developed by the national statistical office, which will ensure the logic and consistency of the forecast indicators.It appears expedient to pay more attention to the analysis of the SNA institutional sectors' activities and the use of this information in the development of macroeconomic forecasts.A promising avenue is the development of a financial balance, which will improve the accuracy of macroeconomic forecasts.It is also necessary to introduce new methods for assessing the quality of forecasts taking into account the latest developments in this field.
There are also problems of an institutional nature in organization of work on macroeconomic forecasting in the Republic of Kazakhstan.It is essential to facilitate coordination of the activities of public authorities in the development of forecasts, increase the transparency of forecasting processes and develop a system of independent assessments of the quality of macroeconomic forecasts.
e. the mean error (ME) and the mean absolute percentage error (MAPE) (see, for example, Ewing, Gruen, & Hawkins, 2005).For growth rate, these indicators are calculated as follows:

Figure 1 .
Figure 1.Forecast and actual values of real GDP growth, % They proposed an alternative index, called Resistant MAPE or R-MAPE based on the calculation of the Huber M-estimator, which allows to overcome this limitation.Chen et al. (2017) suggested a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which has the best features of other different measures.The results show, that the new measure performs as well as or better than other common measures.Still, despite all the alternatives and improvements, MAPE remains one of the most popular measures of forecast accuracy.

Table 1 .
Errors in the forecast of the real sector of economy, pp.*Source: Calculated by the author using the data of the Ministry of the National Economy of the Republic of Kazakhstan (the MNE of the RK).
Note: * Percentage points, a) mean error; b) mean absolute percentage error.

Table 2 .
Errors in the forecast of monetary statistics indicators, pp.*Source: Calculated by the author using the data of the National Bank of the Republic of Kazakhstan.

Table 3 .
Errors in the forecast of the government budget indicators, pp.*Source: Calculated by the author using the data of the MNE of the RK.

Table 5 .
Forecast indicators for 2018Source: The MNE of the RK.

Table 6 .
Indicators of net lending/borrowing, trillion tengeSource: Statistics Committee of the MNE of the RK, calculated by the author.