Incrementality study of the Canada Small Business Financing Program

Table of Contents

• Executive Summary
• Acknowledgements
  1. Introduction
  2. Literature Review
  3. Data
  4. Analytical framework
  5. Variables and descriptive statistics
  6. Results
  7. Conclusion
• References
• Appendix

Executive Summary

The Canada Small Business Financing Program (CSBFP) is a statutory loan loss-sharing program administered by Innovation, Science and Economic Development Canada (ISED) that helps Canadian small businesses obtain access to financing that would not otherwise be available (full incrementality) or would only be available under less favourable terms (partial incrementality).

The purpose of this study is to empirically estimate the level of full incrementality of the CSBFP by analyzing data from Statistics Canada's Survey on Financing and Growth of Small and Medium Enterprises, 2020 and the Linkable File Environment that comprises various business administrative data. Partial incrementality is not considered in this study due primarily to data limitations; however, it will be examined in the upcoming lender study.

The main finding of the paper is that the CSBFP is 75.8% fully incremental. In other words, 75.8% of debt financing requests from CSBFP borrowers would have been denied without the existence of the CSBFP.

Acknowledgements

The author would like to thank Lyming Huang, Jiong Tu and Ako Viou Bahun-Wilson from Innovation, Science and Economic Development Canada and Bassirou Gueye, Amélie Lafrance-Cooke and Manassé Drabo from Statistics Canada for their helpful comments and suggestions.

1. Introduction

The Canada Small Business Financing Program (CSBFP) is a statutory program designed to facilitate access to financing for Canadian small businesses. Established in 1999, it is governed by the Canada Small Business Financing Act (CSBFA) and administered by Innovation, Science and Economic Development Canada (ISED). Small and medium-sized enterprises (SMEs)^{Footnote 1} play an important role in the Canadian economy. Around 99.8% of employer businesses are SMEs and they contributed to 50% of the gross domestic product in Canada's private sector in 2019.^{Footnote 2} In 2019–2020, businesses received 5,746 CSBFP loans for a total value of $1.3 billion.

Under the CSBFP, the Government of Canada shares the risk of lending with financial institutions by covering 85% of eligible losses on defaulted loans registered under the program, which is designed to increase the overall level of financing provided to SMEs.

Under the program^{Footnote 3}, eligible enterprises can finance a maximum loan amount of $1.15 million, which includes the following: $1 million for term loans, of which a maximum of $500,000 can be used for equipment and leasehold improvement loans and a maximum of $150,000 can be used for intangible assets and start-up costs; plus a maximum of $150,000 for lines of credit for working capital costs.

Small businesses or start-ups operating in Canada in all sectors (except agriculture) with a gross annual revenue of $10 million or less are eligible under the program. Participating financial institutions—which include chartered banks, credit unions and caisses populaires—deliver the program and are solely responsible for approving the loan and disbursing the funds.

As legislated by the CSBFA, the program operates on a statutory five-year review cycle. The purpose of the review is to examine the extent to which the program meets its objectives, outline the rationale and relevance of the program in meeting the financing needs of SMEs, and suggest possible program improvements. The next report will cover the fifth such review period from April 1, 2019, to March 31, 2024.

One way of assessing the program's relevance is to determine whether borrowers would have been approved in the absence of the program. In other words, the program will have met its objective if those loans would have been rejected without it. This notion is called incrementality or additionality. This study only measures full incrementality. In theory, if the program fully accomplishes its role to remedy market failures in financing, the incrementality rate of the CSBFP would be 100%.

Partial incrementality—loans that would have been made but under less favourable lending conditions such as higher interest rates or lower loan amounts—is not considered here mainly because of data limitations in assessing partial incrementality.

The goal of this study is to empirically estimate the full incrementality rate of the CSBFP using data from the Survey on Financing and Growth of Small and Medium Enterprises, 2020 and Statistics Canada's Linkable Environment File.

The paper is divided as follows:

• Section 2 presents a short literature review of previous studies on empirically estimating the incrementality rate of the CSBFP
• The data sources used for this study are presented in section 3
• Section 4 presents in detail the empirical strategy and analytical framework used to estimate the incrementality rate of the CSBFP
• Descriptive statistics and the definitions of variables are provided in section 5
• The results and main findings are discussed in section 6
• Finally, section 7 offers some conclusions

2. Literature review

Over the past 20 years, several studies have used similar methodology to empirically estimate the incrementality rate of the CSBFP. One advantage of having used similar methodology is that the results are, broadly, comparable over time. The table below provides an overview^{Footnote 4} of the results of those studies and the data sources used. The methodology used by researchers applies a credit scoring model, that is, a model that helps determine whether a business loan application should be approved or denied given certain parameters and factors.

The methodology used in previous studies consisted of a two-step approach: in the first step, parameters are estimated using a logit regression applied to firms that were non-CSBFP borrowers. The model estimates a firm's probability of having its debt financing request approved or rejected. The second step consists in applying the model's estimates to CSBFP borrowers to calculate each firm's probability of being approved or rejected. The proportion of firms that the model identifies as being rejected is the measure of incrementality.

These studies have consistently found the CSBFP to be incremental with an incrementality rate varying from 66% to 85% (Table 2-1).

More recently, Legendre (2022) empirically estimated the incrementality rate of the CSBFP using a different modelling approach. His results differ greatly from, and are not comparable with, those reported in Table 2-1, as explained below. The author obtained an incrementality rate of between 5% to 7%.

The two-stage loan adjudication approach used by Legendre (2022)^{Footnote 5} includes financial variables from administrative data that tend to be missing for several businesses (for example, total interest expense). By contrast, Seens and Song (2015) and Rivard (2018) use mostly financial variables in their model that are mandatory for businesses to fill in the T2 Corporation Income Tax Return (see for instance the General Index of Financial Information), which helps to reduce the loss of observations due to missing values.

To the extent that including such variables would tend to result in dropping observations, this may limit the representativeness of the results; in particular, if many firms had missing values of total interest expense and were dropped from the analysis, the estimates would be representative only of the sample used in the analysis rather than of all CSBFP borrowers.

However, Legendre (2022) does not indicate exactly how many borrowers and non-borrowers were lost in his models due to missing values, so it is difficult to evaluate how much this contributed to the substantial differences in estimated incrementality. Differences would also arise from the first-stage models containing different sets of covariates. In light of these differences in both model and sample, the estimated rate of incrementality in Legendre (2022) is not comparable with that of previous incrementality analyses.

3. Data

The data used for this analysis are from the Survey on Financing and Growth of Small and Medium Enterprises (SFGSME) 2020 as well as administrative data from Statistics Canada's Linkable File Environment. This includes the Canada Revenue Agency's General Index of Financial Information Incorporated Businesses (GIFI-T2) and Payroll Deductions Account (PD7) data for years 2016 to 2020. The survey is designed to collect information on SMEs and their attempt to obtain new financing such as debt financing, government financing, lease financing, trade credit financing and equity financing. It also collects information on demographic ownership characteristics and the business's involvement in innovation and intellectual property.

The SFGSME sample was comprised of active enterprises selected from Statistics Canada's Business Register. Non-employers, businesses with 500 employees or more and with less than $30,000 in gross revenue are excluded. Also excluded are not-for profit businesses, joint ventures, government agencies and businesses in specific industries.^{Footnote 6} The base sample size was 19,283 businesses representing a population of 788,690 SMEs and the survey response rate was 56%. The data collection occurred between April 2021 and August 2021.

A sample of 1,334 businesses that were recipients of the CSBFP was included with the sample for the main population, representing a population of 2,459 SMEs under the CSBFP. The response rate for this sub-population was around 50%.

For the purpose of this analysis, the sample includes only the SMEs that had requested debt financing in 2020, more specifically, a non-residential mortgage or a term loan. Under the CSBFP, enterprises can borrow for land or buildings used for commercial purposes, equipment or leasehold improvements.^{Footnote 7}

The total number of SMEs included in the final sample^{Footnote 8} was less than 2,000.^{Footnote 9} The sample consisted of less than 600 CSBFP borrowers.

4. Analytical framework

The analytical framework used in this report consists of a two-step approach. In the first step, we estimate the probability that a firm's application for a debt financing request will be approved, conditionally on explanatory variables related to a firm's characteristics and the owner's or primary decision maker's characteristics. At this stage, only SMEs that requested debt financing (either non-residential mortgage or term loan) are included and CSBFP borrowers are excluded. The first step mimics a financial institution's decision to approve or deny a business debt financing request. The base equation that is estimated is given by

$y_i*=\beta x_i+{\epsilon }_i$                         (4.1)

where $i_{}^{}$ = 1, ..., N and $y_i^{*}$ denote a latent dependent variable that represents the propensity of a firm's loan application to be approved or denied. In the equation, ${\epsilon }_i$ denotes the error term. As the latent variable is not directly observed in the data, we will use another variable, denoted by $y_i^{}$ , which is 1 if the application was approved and 0, otherwise. This binary variable indicates in which category $y_i^{*}$  falls:

$y_i=\{\begin{array}{c}1\mathrm{if}{y}_i^{*}>0;\\ 0\mathrm{if}{y}_i^{*}\le 0.\end{array}$

A logistic model is used to estimate the probability that the firm's debt financing request is approved:

$P\left(y_i={\mathrm{1|}x}_i\right)=\frac{\exp \left(\beta x_i\right)}{1+\exp \left(\beta x_i\right)}$

The non-CSBFP borrowers are classified as being "approved" or "rejected" using a specific value for the estimated probability as a threshold: observations with estimated probability below this threshold would be classified as "rejected" (and will be attributed the value 0) and observations with estimated probability above the threshold would be classified as "approved" (and will be attributed the value 1).

Correctly or incorrectly classifying observations leads to the notion of a particular table, called the confusion matrix, used widely in predictive analysis (Table 4-1).^{Footnote 10} For example, in our case, a business with an approved debt financing request that would be classified as rejected would be called a false negative. The actual value is positive (1), but the predicted value is negative (0), assuming that being approved is 1 and being rejected is 0.

Two underlying concepts are also related to the confusion matrix: sensitivity and specificity. Sensitivity is the proportion of those that are classified as positive among those that are actually positive. Sensitivity is defined as follows:

In the case of this analysis, sensitivity corresponds to the proportion of businesses that obtain a predicted value of "approved" among the businesses that were actually approved. Specificity corresponds to the proportion of those that are classified as negative among those that were actually negative. Specificity is defined as follows:

The proportion of businesses that obtain a predicted value of "rejected" among the businesses that were actually rejected is the specificity.

One important aspect of classification is to determine a specific threshold that would be utilized to classify the observations in 2 categories (1/"approved" or 0/"rejected"). Sensitivity and specificity are metrics that are often used to maximize the proportion of observations with outcomes that are predicted correctly.

The Receiver-Operating Characteristics (ROC) curve is commonly used to visually represent the value obtained for sensitivity and specificity across a range of thresholds. In fact, one minus specificity is used for the ROC. This is the equivalent of having the proportion of false positives. Figure 4-1 shows a couple of theoretical examples of such a curve and Figure 4-2 presents a ROC curve extracted from a set of data. The closer the ROC curve is to the point (0,1), the better the classification is. The dash line in Figure 4-1 shows what we obtain if each observation has a 50-50 chance of being correctly classified or misclassified. In this case, the model used for classification would not be better than a random classification. Thus, the closer the ROC curve is to this line, the worse the classification.

Figure 4-1: Theoretical Receiver-Operating Characteristics curves (ROC)

Figure 4-2: Empirical example of the Receiver-Operating Characteristics curve (ROC)

Various methods exist to select the optimal threshold, for example the Youden method (Youden, 1950) or the Liu method (Liu, 2012).^{Footnote 11} The first method would optimize the sum of sensitivity and specificity, while the second would optimize the product. Another important metric that helps to assess the overall performance of the classifiers is the area under the curve (AUC). The closer this value is to one, the better the threshold is to correctly classify observations.

In a perfect scenario, all observations are correctly classified, which implies that sensitivity and specificity are respectively equal to one (with one minus specificity equals zero). In this context, the ROC curve would take the form of the singleton (0,1) with AUC equal to one (assuming that the AUC is the area of the rectangle [0,1] x [0,1]). In Figure 4-2, the AUC is estimated at 0.71.

The second step of the analysis uses the resulting model and the estimated coefficients to estimate the probability of a CSBFP borrower being approved. The estimated value is between zero and one for each CSBFP borrower. Then, the classification method used in step two is used to classify CSBFP borrowers as either "approved" or "rejected".

The estimated rate of incrementality is the estimated proportion of CSBFP borrowers that are classified as rejected and would have been turned down in the absence of the CSBFP.

5. Variables and descriptive statistics

The dependent variable associated with equation (4.1) is 1 if a debt financing (term loan or non-residential mortgage) request is fully or partially approved^{Footnote 12} and 0 if rejected. The independent variables included in the logit regression are presented in the table below. Those variables are related to the firm's characteristics and firm's financial performance. They also include owner's or primary decision maker's characteristics. The financial performance variables^{Footnote 13} used in the model are also lagged and represent the firm's performance in 2019. Financial institutions that assessed a firm's debt financing request in 2020 likely used information from past years.

In the next part of this paper, we will consider three different mutually exclusive sub-groups: CSBFP borrowers, SMEs that had their debt financing request approved, and SMEs that had their request rejected. The two last sub-groups are comprised of non-CSBFP SMEs. Tables 5-2 to 5-5 show the average for each variable indicated or the percentage for each categorical variable.

CSBFP borrowers tend to be smaller (business size) and younger (business age) than SMEs that were approved (Table 5-2). They also have less revenue and their net income was lower than those of approved SMEs.

CSBFP borrowers have a lower solvency ratio in 2019 than that of the SMEs that were approved. SMEs that were rejected had a higher debt ratio than approved SMEs and a lower solvency ratio. Also, their net income in 2019 was negative compared with a positive net income for approved SMEs. Rejected SMEs also had an average revenue and an average total value of assets lower than those of approved SMEs.

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Note: *The total number of SMEs is the weighted counts and is based on the survey's weights. It indicates how many SMEs are represented for each category by the survey sample.

Table 5-3 shows that more CSBFP borrowers were majority owned by women and visible minorities compared with SMEs that requested debt financing in 2020. Furthermore, CSBFP borrowers were more likely to have post-secondary education than non-CSBFP borrowers.

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Notes: Approved and rejected SMEs were aggregated to avoid suppression. "x" indicates that data were suppressed to meet confidentiality requirements of the Statistics Act.

The majority of CSBFP borrowers were in the accommodation and food services (40%) and the retail trade or wholesale trade (18%) industry sectors (Table 5-4). In comparison, 9% of non-CSBFP borrowers in 2020 were in the accommodation and food services and 14% in the retail trade and wholesale trade sectors.

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Notes: Approved and rejected SMEs were aggregated to avoid suppression. *The primary sector includes agriculture, forestry, fishing and hunting and mining, quarrying, and oil and gas extraction.

Most of the CSBFP borrowers and the non-CSBFP borrowers were located in Ontario and Quebec (Table 5-5). The table shows that CSBFP borrowers are over-represented in Ontario and slightly over-represented in the Atlantic region and under-represented to a small extent in Quebec and Alberta.

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Note: Approved and rejected SMEs were aggregated to avoid suppression.

6. Results

The estimated coefficients for the logit regression^{Footnote 14} used on the non-CSBFP borrowers (i.e., the SMEs that had their debt financing approved or rejected in 2020) are reported in Table A-1 (see Appendix).

Table 6-1 gives the estimated average marginal effect of the variables. In this context, they quantify the effect of a small variation of the continuous explanatory variable (or unit change for binary variable) on the firm's probability of being approved or rejected. Average marginal effect is useful insofar as it gives one a sense of the magnitude of the impact on the firm's probability of being approved or rejected.

Two variables in particular stand out from the table: age and debt ratio. Debt financing requests for older SMEs are more likely to be approved. SMEs with higher debt ratio are less likely to be approved. In magnitude, the effect of age and debt ratio is similar (in absolute terms).

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Notes: For some of the variables, only the sign of the estimated average marginal effects appears in the table in order to meet the confidentiality requirements of the Statistics Act. Robust standard errors are in parentheses and are estimated using the delta-method. * Statistically significant at the 10%-level, ** Statistically significant at the 5%-level, *** Statistically significant at the 1%-level.

As mentioned in the previous section, the next step is to use the estimated coefficients obtained with the logit regression but apply them to the CSBFP borrowers to estimate the probability of being approved or rejected. Then the classification method used for non-CSBFP borrowers is used for the CSBFP borrowers to determine firms that are approved and rejected. The three methods used (Liu's method, Youden's method and the nearest point) give the same results.

The empirical optimal threshold is approximatively 0.98 with sensitivity equals to 0.68 and specificity equals to 0.74. The area under the ROC curve is estimated at 0.71. Approximately 70% of the observations were correctly classified. Incrementality corresponds to the percentage of CSBFP borrowers that would have been rejected based on this methodology. We also use bootstrap methods to estimate the standard error and to compute the confidence interval of the estimated incrementality value obtained. The results are reported in Table 6-2.

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Notes: Standard errors were estimated using bootstrapping method with 5,000 replications. Results from Liu's method are reported.

We find that the estimated value for incrementality is 75.8%. In other words, 75.8% of CSBFP borrowers would have been rejected had the program not existed. The bias-corrected^{Footnote 15} confidence interval in this case is [72.6%,90.5%]. The incrementality rate is estimated to be between 72.6% and 90.5%.

Like any empirical studies, the model used in this paper also has some limitations. The incrementality rate estimate may suffer from the omitted variables bias or the selection bias. Even though the model captures several characteristics of the business, the owner or the principal decision maker, variables that could be correlated with the probability of being approved or denied could be missing in our model.

For example, anything related to the credit history of the business, or the owner, is not included in the logit regression. At the same time, the financial ratios used in the model such as the debt ratio or the solvency ratio would capture the business's ability to pay and its debt level.

The loss of observations due to missing values could also have an impact on the estimate. We tried to limit it by using financial variables that are in general mandatory for businesses to provide in their corporation tax form. Also, administrative data tend to be missing for start-ups and very young small businesses. This could have an impact on the population representativeness of the sample resulting in a biased estimate of the incrementality rate.

7. Conclusion

The Canada Small Business Financing Program (CSBFP) helps small businesses to get loans from financial institutions by sharing the risk with lenders. The CSBFP is a statutory loan loss-sharing program governed by the Canada Small Business Financing Act and administered by Innovation, Science and Economic Development Canada (ISED). This study was designed to provide an empirical estimate of the level of incrementality of the CSBFP in 2020. In this context, incrementality corresponds to the percentage of CSBFP borrowers whose debt financing request would have been rejected without the existence of the program.

Using data from Statistics Canada's Survey on Financing and Growth of Small and Medium Enterprises, 2020 and the Linkable File Environment, the incrementality of the CSBFP is estimated at 75.8%. In other words, the proportion of CSBFP borrowers that would have had their debt financing request denied in 2020 is 75.8%, if the program did not exist.

References

Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics Methods and Applications. Cambridge: Cambridge University Press.

Cameron, A. C., & Trivedi, P. K. (2022). Microeconometrics Using Stata, Volume 1: Cross-Sectional and Panel Regression Methods. College Station: Stata Press Publication.

Equinox. (2003). "Incrementality of CSBF Program Lending, Volume 1: Insights from SME FDI Data." Ottawa: Equinox Management Consultants Ltd.

Innovation, Science and Economic Development Canada. (2022). "Key Small business Statistics." Ottawa: Innovation, Science and Economic Development Canada.

Legendre, N. (2022). Essays on SME Growth and Financing. Doctoral Thesis, University of Ottawa.

Liu, X. (2012). "Classification accuracy and cut point selection." Statistics in Medicine, 31(23), 2676–2686.

Riding, A. (2009). "Canada Small Business Financing Program: Updated Analysis of Incrementality." Ottawa: Canada Works Limited.

Riding, A., Madill, J., & Haines Jr., G. (2007). "Incrementality of SME Loan Guarantees." Small Business Economics, 29(1), 47–61.

Rivard, P. (2018). "Incrementality Study of the Canada Small Business Financing Program." Ottawa: Innovation, Science and Economic Development Canada.

Seens, D., & Song, M. (2015). "Requantifying the Rate of Incrementality for the Canada Small Business Financing Program." Ottawa: Industry Canada.

Youden, W. J. (1950). "Index for rating diagnostic tests." Cancer, 3(1), 32–35.

Appendix

Sources: Statistics Canada, Survey on Financing and Growth of Small and Medium Enterprises, 2020; Statistics Canada, Linkable File Environment; and author's calculations.

Notes: Controls for industry sector and provinces or territories are included. Reference category for region is British Columbia and Territories. The reference category for industry sector is retail. Coefficients were computed using survey weights. For some of the variables, only the sign of the estimated coefficients appears in the table in order to meet confidentiality requirements of the Statistics Act. Robust standard errors are in parenthesis. * Statistically significant at the 10%-level, ** Statistically significant at the 5%-level, *** Statistically significant at the 1%-level.