Then by using the layout of the confusion matrix plotted in Figure 6, the four regions are divided as True Positive (TN), False Positive (FP), False Negative (FN) and True Negative (TN) ifвЂњSettledвЂќ is defined as positive and вЂњPast DueвЂќ is defined as negative,. Aligned with all the confusion matrices plotted in Figure 5, TP could be the loans that are good, and FP could be the defaults missed. Our company is interested in those two regions. To normalize the values, two widely used mathematical terms are defined: true rate that is positiveTPR) and False Positive Rate (FPR). Their equations are shown below:
In this application, TPR may be the hit price of good loans, and it also represents the capacity of earning cash from loan interest; FPR is the lacking rate of standard, also it represents the likelihood of losing profits.
Receiver Operational Characteristic (ROC) bend is considered the most widely used plot to visualize the performance of the category model no credit check payday loans Manchester MO after all thresholds. In Figure 7 left, the ROC Curve associated with Random Forest model is plotted. This plot really shows the partnership between TPR and FPR, where one always goes into the exact same way as one other, from 0 to at least one. a classification that is good would will have the ROC curve over the red standard, sitting by the вЂњrandom classifierвЂќ. The region Under Curve (AUC) can be a metric for assessing the category model besides accuracy. The AUC of this Random Forest model is 0.82 away from 1, that will be decent.
Although the ROC Curve plainly shows the partnership between TPR and FPR, the limit can be an implicit adjustable. The optimization task cannot purely be done by the ROC Curve. Therefore, another dimension is introduced to add the threshold adjustable, as plotted in Figure 7 right. Considering that the orange TPR represents the capacity of getting FPR and money represents the possibility of losing, the instinct is to look for the limit that expands the gap between curves whenever you can. In cases like this, the sweet spot is about 0.7.
You can find limits to the approach: the FPR and TPR are ratios. Also we still cannot infer the exact values of the profit that different thresholds lead to though they are good at visualizing the impact of the classification threshold on making the prediction. The FPR, TPR vs Threshold approach makes the assumption that the loans are equal (loan amount, interest due, etc.), but they are actually not on the other hand. Individuals who default on loans could have a greater loan quantity and interest that have to be repaid, also it adds uncertainties towards the modeling outcomes.
Luckily for us, detail by detail loan amount and interest due are available from the dataset it self.
The only thing staying is to locate ways to link these with the limit and model predictions. It isn’t tough to determine a manifestation for revenue. These two terms can be calculated using 5 known variables as shown below in Table 2 by assuming the revenue is solely from the interest collected from the settled loans and the cost is solely from the total loan amount that customers default