Introduction
Recent technological advancements have led to complex artificial intelligence (AI) systems that require effective training for optimal performance. Interpretability is the extent to which the internal mechanics and rationale behind machine learning models’ decisions can be understood. While interpretable models are designed to be naturally understandable, model-agnostic methods use tools and techniques to extract insights from complex AI systems.
Review and Comparison
Interpretable, model-agnostic methods have distinct characteristics that set them apart. The interpretable models allow individuals to understand their decision-making rationale by directly examining their internal workings. Model-agnostic approaches are those that can be applied to any AI system, regardless of its internal structure (Molnar, 2023). When compared, interpretable models are easier to interpret but have lower predictive power than model-agnostic methods.
Interpretability Methods
Linear regression and decision trees are common methods for achieving interpretability in machine learning models. While decision trees visualize the rules for decision-making, linear regression predicts continuous outputs by modeling the relationships among variables (Hassija et al., 2023). LIME, Permutation Importance (PIMP), Partial Dependence Plots (PDPs), Individual Conditional Expectation (ICE) plots, global surrogate models, and SHAP are some of the interpretability methods used in model-agnostic methods. Therefore, the choice of the interpretability method depends on the system’s needs and complexity.
Characteristics
Model-agnostic methods and interpretable models have distinct characteristics. LIME applies a simple model based on a specific prediction to explain its rationale for a given decision (Ribeiro et al., 2016). Another characteristic of LIME is that it is based on perturbation: it perturbs the input features and observes the resulting changes in the output.
Meanwhile, SHAP utilizes game theory to distribute credit for prediction among features (Lundberg & Lee, 2017). Additionally, SHAP is characterized by fair attribution, as it uses Shapley values to attribute each feature’s contribution to each prediction. On the other hand, the global surrogate models, PIMP, and ICE plots provide simplified local interpretations that make them easy to understand even for complex systems.
The characteristics of interpretable models can also be explored by considering individual interpretability methods. Linear regression is associated with key assumptions, such as homoscedasticity, linearity, and normality of errors (Molner, 2023). Moreover, the model uses cases to infer relationships between variables.
Meanwhile, decision trees are considered the easiest models to interpret. The latter uses nodes to split decisions, making it easy to understand why a particular decision was made (Molner, 2023). The model is also flexible, as it can handle both categorical and numerical data. Therefore, while model-agnostic methods have complex characteristics, interpretable methods are easy to understand.
Pros and Cons
The two strategies have distinct advantages and disadvantages that should be considered before adoption. Linear regression and decision trees are simple and straightforward since less data is needed (Ribeiro et al., 2016). Additionally, they are computationally inexpensive, making them effective and economical for large datasets.
However, the models perform poorly due to scalability issues and incompatibility with various systems. The main advantage of model-agnostic methods is their compatibility with complex systems and their universal applicability. Moreover, SHAP, LIME, PIMP, ICE, and other models can be extended to new data inputs, making them versatile (Ribeiro et al., 2016).
Nevertheless, model-agnostic methods suffer from accuracy issues because they do not account for a particular model’s mechanics. They are also more expensive than other models due to their complexity. It is important to maximize the benefits of both strategies to achieve optimal output.
Conclusion
The interpretable models are easy to understand and include linear regression and decision trees. The model-agnostic methods are complex and include SHAP and LIME. The latter offers insights into the system’s behavior regardless of its structure, but it is expensive. Therefore, it is important to consider the structure of an AI system before choosing an interpretability method to understand its behavior.
References
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2023). Interpreting black-box models: A review on explainable artificial intelligence. Cognitive Computation, 16.
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
Molnar, C. (2023). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (2nd ed.). Creative Commons.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Model-Agnostic interpretability of machine learning. ArXiv (Cornell University).