Authors

A generalized attention model for survival analysis of complex objects under censored data

A wide variety of applied fields, including medicine, security, economics, and industry, are concerned with modeling the processes of various events occurring, such as a patient’s recovery, a company’s financial bankruptcy, industrial equipment failure, etc. Their modeling can be performed within the framework of survival analysis, a statistical method for analyzing time-to-event data whose distinctive feature, setting it apart from many other statistical and machine learning methods, is the presence of censored data. This occurs when an event is not observed and it is only known that it did not happen before a certain point in time. Censored data significantly complicates the modeling and prediction of critical events. Machine learning is an effective tool for survival analysis in the presence of censored data. In particular, modern transformer-based machine learning models demonstrate promising results in survival analysis due to their ability to account for complex dependencies. However, the standard attention mechanism in these models often ignores the fundamental structure of time-to-event data, namely, the distinction between censored and uncensored observations. To overcome this shortcoming, this paper proposes a new model and a new approach to implementing an attention mechanism that redefines attention weights by incorporating prior characteristics of survival analysis based on the Beran estimator or the Cox model. Instead of relying solely on distances between feature vector representations, as is done in current models, the proposed model computes attention weights as a weighted linear combination of components derived from key prior characteristics of survival analysis, such as distances between survival function estimates or time-to-event expectations for different training objects. The proposed approach enables a significant expansion of the class of transform models for survival analysis, achieving higher prediction accuracy. The algorithm implementing the proposed model is the basis for transformers. Experiments on real datasets confirm that the generalized model provides the best prediction among a number of known models. Read more...