Методы обучения представлений для оптимальных процедур детектирования разладок / Representation learning methods for optimal change point detection procedures тема диссертации и автореферата по ВАК РФ 00.00.00, кандидат наук Романенкова Евгения Дмитриевна

• Романенкова Евгения Дмитриевна
• кандидат наук
• 2025

Оглавление диссертации кандидат наук Романенкова Евгения Дмитриевна

Content

Introduction

Chapter 1. Related Work

1.1 State-of-the-art in Change Point Detection area

1.2 Practical applications: Review on Change Point Detection for Oil&Gas industry problems

1.3 Conclusion

Chapter 2. Change Point Detection via Machine Learning

Classifiers

2.1 Methods

2.1.1 Machine Learning classifier to embed input signal

2.1.2 Change Point Detection approaches

2.1.3 Validation metrics

2.2 Practical application: rock type identification during drilling

2.2.1 Data overview and preprocessing

2.2.2 Domain-specific metrics for task-oriented validation

2.3 Experiments and Results

2.3.1 Rock type classification

2.3.2 Results on refinement with statistical change detection

2.4 Discussion and Future work

Chapter 3. Instant Disorder Detection via a Principled Neural

Network

3.1 Methods

3.1.1 Loss function for InDiD approach

3.1.2 InDiD approach

3.1.3 Baselines

3.1.4 Benchmarks and Metrics

3.1.5 Datasets

3.2 Experiments and Results

3.2.1 Comparison details

3.2.2 Results on Change Point Detection

3.2.3 Model understanding

3.3 Discussion and Future work

Chapter 4. Detecting Akin and Changing Patterns Using

Self-Supervised Similarity Learning

4.1 Methods

4.1.1 Classification-based and contrastive-based approaches to similarity learning

4.1.2 Generative-based approach to similarity learning

4.1.3 Similarity learning models

4.1.4 Change Point Detection based on similarity measure

4.1.5 Validation metrics

4.2 Practical application: well-log similarity estimation and changes

detections

4.2.1 Data overview and preprocessing

4.3 Experiments and Results

4.3.1 Implementation details

4.3.2 Similarity models performance

4.3.3 Comparison of representations and expert's labels

4.3.4 Results on adaptation to the new data

4.3.5 Results on Change Point Detection based on well-log similarity

4.4 Discussion and Future work

Conclusion

Acknowledgements

List of Abbreviations

List of figures

List of tables

Appendix A. Evidence of Industrial Application

Введение диссертации (часть автореферата) на тему «Методы обучения представлений для оптимальных процедур детектирования разладок / Representation learning methods for optimal change point detection procedures»

Introduction

Modern industry uses complicated systems that continuously work online and are vital for the well-being of large companies and humankind in general. Sudden collapses or prolonged unavailabilities of systems lead to significant losses to business owners. Thus, it is essential to detect deviations or, in other words, change points in a system's behavior as fast and accurately as possible.

Typically, data for detection come from a sequential stream represented as either multivariate vectors from sensors or so-called semi-structured data such as videos and image sequences [1—3]. We can also describe disorders, ruptures, shifts, and other changes in sequential data as point anomalies [4]. Currently, the anomaly detection area for sequential data concentrates on applying machine learning and deep learning techniques. It adopts widespread methods while considering more specific models and loss functions [5]. Most existing methods from this area concentrate on classifying whole sequences, whether they contain anomalies or are normal [2; 5; 6]. However, this problem statement does not reflect industrial needs, where the reaction to an anomaly should be as close in time to its moment as possible to prevent unexpected expenses.

An alternative approach comes from the Change Point Detection (CPD) field: we aim to directly minimize the delay of disorder detection while maintaining the number of false alarms low [4; 7]. Unlike the anomaly detection formulation, the CPD is naturally suited for various monitoring systems where the cost of waiting for reaction is high [8—11]

Change points can also appear in normal processes, defining natural alteration of the underlying systems. Understanding the process regime segments and their shift moments is necessary to organize work properly. In this case, the objective is a precise segmentation of a sequence [1; 12—14].

Some methods can deal with different simple CPD problems [4; 7]. Under strong assumptions, theoretically optimal solutions are available [1; 15]. These approaches are still popular and have tremendous real-world applications, including heavy industrial sensor management [8; 16—20], financial data analysis [9], control of high-loaded systems [10; 11; 21], health monitoring [22; 23], genome sequencing [24], and others. The key particularity is the usage of a detector «as it is» while ignoring the violation of assumptions and the specificity of the considered task. For

example, the paper [8] uses the standard CUSUM procedure on top of residuals from autoregressive model ARIMA for monitoring rotary machines. However, to provide theoretical optimality, CUSUM assumes independence of the input data [25], which is generally not true for ARIMA model residuals. An additional gap is that there are little or no theoretical guarantees in realistic high-dimensional cases such as controlling video streams [2; 26].

Nowadays, authors adapt classic theory to stream data of extreme dimensionality when the vector size is much greater than the sequence length. This problem statement poses the overall difficulty of theoretical analysis: the appearance of the curse of dimensionality, considering many noise features and dealing with computational issues connected with vector size [27—30]. For example, the authors [28] suggest that the detection delay is of order yfD for traditional parametric models [25], where D is a single observation dimension. This delay is obviously prohibitively high for many real problems. Such a special real-world task can be video surveillance, where one operates with data containing several thousands of features. To overcome the above issue, recent works effort to bound the class of parametric models and concentrate on particular high-dimensional cases: multivariate Gaussian vectors [28; 31], various graph representations [9; 27], e.t.c. Another possible solution for CPD, such as an application of non-parametric CPD approaches [7; 15], does not assume any specific model structure. However, it often relies on techniques that also may be limited in high-dimensional settings: density estimation [32], choice of optimal kernel [33], or the consideration of long-run dependency [30]. While additional assumptions like sparsity allow for reducing the delay, practitioners go a different road, hoping that simpler models can learn to detect changes.

A natural impulse is to use the available historical information to create a data-based change point detector via the representation learning process [34]. To be precise, representation learning aims to obtain compact low-dimensional data embeddings by incorporating the maximum amount of information relevant to the application task at hand. In relation to sequential data, the task is challenging, as we should take into account the context, but it seems to be the only way to get high-performing models [26]. Some modern studies use this paradigm for change point detection while still considering low-dimensional time series [32; 33; 35—37]. Moreover, authors attempt to adapt solutions from other fields like self-supervised learning [36; 37] or generative models [33; 35] and do not incorporate the theoretical nature of the problem into a learning process.

To sum up, theoretical-based approaches related to change detection often operate with too restrictive assumptions and have a limited ability to work with complex data. The modern CPD suggests a representation learning procedures as an answer while still concentrating on simple time series. Both ways often consider general, vague solutions, ignoring the task specificity of real industrial problems. CPD-related tasks also appear in sophisticated data modalities with hundreds and thousands of features, like controlling images and video streams. Existing methods from anomaly detection, closest to the CPD area, deal with these structures through efficient neural network-based solutions with corresponding representations. Nevertheless, they rarely can point at precise change moments online, which is crucial for many applications. Therefore, the CPD area needs a theoretical-grounded representation learning scheme for data of different dimensionality and an understanding of how to involve concrete task particularities.

The aim of the work is to develop a change point detection framework suitable for data of different modalities. As for such data, we consider diverse datasets with low and moderate dimensionality from industrial applications and complex high-dimensional sequences such as video streams.

To achieve the stated aim, it is necessary to solve the following tasks:

1. To investigate well-established approaches for change point detection on time series of moderate dimensionality and to apply them to real industrial problems, taking into account data specificity.

2. To develop optimal change point detection procedures for high-dimensional data and to create the corresponding software implementation.

3. To incorporate the representation learning framework for change point detection-related problems as well as for industrial sensors data and more complex video data.

4. To design a proper validation procedure for testing developed methods based on open datasets for data of low and moderate dimensionality. For testing on high-dimensional data, to collect a dataset of change points in videos with the corresponding markup.

Scientific novelty. The majority of existing research in the current change point detection area concentrates on unsophisticated numerical time series with low and moderate dimensionality. At the same time, examples of the change point detection application are wider and include complex high-dimensional structures, such as the incident registration for surveillance video. Recent approaches to process-

ing sequential data are often based on representation learning strategies. However, there is an apparent lack of a representation-learning procedure that provides a theoretically motivated, efficient change point detection method suitable for complex data structures.

On the other hand, existing change point detection methods help to solve real industrial tasks connected with time series of low dimensionality. Nevertheless, they suffer from strong assumptions that are often violated in real-world data. Thus, incorporating industrial domain specifics into the framework of change point detection is also a crucial task.

The gaps mentioned above in the current research area stage lead to claim the following novelty of the results:

1. The work introduces a comprehensive framework for change point detection based on representation learning depicted in Figure 1. Such methods for change point detection were understudied before despite their proven effectiveness in other fields. The proposed methods offer a flexible toolkit that allows users to efficiently work with sequential data of varying dimensions, from low to high. Moreover, the framework can be applied both in supervised and unsupervised scenarios, making it a versatile solution for different types of data analysis.

2. For the first time, the change point detection paradigm is used to enhance the traditional machine learning approaches. The suggested method ensures high-quality supervised change point detection while minimizing false alarms. The developed solution has been tested for a real industrial task, resulting in a new model for rock-type classification based on logging while drilling data.

3. A novel loss function for change point detection via neural networks is developed. The presented loss function explicitly controls detection delay and time to false alarms. Such an approach opens up new opportunities for detecting change points in video data that have not been previously researched. Moreover, the first markup for solving the CPD problem on videos for further research has been presented.

4. The dissertation contributes to the area of self-supervised learning for change point detection by developing a model for obtaining universal representations. Unlike previous studies, the presented model requires no specific modifications to address change point detection, which broadens its prac-

tical applications. For the first time, the thesis findings demonstrate a strong correlation between the qualities of similarity estimation and change point detection, highlighting the effectiveness of the proposed method. The solution has been successfully adapted to the practical industrial task of well-log analysis, which resulted in the development of a new approach for detecting similar and changing patterns in well-logs.

Representation learning-based framework for optimal change point detection

Supervised

Low-dimensional data

High-dimensional data

Chapter 2. Two-staged system: ML + statistical CPD

Chapter 3. Theoretical-grounded loss function for Neural CPD

Unsupervised

Low-dimensional data

High-dimensional data

Chapter 4. Universal SSL representations for CPD

Can be adapted to high-dimensional case

TBD in further works

Figure 1 — A scheme of our framework for optimal change point detection based on representation learning. The proposed solution works effectively regardless of the expert's markup availability and input data dimensionality.

Scientific and practical significance. From a theoretical value point of view, the work provides a new loss function that is the lower bound for the classic rigorous solution. This also has important implications in practical applications, as it opens up new avenues for efficient solutions related to data of different dimensions and modalities. Another focus of the presented work is the data of low dimensionality from the industry and the corresponding tasks. Understanding how the change point detection methods can be applied to actual industrial problems and which data representation is necessary enforces the quality of real processes.

Research methodology. The research focuses on machine learning and deep learning techniques to address the stated objectives. More precisely, we investigate supervised and self-supervised paradigms for learning representations suitable for change point detection problems. To work accurately and efficiently, the proposed approaches rely on the statistical theory of change point detection methods and corresponding developments. Precise mathematical language allows the concise formulation of obtained results regarding their properties. Geological experts support

the research, providing an understanding of oil&gas domain specificity in the corresponding thesis chapters. The methods are implemented in Python programming language and its low-level libraries.

Propositions submitted for the defense:

1. A consistent representation learning framework for change point detection is developed. The solution effectively generates representations for multivariate time series data, as well as high-dimensional data, regardless of the availability of labels. The approach has solid theoretical properties under natural assumptions and has been validated through diverse real-world applications, including human activity recognition, rock-type detection during drilling, well-log analysis, and video surveillance.

2. In the context of low-dimensional data, a novel two-stage system for change point detection is introduced. In the first step, the machine learning classifier is trained to output change point probabilities. Then, by leveraging these probabilities as representations, the new detection statistic accurately identifies specific types of change points. Empirical and theoretical analyses confirm that this approach significantly reduces the number of false alarms compared to existing baselines. In practical scenarios, it achieved a tenfold reduction in false alarms and detection delay, which substantially improved existing methods.

3. The dissertation presents a theoretically grounded loss function for change point detection through supervised representation learning. Following the genuine criteria for the detection delay and time to false alarm, the proposed differentiable loss approximates the corresponding infinite sums with their truncated versions and novel correction terms. A presented theoretical analysis establishes two key properties of the suggested approximation: firstly, it provides a valid lower bound for the true criteria, and secondly, it is asymptotically tight. Experimental evidence complements this analysis, suggesting that deep networks trained with the proposed loss function ensure robust performance across various scenarios from low-dimensional to ultra-high-dimensional data.

4. Finally, the dissertation introduces a method for constructing representations based on self-supervised similarity learning without using expert markup. By adapting and refining existing techniques based on self-supervised loss function, this method can handle the intricacies of multivariate

time series data, particularly those with complex, non-i.i.d. characteristics. Obtained embeddings have a broad range of applicability - and, in particular, allow unsupervised change point detection, completing the proposed framework.

The validity and reliability of the obtained results and conclusions are provided by comprehensive validation procedures that are in accordance with the leading works in the area. We also rely on the precise methods used to investigate the properties of change point detection methods, utilizing common assumptions in this field. All approaches are tested from different points of view, including various metrics groups, usage of several datasets (if applicable), and considering additional downstream tasks.

Approbation of the results. The results and main provisions of the work were presented at international and Russian conferences:

1. SPE Annual Technical Conference and Exhibition, Calgary, Alberta, Canada, September 2019;

2. «Information Technologies and Systems» Interdisciplinary Conference, online, November 2021;

3. 30th ACM International Conference on Multimedia (A* rank), Portugal, Lisboa, October 2022;

4. 64-th Russian Conference of MIPT, Moscow, Russia, November 2022;

5. 65-ve Russian Conference of MIPT, Moscow, Russia, April 2023;

6. Workshop «Methods of Artificial Intelligence for Industrial Tasks of Sustainable Development», Belokurikha, Russia, August 2023;

7. International Conference «Complex time series analysis: high-dimensionality, change-points, forecasting and causality», Sanya, Hainan, China, January 2024;

8. International Conference «Data Fusion» 2024, Moscow, Russia, April 2024.

Personal contribution. The content of the dissertation and the main

statement submitted for defense reflect the author's personal contribution to the published works and have been received personally by the author or with her direct participation. The author's contribution includes setting research objectives, studying literature corresponding to the topic, designing and carrying out experiments, and analyzing the received results. The problem statements were formulated by A. Zaytsev, the author's scientific supervisor, and the research results were discussed with the co-authors.

Publications. The publications in peer-reviewed scientific journals also support the credibility of research results. The dissertation materials were published in 6 works, including 3 papers in Q1 scientific journals indexed by Web of Science and Scopus, 1 paper in Q2 journal indexed by Web of Science and Scopus, 1 extended abstract at international conferences, and 1 article in the proceedings of the A* conference also indexed by Scopus. In 4 publications, the author of the dissertation is the first author or has a shared contribution with a first author. In addition, 1 computer program was registered by the state.

The volume and structure of the work. The dissertation consists of an abstract, introduction, 4 chapters, conclusions, lists of abbreviations, figures, tables and 1 appendix. The full thesis' volume is 118 pages, including 21 figure and 16 table. The list of references consists of 140 sources.

Заключение диссертации по теме «Другие cпециальности», Романенкова Евгения Дмитриевна

Conclusion

This thesis investigates the optimal change point detection procedures for data of varying dimensionality and presents a versatile framework based on representation-learning techniques to address this challenge. Through comprehensive analysis and experiments, we assess a diverse range of data types and settings, encompassing both supervised and unsupervised cases, ranging from real-world sensor data to videos with change points. Our research is grounded in both theoretical justification and practical applications. The main results are the following:

1. The dissertation presents a consistent representation learning framework for change point detection. Throughout our work, we have accurately justified that each part of our solutions effectively solves real-world CPD problems. Altogether, they provide a flexible tool suitable for different scenarios, from simple signals with moderate dimensionality to challenging high-dimensional CPD on video data, regardless of the availability of markup.

2. For low-dimensional streams, a framework suggests a two-stage system that combines the effectiveness of classic machine learning with the CPD-ori-ented approaches. This part is tested on the important industrial challenge of rock type identification based on information from sensors used during directional drilling. Our procedure reduces the detection delay from 20 to 1.8 meters and the number of false positive alarms from 43 to 6 per well. This technology has been implemented by a major oil and gas company, saving billions of rubles every year.

3. Our work presents a new theoretical-grounded universal loss function for supervised change point detection via representation learning. The suggested loss function approximates the classic criteria while balancing change detection delay and time to a false alarm. We have shown that it is a dif-ferentiable tight lower bound on the conventional criteria with a power-law convergence rate. According to the findings, the neural networks trained with our loss function provide meaningful representations tailored to the specific nature of the CPD and complex data structures. Such a model can be applied to a wide range of data types, including sensor data and videos.

For example, for explosion detection in video surveillance, the F1 score for our method is 0.53 compared to baseline scores of 0.31 and 0.35 4. We have also developed a model for generating universal representations through similarity learning in an unsupervised scenario. Unlike other CPD-oriented approaches, these embeddings can be applied to a broader spectrum of tasks. For instance, our method highly correlates with expert labeling of intra- and inter-well correlations with ARI and AMI as high as 0.660 and 0.727, respectively, compared to 0.340 and 0.447 for the baseline. At the same time, our approach effectively identifies the shifting of strati-graphical layers and lithotype changes. The F1 scores for these tasks are up to 0.454 and 0.630, respectively, outperforming the most state-of-the-art CPD approach, which achieves F1 scores of 0.387 and 0.511. The presented results provide a significant step forward in studying various representation-learning techniques for change point detection tasks and open up new avenues for further refinement.

The obvious restriction of our two-staged ML-CPD system and InDiD approach is their supervised nature. Typically, there is a lack of annotated data, and the labeling process requires additional expenses and time-consuming expert efforts. We aim to fill this gap by presenting the unsupervised method based on similarity learning. However, the InDiD loss function can also be adapted to an unsupervised setting similar to works [31; 40], where the authors considered all possible sequence partitions on two segments.

The dissertation insights have become the foundation for various works. Several colleagues' papers have already complemented our research on a fundamental representation-learning solution for industrial sequences [70; 131]. Moreover, we have evaluated which model type provides desired properties for different scales of tasks on real-world event sequences with irregular grid [110]. Finally, we have studied how to incorporate CP principles in similarity learning. More particularly, we have found that Spectral Normalization [122] ensures shifts in embedding space, and Layer Normalization [132] spreads information about the changes facilitating their detection.

Список литературы диссертационного исследования кандидат наук Романенкова Евгения Дмитриевна, 2025 год

References

1. Burg, G. J. van den. An evaluation of change point detection algorithms / G. J. van den Burg, C. K. Williams // arXiv preprint arXiv:2003.06222. — 2020.

2. Sultani, W. Real-world anomaly detection in surveillance videos / W. Sul-tani, C. Chen, M. Shah // Proceedings of the IEEE conference on computer vision and pattern recognition. — Salt Lake City, Utah, USA : IEEE, 2018. — P. 6479—6488.

3. Epstein, D. Oops! predicting unintentional action in video / D. Epstein, B. Chen, C. Vondrick // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. — Virtual : IEEE, 2020. — P. 919—929.

4. Sequential (quickest) change detection: Classical results and new directions / L. Xie [et al.] // IEEE Journal on Selected Areas in Information Theory. — 2021. — Vol. 2, no. 2. — P. 494—514.

5. A review of time-series anomaly detection techniques: A step to future perspectives / K. Shaukat [et al.] // Advances in Information and Communication: Proceedings of the 2021 Future of Information and Communication Conference, Volume 1. — Springer. 2021. — P. 865—877.

6. Anomaly detection with generative adversarial networks for multivariate time series / D. Li [et al.] // arXiv preprint arXiv:1809.04758. — 2018. — Vol. abs/1809.04758.

7. Aminikhanghahi, S. A survey of methods for time series change point detection / S. Aminikhanghahi, D. J. Cook // Knowledge and information systems. — 2017. — Vol. 51, no. 2. — P. 339—367.

8. Wang, T. Multi-sensors based condition monitoring of rotary machines: An approach of multidimensional time-series analysis / T. Wang, G. Lu, P. Yan // Measurement. — 2019. — Vol. 134. — P. 326—335.

9. Optimal network online change point localisation / Y. Yu [et al.] // arXiv preprint arXiv:2101.05477. — 2021.

10. Artemov, A. Ensembles of detectors for online detection of transient changes / A. Artemov, E. Burnaev // Eighth International Conference on Machine Vision. Vol. 9875. — International Society for Optics, Photonics. 2015. — 98751Z.

11. Kurt, M. N. Real-time nonparametric anomaly detection in high-dimensional settings / M. N. Kurt, Y. Yilmaz, X. Wang // IEEE transactions on pattern analysis and machine intelligence. — 2020. — Vol. 43, no. 7. — P. 2463—2479.

12. Truong, C. Selective review of offline change point detection methods / C. Truong, L. Oudre, N. Vayatis // Signal Processing. — 2020. — Vol. 167. — P. 107299.

13. Braun, J. V. Multiple changepoint fitting via quasilikelihood, with application to DNA sequence segmentation / J. V. Braun, R. Braun, H.-G. Miiller // Biometrika. — 2000. — Vol. 87, no. 2. — P. 301—314.

14. Unsupervised offline changepoint detection ensembles / I. Katser [et al.] // Applied sciences. — 2021. — Vol. 11, no. 9. — P. 4280.

15. Tartakovsky, A. Sequential analysis: Hypothesis testing and changepoint detection / A. Tartakovsky, I. Nikiforov, M. Basseville. — Chapman & Hall, 2014. — 603 p.

16. Manufacturing process monitoring with nonparametric change-point detection in automotive industry / S. Guo [et al.] // Journal of Manufacturing Science and Engineering. — 2019. — Vol. 141, no. 7. — P. 071013.

17. Anomaly detection based on sensor data in petroleum industry applications / L. Marti [et al.] // Sensors. — 2015. — Vol. 15, no. 2. — P. 2774—2797.

18. Ide, T. Sparse Gaussian Markov random field mixtures for anomaly detection / T. Ide, A. Khandelwal, J. Kalagnanam // IEEE 16th International Conference on Data Mining. — 2016. — P. 955—960.

19. A change point detection approach for intelligent real-time identification of lost circulation events during drilling operations / F. Cannarile [et al.] // Abu Dhabi International Petroleum Exhibition and Conference. — SPE. 2021. — D012S142R002.

20. Yan, W. On accurate and reliable anomaly detection for gas turbine combus-tors: a deep learning approach / W. Yan, L. Yu // Annual Conference of the PHM Society. Vol. 7. — 2015.

21. Artemov, A. Detecting Performance Degradation of Software-Intensive Systems in the Presence of Trends and Long-Range Dependence / A. Artemov, E. Burnaev // IEEE 16th Int. Conf. on DM Workshops. — 2016. — P. 29—36.

22. Liu, S. Change-point detection method for clinical decision support system rule monitoring / S. Liu, A. Wright, M. Hauskrecht // Artificial intelligence in medicine. — 2018. — Vol. 91. — P. 49—56.

23. Zhang, W. A comprehensive study of smartphone-based indoor activity recognition via Xgboost / W. Zhang, X. Zhao, Z. Li // IEEE Access. — 2019. — Vol. 7. — P. 80027—80042.

24. Jia, S. Efficient change-points detection for genomic sequences via cumulative segmented regression / S. Jia, L. Shi // Bioinformatics. — 2022. — Vol. 38, no. 2. — P. 311—317.

25. Shiryaev, A. N. Stochastic Change-Point Detection Problems / A. N. Shiryaev. — Moscow : MCME, 2017. — 393 p.

26. Object-centric auto-encoders and dummy anomalies for abnormal event detection in video / R. T. Ionescu [et al.] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. — Long Beach, CA, USA : IEEE, 2019. — P. 7842—7851.

27. Chu, L. Sequential change-point detection for high-dimensional and non-euclidean data / L. Chu, H. Chen // IEEE Transactions on Signal Processing. — 2022. — Vol. 70. — P. 4498—4511.

28. Chen, Y. High-dimensional, multiscale online changepoint detection / Y. Chen, T. Wang, R. J. Samworth // Journal of the Royal Statistical Society Series B: Statistical Methodology. — 2022. — Vol. 84, no. 1. — P. 234—266.

29. Feature extraction for change-point detection using stationary subspace analysis / D. A. Blythe [et al.] // IEEE Transactions on Neural Networks and Learning Systems. — 2012. — Vol. 23, no. 4. — P. 631—643.

30. Change-point detection with feature selection in high-dimensional time-series data / M. Yamada [et al.] // Twenty-Third International Joint Conference on Artificial Intelligence. — 2013.

31. Enikeeva, F. High-dimensional change-point detection under sparse alternatives / F. Enikeeva, Z. Harchaoui // The Annals of Statistics. — 2019. — Vol. 47, no. 4. — P. 2051—2079.

32. Hushchyn, M. Online Neural Networks for Change-Point Detection / M. Hushchyn, K. Arzymatov, D. Derkach // arXiv preprint arXiv:2010.01388. — 2020.

33. Kernel change-point detection with auxiliary deep generative models / W.-C. Chang [et al.] // International Conference on Learning Representations. — Vancouver, Canada : ICLR, 2018.

34. Bengio, Y. Representation learning: A review and new perspectives / Y. Ben-gio, A. Courville, P. Vincent // IEEE transactions on pattern analysis and machine intelligence. — 2013. — Vol. 35, no. 8. — P. 1798—1828.

35. De Ryck, T. Change point detection in time series data using autoencoders with a time-invariant representation / T. De Ryck, M. De Vos, A. Bertrand // IEEE Transactions on Signal Processing. — 2021. — Vol. 69. — P. 3513—3524.

36. Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding / S. Deldari [et al.] // Proceedings of The Web Conference 2021. — Association for Computing Machinery, 2021.

37. A self-supervised contrastive change point detection method for industrial time series / X. Bao [et al.] // Engineering Applications of Artificial Intelligence. — 2024. — Vol. 133. — P. 108217.

38. Lorden, G. Procedures for reacting to a change in distribution / G. Lorden // The annals of mathematical statistics. — 1971. — P. 1897—1908.

39. Willsky, A. A generalized likelihood ratio approach to the detection and estimation of jumps in linear systems / A. Willsky, H. Jones // IEEE Transactions on Automatic control. — 1976. — Vol. 21, no. 1. — P. 108—112.

40. Puchkin, N. A contrastive approach to online change point detection / N. Puchkin, V. Shcherbakova // International Conference on Artificial Intelligence and Statistics. — PMLR. 2023. — P. 5686—5713.

41. M-statistic for kernel change-point detection / S. Li [et al.] // Advances in Neural Information Processing Systems. — 2015. — Vol. 28.

42. Kawahara, Y. Change-point detection in time-series data by direct density-ratio estimation / Y. Kawahara, M. Sugiyama // Proceedings of the 2009 SIAM international conference on data mining. — SIAM. 2009. — P. 389—400.

43. Hushchyn, M. Generalization of change-point detection in time series data based on direct density ratio estimation / M. Hushchyn, A. Ustyuzhanin // Journal of Computational Science. — 2021. — Vol. 53. — P. 101385.

44. Change-point detection in time-series data by relative density-ratio estimation / S. Liu [et al.] // Neural Networks. — 2013. — Vol. 43. — P. 72—83.

45. A kernel two-sample test / A. Gretton [et al.] // The Journal of Machine Learning Research. — 2012. — Vol. 13, no. 1. — P. 723—773.

46. Killick, R. Optimal detection of changepoints with a linear computational cost / R. Killick, P. Fearnhead, I. A. Eckley // Journal of the American Statistical Association. — 2012. — Vol. 107, no. 500. — P. 1590—1598.

47. Arlot, S. A kernel multiple change-point algorithm via model selection / S. Arlot, A. Celisse, Z. Harchaoui // Journal of machine learning research. — 2019. — Vol. 20, no. 162. — P. 1—56.

48. Dette, H. A likelihood ratio approach to sequential change point detection for a general class of parameters / H. Dette, J. Gosmann // Journal of the American Statistical Association. — 2020. — Vol. 115, no. 531. — P. 1361—1377.

49. Bishop, C. M. Pattern recognition and machine learning. Vol. 4 / C. M. Bishop, N. M. Nasrabadi. — Springer, 2006. — 738 p.

50. Zambon, D. Concept drift and anomaly detection in graph streams / D. Zambon, C. Alippi, L. Livi // IEEE transactions on neural networks and learning systems. — 2018. — Vol. 29, no. 11. — P. 5592—5605.

51. Sketching for sequential change-point detection / Y. Cao [et al.] // EURASIP Journal on Advances in Signal Processing. — 2019. — Vol. 2019. — P. 1—22.

52. Unsupervised change analysis using supervised learning / S. Hido [et al.] // Advances in Knowledge Discovery and Data Mining: 12th Pacific-Asia Conference, PAKDD 2008 Osaka, Japan, May 20-23, 2008 Proceedings 12. — Springer. 2008. — P. 148—159.

53. Ahad, N. Semi-supervised Sequence Classification through Change Point Detection / N. Ahad, M. A. Davenport // Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35/8. — 2021. — P. 6574—6581.

54. Ma, X. A deep Q-network based approach for online bayesian change point detection / X. Ma, L. Lai, S. Cui // 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing. — IEEE. Queensland, Australia : IEEE, 2021. — P. 1—6.

55. Generative adversarial nets / I. Goodfellow [et al.] // Advances in neural information processing systems. — 2014. — Vol. 27.

56. Moustakides, G. V. Training neural networks for likelihood/density ratio estimation / G. V. Moustakides, K. Basioti // arXiv preprint arXiv:1911.00405. — 2019.

57. Oord, A. v. d. Representation learning with contrastive predictive coding / A. v. d. Oord, Y. Li, O. Vinyals // arXiv preprint arXiv:1807.03748. — 2018.

58. False alarm reduction in drilling process monitoring using virtual sample generation and qualitative trend analysis / Y. Li [et al.] // Control Engineering Practice. — 2023. — Vol. 133. — P. 105457.

59. Drilling modeling and simulation: current state and future goals / J. Sugiura [et al.] // SPE. — 2015. — SPE-173045—MS.

60. Detournay, E. A phenomenological model for the drilling action of drag bits / E. Detournay, P. Defourny // International journal of rock mechanics and mining sciences & geomechanics abstracts. — 1992. — Vol. 29, no. 1. — P. 13—23.

61. Downton, G. Challenges of modeling drilling systems for the purposes of automation and control / G. Downton // IFAC Proceedings Volumes. — 2012. — Vol. 45, no. 8. — P. 201—210.

62. Hegde, C. Use of machine learning and data analytics to increase drilling efficiency for nearby wells / C. Hegde, K. Gray // Journal of Natural Gas Science and Engineering. — 2017. — Apr. — Vol. 40. — P. 327—335.

63. Data-driven model for the identification of the rock type at a drilling bit / N. Klyuchnikov [et al.] // Journal of Petroleum science and Engineering. — 2019. — Vol. 178. — P. 506—516.

64. A hybrid GP regression and clustering approach for characterizing rock properties from drilling data / H. Zhou [et al.] // Technical Report ACFR-TR-2011-001. — 2010.

65. LaBelle, D. Lithological classification by drilling / D. LaBelle. — Carnegie Mellon University, The Robotics Institute, 01/2001. — 46 p.

66. Rock recognition from MWD data: a comparative study of boosting, NNs, and fuzzy logic / A. Kadkhodaie-Ilkhchi [et al.] // IEEE Geoscience and Remote Sensing Letters. — 2010. — Vol. 7, no. 4. — P. 680—684.

67. Determination of lithologic difference at the bottom of wells using cognitive technologies / I. Simon [et al.] // SPE Russian Petroleum Technical Conference. — Society of Petroleum Engineering. 2018.

68. Similarity learning for well logs prediction using machine learning algorithms / A. Rogulina [et al.] // International Petroleum Technology Conference. — OnePetro. 2022.

69. Shakirov, A. Automated Well Log-Based Prediction of Formation Tops: Case Study of Norway Offshore Data / A. Shakirov, A. Lipko, M. Mezghani // Abu Dhabi International Petroleum Exhibition and Conference. — SPE. 2023. — D011S022R004.

70. Marusov, A. Non-contrastive representation learning for intervals from well logs / A. Marusov, A. Zaytsev // IEEE Geoscience and Remote Sensing Letters. — 2023.

71. Startzman, R. A. A rule-based system for well log correlation / R. A. Startz-man, T.-B. Kuo // SPE Formation Evaluation. — 1987. — Vol. 2, no. 03. — P. 311—319.

72. Verma, A. K. Assessment of similarity between well logs using synchronization measures / A. K. Verma, A. Routray, W. K. Mohanty // IEEE Geoscience and Remote Sensing Letters. — 2014. — Vol. 11, no. 12. — P. 2032—2036.

73. Jong-Se, L. Interwell log correlation using artificial intelligence approach and multivariate statistical analysis / L. Jong-Se, J. M. Kang, K. Jungwhan // SPE Asia Pacific Oil and Gas Conference and Exhibition. — OnePetro. 1999.

74. Zoraster, S. Curve alignment for well-to-well log correlation / S. Zoraster, R. Paruchuri, S. Darby // SPE Annual Technical Conference and Exhibition. — OnePetro. 2004.

75. Accelerating and enhancing petrophysical analysis with machine learning: a case study of an automated system for well log outlier detection and reconstruction / R. Akkurt [et al.] // SPWLA 59th Annual Logging Symposium. — Society of Petrophysicists, Well-Log Analysts. 2018.

76. Machine learning-A novel approach of well logs similarity based on synchronization measures to predict shear sonic logs / M. Ali [et al.] // Journal of Petroleum Science and Engineering. — 2021. — Vol. 203. — P. 108602.

77. Forecasting the abnormal events at well drilling with machine learning / E. Gu-rina [et al.] // Applied Intelligence. — 2022. — P. 1—16.

78. Kaya, M. Deep metric learning: A survey / M. Kaya, H. §. Bilge // Symmetry. — 2019. — Vol. 11, no. 9. — P. 1066.

79. Time-series representation learning via temporal and contextual contrasting / E. Eldele [et al.] // Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. — 2021.

80. Byol for audio: Self-supervised learning for general-purpose audio representation / D. Niizumi [et al.] // 2021 International Joint Conference on Neural Networks (IJCNN). — IEEE. 2021. — P. 1—8.

81. The connectivity evaluation among wells in reservoir utilizing machine learning methods / S. Du [et al.] // IEEE Access. — 2020. — Vol. 8. — P. 47209—47219.

82. A machine-learning-based approach to assistive well-log correlation / S. Brazell [et al.] // Petrophysics-The SPWLA Journal of Formation Evaluation and Reservoir Description. — 2019. — Vol. 60, no. 04. — P. 469—479.

83. Do we need hundreds of classifiers to solve real world classification problems / M. Fernández-Delgado [et al.] //J. Mach. Learn. Res. — 2014. — Vol. 15, no. 1. — P. 3133—3181.

84. Chen, T. XGBoost: A scalable tree boosting system / T. Chen, C. Guestrin // Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. — ACM. 2016. — P. 785—794.

85. Hamedi Shokrlu, Y. Improving ESP Production for Unconventional Wells Through Real-Time Machine Learning Based Changepoint Detection / Y. Hamedi Shokrlu, J. Bazile // SPE Western Regional Meeting. — SPE. 2024. — D011S004R001.

86. Deep learning. Vol. 1 / I. Goodfellow [et al.]. — MIT press Cambridge, 2016. — 692 p.

87. Ahmad, M. W. Trees vs Neurons: Comparison between random forest and ANN for high-resolution prediction of building energy consumption / M. W. Ahmad, M. Mourshed, Y. Rezgui // Energy and buildings. — 2017. — Vol. 147. — P. 77—89.

88. Hornik, K. Multilayer feedforward networks are universal approximators / K. Hornik, M. Stinchcombe, H. White // Neural networks. — Oxford, UK, UK, 1989. — July. — Vol. 2, no. 5. — P. 359—366.

89. Dowell, I. A. Logging while drilling borehole imaging and dipmeter device / I. A. Dowell, D. D. Seiler, P. L. York. — 1999. — US Patent 5,899,958.

90. Leggett III, J. V. Measurement-while-drilling acoustic system employing multiple, segmented transmitters and receivers / J. V. Leggett III. — 2000. — US Patent 6,084,826.

91. Azmi, S. S. An overview of boosting decision tree algorithms utilizing Ad-aBoost and XGBoost boosting strategies / S. S. Azmi, S. Baliga // Int. Res. J. Eng. Technol. — 2020. — Vol. 7, no. 5. — P. 6867—6870.

92. Nair, V. Rectified linear units improve restricted boltzmann machines / V. Nair, G. E. Hinton // Proceedings of the 27th international conference on machine learning (ICML-10). — 2010. — P. 807—814.

93. Bai, P. Multiple change point detection in reduced rank high dimensional vector autoregressive models / P. Bai, A. Safikhani, G. Michailidis // Journal of the American Statistical Association. — 2023. — Vol. 118, no. 544. — P. 2776—2792.

94. Temporal recurrent networks for online action detection / M. Xu [et al.] // Proceedings of the IEEE/CVF International Conference on Computer Vision. — Seoul, Korea : IEEE, 2019. — P. 5532—5541.

95. Hochreiter, S. Long short-term memory / S. Hochreiter, J. Schmidhuber // Neural computation. — 1997. — Vol. 9, no. 8. — P. 1735—1780.

96. Kingma, D. P. Adam: A method for stochastic optimization / D. P. Kingma, J. Ba // arXiv preprint arXiv:1412.6980. — 2014.

97. Zhang, M. USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors / M. Zhang, A. A. Sawchuk // Proceedings of the 2012 ACM Conference on Ubiquitous Computing. — Pittsburgh, Pennsylvania : ACM, 2012. — P. 1036—1043.

98. LeCun, Y. MNIST handwritten digit database / Y. LeCun, C. Cortes. — 2010. — URL: http://yann.lecun.com/exdb/mnist/.

99. Sohn, K. Learning structured output representation using deep conditional generative models / K. Sohn, H. Lee, X. Yan // Advances in neural information processing systems. — 2015. — Vol. 28. — P. 3483—3491.

100. Slowfast networks for video recognition / C. Feichtenhofer [et al.] // Proceedings of the IEEE/CVF international conference on computer vision. — 2019. — P. 6202—6211.

101. Van der Maaten, L. Visualizing data using t-SNE. / L. Van der Maaten, G. Hinton // Journal of machine learning research. — 2008. — Vol. 9, no. 11.

102. Data labeling: an empirical investigation into industrial challenges and mitigation strategies / T. Fredriksson [et al.] // International Conference on Product-Focused Software Process Improvement. — Springer. 2020. — P. 202—216.

103. Truong, C. Supervised change-point detection with dimension reduction, applied to physiological signals / C. Truong, L. Oudre // NeurIPS 2022 Workshop on Learning from Time Series for Health. — 2022.

104. A survey on contrastive self-supervised learning / A. Jaiswal [et al.] // Technologies. — 2021. — Vol. 9, no. 1. — P. 2.

105. Self-supervised learning: Generative or contrastive / X. Liu [et al.] // IEEE Transactions on Knowledge and Data Engineering. — 2021.

106. Self-supervised learning for time series analysis: Taxonomy, progress, and prospects / K. Zhang [et al.] // IEEE Transactions on Pattern Analysis and Machine Intelligence. — 2024.

107. Big self-supervised models are strong semi-supervised learners / T. Chen [et al.] // Advances in neural information processing systems. — 2020. — Vol. 33. — P. 22243—22255.

108. A simple framework for contrastive learning of visual representations / T. Chen [et al.] // International conference on machine learning. — PMLR. 2020. — P. 1597—1607.

109. BERT: Pre-training of deep bidirectional transformers for language understanding / J. Devlin [et al.] //. — 2018.

110. Universal representations for financial transactional data: embracing local, global, and external contexts / A. Bazarova [et al.] // arXiv preprint arXiv:2404.02047. — 2024.

111. Tclr: Temporal contrastive learning for video representation / I. Dave [et al.] // Computer Vision and Image Understanding. — 2022. — Vol. 219. — P. 103406.

112. Ts2vec: Towards universal representation of time series / Z. Yue [et al.] // Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. — 2022. — P. 8980—8987.

113. Coles: Contrastive learning for event sequences with self-supervision / D. Babaev [et al.] // Proceedings of the 2022 International Conference on Management of Data. — 2022. — P. 1190—1199.

114. Negative sampling for contrastive representation learning: A review / L. Xu [et al.] // arXiv preprint arXiv:2206.00212. — 2022.

115. Arcface: Additive angular margin for deep face recognition / J. Deng [et al.] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. — 2019. — P. 4690—4699.

116. Wang, F. Understanding the behaviour of contrastive loss / F. Wang, H. Liu // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. — 2021. — P. 2495—2504.

117. An introduction to variational autoencoders / D. P. Kingma, M. Welling, [et al.] // Foundations and Trends® in Machine Learning. — 2019. — Vol. 12, no. 4. — P. 307—392.

118. Kullback, S. On information and sufficiency / S. Kullback, R. A. Leibler // The annals of mathematical statistics. — 1951. — Vol. 22, no. 1. — P. 79—86.

119. The elements of statistical learning. Vol. 1 / J. Friedman, T. Hastie, R. Tib-shirani, [et al.]. — Springer series in statistics New York, 2001. — 745 p.

120. Snorovikhina, V. Unsupervised anomaly detection for discrete sequence healthcare data / V. Snorovikhina, A. Zaytsev // Analysis of Images, Social Networks and Texts: 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15-16, 2020, Revised Selected Papers 9. — Springer. 2021. — P. 391—403.

121. LSTM: A search space odyssey / K. Greff [et al.] // IEEE transactions on neural networks and learning systems. — 2016. — Vol. 28, no. 10. — P. 2222—2232.

122. Bazarova, A. Normalizing self-supervised learning for provably reliable Change Point Detection / A. Bazarova, E. Romanenkova, A. Zaytsev // arXiv preprint arXiv:2410.13637. — 2024.

123. Adjusting for chance clustering comparison measures / S. Romano [et al.] // Journal of Machine Learning Research. — 2016. — Vol. 17, no. 134. — P. 1—32.

124. Murtagh, F. Ward's hierarchical agglomerative clustering method: which algorithms implement Ward's criterion? / F. Murtagh, P. Legendre // Journal of classification. — 2014. — Vol. 31, no. 3. — P. 274—295.

125. Koroteev, D. Artificial intelligence in oil and gas upstream: Trends, challenges, and scenarios for the future / D. Koroteev, Z. Tekic // Energy and AI. — 2021. — Vol. 3. — P. 100041.

126. IBM research. The New Zealand Petroleum & Minerals Online Exploration Database / IBM research. — 2015. — URL: https://data.nzpam.govt.nz/ GOLD/system/mainframe.asp ; Last accessed 12 June 2024.

127. NPD. FactPages / NPD. — 2021. — URL: https://factpages.npd.no/en/ wellbore ; Last accessed 12 June 2024.

128. Hermans, A. In defense of the loss for person re-identification / A. Hermans, L. Beyer, B. Leibe // arXiv preprint arXiv:1703.07737. — 2017.

129. Time series feature extraction on basis of scalable hypothesis tests (tsfresh-a python package) / M. Christ [et al.]. — 2018.

130. Krizhevsky, A. Imagenet classification with deep convolutional neural networks / A. Krizhevsky, I. Sutskever, G. E. Hinton // Advances in neural information processing systems. — 2012. — Vol. 25.

131. Robust Representation Learning via Sparse Attention Mechanism for Similarity Models / A. Ermilova [et al.] // IEEE Access. — 2024. — Vol. 12. — P. 97833—97850.

132. Ba, J. L. Layer normalization / J. L. Ba, J. R. Kiros, G. E. Hinton // arXiv preprint arXiv:1607.06450. — 2016.

133. Корябкин, В. В. Цифровые инструменты «Газпром нефти» повысят эффективность бурения скважин / В. В. Корябкин. — 2019. — URL: https: / / ntc. gazprom- neft. ru / press- center / news / tsifrovye- instrumenty- gazprom-nefti-povysyat-effektivnost-bureniya-skvazhin/ ; Last accessed 12 June 2024.

Publications of the Author on the Subject of the Dissertation

134. Application of machine learning to accidents detection at directional drilling / E. Gurina [et al.] // Journal of Petroleum Science and Engineering. — 2020. — Vol. 184. — P. 106519.

135. Real-time data-driven detection of the rock-type alteration during a directional drilling / E. Romanenkova [et al.] // IEEE Geoscience and Remote Sensing Letters. — 2019. — Vol. 17, no. 11. — P. 1861—1865.

136. InDiD: Instant Disorder Detection via a Principled Neural Network / E. Romanenkova [et al.] // Proceedings of the 30th ACM International Conference on Multimedia. — 2022. — P. 3152—3162.

137. Data-driven model for the drilling accidents prediction / K. Antipova [et al.] // SPE Annual Technical Conference and Exhibition. — SPE. 2019. — D012S067R001.

138. Similarity learning for wells based on logging data / E. Romanenkova [et al.] // Journal of Petroleum Science and Engineering. — 2022. — Vol. 215. — P. 110690.

139. Universal representations for well-logging data via ensembling of self-supervised models / V. Zholobov [et al.] // Doklady Mathematics. Vol. 110. — Springer. 2024. — S126—S136.

140. Свидетельство о гос. регистрации программы для ЭВМ. РПСП в процессе бурения. Очередь 1 / К. Антипова [и др.] ; О. с ограниченной ответственностью «ЛУКОЙЛ-Технологии». — № 2023612725 ; заявл. 25.01.2023 ; опубл. 07.02.2023, 2023611062 (Russia).