Special issue on deep learning modeling in real life: anomaly detection, biomedical, concept analysis, finance, image analysis, recommendation

Abstract

Machine learning (ML) and more specifically deep learning (DL) algorithms are considered among the most paramount technologies of both artificial intelligence (AI) and 4th industrial revolution. As such, they undoubtedly have significant influence in our daily and professional lives. Artificial intelligence is no longer a futuristic concept. Its algorithms are widely employed to mimic human reasoning. They are utilizing both structured and unstructured data, in order to achieve extremely complex tasks, with high efficiency and accuracy. The industrial and research applications of AI are related but not limited to Self-Driving cars, Natural Language, Visual and Speech Recognition, Fraud Detection, Cyber Security, Healthcare and several other real-life domains. The potential applications are endless, and they have a positive impact to our quality of life. This is the Editorial of the “Deep Learning Modeling in Real Life: Anomaly Detection, Biomedical, Concept Analysis, Finance, Image analysis, Recommendation” Issue of the Neural Computing and Applications (NCA) Journal. It presents timely cases of emerging deep learning advances, and applications in a wide range of scientific and engineering areas. Overall, 36 original research papers were submitted to be considered for publication in this Special Issue of the NCA Journal. Only sixteen (16) of them (44.4%) have been carefully selected for publication, after passing successfully through a peer review process by independent academic referees. All these high-quality papers are presenting innovative research, falling in the scientific domain that was specified by the journal’s call. The first paper is entitled “Intelligent fault diagnosis of rolling bearings based on LSTM with Large Margin Nearest Neighbor algorithm” and it is authored by Anas H. Aljemely, Jianping Xuan, Osama Al-Azzawi, Farqad K. J. Jawad, of Huazhong University of Science and Technology, Wuhan, People’s Republic of China. This paper introduces a hybrid model, that employs long short-term memory (LSTM) combined with a large margin nearest neighbor (LMNN) approach. The proposed model effectively recognizes multi-faults in mechanical rotating machines due to excessive working stress. Different from traditional LSTMs, the proposed LSTM-LMNN utilizes a powerful orthogonal weight initialization technique. Moreover, it manages to memorize the critical information of faults during parameters’ updating. The second paper is entitled “Neural intuitionistic fuzzy system with justified granularity” and it is authored by Peter Hajek, University of Pardubice, Czech Republic, Wojciech Froelich, University of Silesia, Sosnowiec, Poland, Vladimir Olej, & Josef Novotny, University of Pardubice, Czech Republic. This research effort proposes a novel robust hybrid model, which is merging Fuzzy reasoning with Neural Networks and with the Justified Granularity Optimization technique (FNN_JGO). The developed model has been used as an innovative time series forecasting approach. It was successfully tested in metal price forecasting. The paper provides evidence that the FNN_JGO is competitive with the current state-of-the-art methods. The third paper is entitled “Multisource financial sentiment analysis for detecting Bitcoin price change indications using Deep Learning” and the authors are: Nikolaos Passalis, Loukia Avramelou, Solon Seficha, Avraam Tsantekidis, from the Computational Intelligence and Deep Learning Group, Department of Informatics, Aristotle University of Thessaloniki, Greece, Stavros Doropoulos and Giorgos Makris, from the DataScouting, Thessaloniki, Greece, and Anastasios Tefas, from the Computational Intelligence and Deep Learning Group, Department of Informatics, Aristotle University of Thessaloniki, Greece. This research effort is inspired by the fact that deep learning (DL)-based trading methods typically rely on a very restricted price-related set of information. As a result, they ignore sentiment-related information, which can have a profound impact and serve as a strong predictor of various assets, such as cryptocurrencies. This paper has a twofold contribution to the literature. First, it examines whether the use of sentiment information, including news articles, is beneficial when training DL agents for trading. Then, given the difficulty of training reliable sentiment extractors for financial applications, the paper evaluates the impact of using different DL models as sentiment extractors. Moreover, the authors employ an unsupervised training pipeline for further improving the performance. Overall, they propose an effective multi-source sentiment fusion approach that has the potential to improve the performance over the rest of the evaluated approaches. The sixth paper is entitled: “Deep autoencoders for acoustic anomaly detection: experiments with working machine and in-vehicle audio” and it has been authored by Gabriel Coelho, Luís Miguel Matos, Pedro José Pereira, and Paulo Cortez ALGORITMI Centre, Department of Information Systems, University of Minho, Guimarães, Portugal, André Ferreira, Bosch Car Multimedia, Braga, Portugal, André Pilastri, EPMQ—IT Engineering Maturity and Quality Lab, CCG ZGDV Institute, Guimarães, Portugal. This interesting paper introduces three deep autoencoders (AE), namely a dense AE, a convolutional neural network AE and a long short-term memory one. They were all developed to successfully perform unsupervised Acoustic Anomaly Detection (AAD) tasks. Development data related to working machines were adopted from public domain audio datasets, in order to perform tuning of the aforementioned architectures. The testing experiments have shown that the proposed Deep AE, when combined with melspectrogram sound preprocessing, outperform a recently proposed AE baseline. Georgios Theodoridis and Athanasios Tsadiras from the Aristotle University of Thessaloniki, Greece, have authored the seventh paper which is entitled “Applying machine learning techniques to predict and explain subscriber churn of an online drug information platform.” This paper provides an in-depth comparison of various machine learning (ML) techniques and advanced preprocessing methods, in an effort to successfully perform online subscriber churn prediction. Moreover, the authors present an overall guide for handling the problem. Numerous methods that belong to different ML categories have been employed to develop a binary classification model based on a real-world dataset originating from a popular online drug information platform. This platform provides information on drugs and drug substances as well as professional tools for pharmacotherapy decision making. In contrast to previous works that address traditional customer churn in relation to telecom, banking or insurance industries, the current study addresses online subscriber churn where users might churn at any given moment. Yannis Kontos, Theodosios Kassandros, Marios Karampasis, Konstantinos Katsifarakis Kostas Karatzas, from the Aristotle University of Thessaloniki Greece, and Kostantinos Perifanos from the National and Kapodistrian University of Athens, Greece, have authored the paper “Machine learning for groundwater pollution source identification and monitoring network optimization.” The contamination of the water table and the water potential of our planet is one of the most important problems that requires immediate action. The aim of this research is the identification of the source in groundwater pollution, following the inverse modeling approach, i.e., identifying the source of the pollutant on the basis of measurements taken within the pollution field. The authors of this paper have developed numerous Classification and Computer Vision models (e.g., random forests, multi-layer perceptrons, convolutional neural networks). For this reason, a theoretical confined aquifer with two pumping wells and six suspected sources is studied. Simulations of combinations of possible source locations, and hydraulic parameters, produce sets of measurement features for a 29 × 29 grid representing potential monitoring wells. The ninth paper is entitled “Self-organizing maps for cultural content delivery,” and it has been authored by Georgios Drakopoulos, Phivos Mylonas from the Ionian University, Greece, and Ioanna Giannoukou, Spyros Sioutas from the University of Patras, Greece. This is an interesting research effort on tailored cultural analytics, which play a key role in the successful delivery of cultural content to huge and diverse groups. It aims in the development of a tensor user distance metric to be used for self-organizing maps (SOMs), and it includes behavioral attributes, both aiming in the enhancement of the descriptive power and clustering flexibility. The introduced SOMs are applied to data taken from a cultural content delivery system. The proposed model is evaluated based on a scoring method assessing both complexity and clustering quality criteria, including the number of epochs, the average cluster distance, and the topological error. The results are encouraging. The tenth paper is authored by Anastasios Panagiotis Psathas, Lazaros Iliadis, Antonios Papaleonidas, and Dimitris Bountas from the Lab of Mathematics and Informatics, School of Engineering, Democritus University of Thrace, Greece. It is entitled “COREM project: a beginning to end approach for cyber intrusion detection.” Network Security (NS) is a hot and timely topic. This research effort aims to tackle NS problems, by introducing the hybrid intrusion detection system (COREM) that has the capacity to detect nine different cyber-attacks. Its architecture comprises of a two-dimensional convolutional neural network (CNN) a recurrent neural network with long short-term memory and a typical multi-layer perceptron. The introduced hybrid model has been successfully tested against the timely Kitsune Network Attack dataset. Hongfei Jia from the School of Artificial Intelligence, Beijing Technology and Business University, Beijing, People’s Republic of China, and Huan Lao from the School of Artificial Intelligence, Guangxi Minzu University, Nanning, Guangxi, People’s Republic of China, have authored the eleventh paper entitled: “Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease.” Alzheimer’s disease is a very serious problem of modern societies, especially met in the elderly. This paper introduces a diagnostic model that effectively diagnoses in fourteen different stages, by fusing functional magnetic resonance imaging (fMRI) and structural MRI (sMRI) information. Several fMRI and sMRI scans are preprocessed, and mean regional homogeneity transformation is performed for the preprocessed fMRI scans. The basic ResNet module is stacked to build a 3DResNet-10 model for feature extraction of sMRI scans. Next, two image features are fused by kernel canonical correlation analysis. Finally, a support vector machine is utilized for the classification of fused features. The 12th paper is entitled: “Consolidating incentivization in distributed neural network training via decentralized autonomous organization,” and it is authored by Spyridon Nikolaidis and Ioannis Refanidis, from the Department of Applied Informatics, University of Macedonia, Thessaloniki, Greece. Deep neural networks have benefited greatly from the unprecedented data availability. Large models with millions of parameters are becoming common, and big data have been proven to be essential for their effective training. The scientific community has come up with several methods to create more accurate models, but most of them require high-performance infrastructure. There is also the issue of privacy, since anyone using leased processing power from a remote data center is putting their data in the hands of a third party. Studies on decentralized and non-binding methods among individuals with commodity hardware are scarce. LEARNAE is a novel ecosystem of interacting distributed technologies, that enable individual machine learning researchers to collaborate in a fully decentralized and democratized environment. This paper seeks to respond to the above challenges, by introducing a totally distributed and fault-tolerant framework of Artificial Neural Networks’ training. It proposes a decentralized mechanism to mitigate the effect of bad actors, such as nodes, that attempt to exploit LEARNAE’s network power without following the established rewarding rules. The 13th paper NCAA-D-21-04898R2 is authored by Sotiria Vernikou, Athanasios Lyras from the Computer Engineering and Informatics Department, University of Patras, Greece, and Andreas Kanavos from the Department of Digital Media and Communication, Ionian University, Kefalonia, Greece. It is entitled “Multiclass Sentiment Analysis on COVID-19 related Tweets using Deep Learning Models.” This paper discusses a sentiment analysis effort, focusing on the classification of users’ sentiment from posts related to COVID-19 originating from Twitter. The period examined is from March until mid-April of 2020, when the pandemic had thus far affected the whole world. The data are processed and linguistically analyzed with the use of several Natural Language processing techniques. Sentiment analysis is implemented by utilizing seven different deep learning models based on LSTM neural networks. The model distinguishes the tweets in three classes, namely negative, neutral and positive. The 14th paper NCAA-D-21-05083R2 is authored by Olympia Giannou, and Georgios Pavlidis from the Computer Engineering & Informatics Department, University of Patras, Greece, Anastasios Giannou, Department of Medicine, University Medical Center Hamburg-Eppendorf, Germany, and Department of General, Visceral, Thoracic Surgery, UKE, Hamburg, Germany, and Dimitra Zazara, from the Division for Experimental Feto-Maternal Medicine and Department of Obstetrics and Fetal Medicine, UKE, Hamburg, Germany, and Department of Pediatrics, UKE, Hamburg, Germany. It is entitled “Automated distinction of neoplastic from healthy liver parenchyma based on machine learning.” Liver segmentation is a basic and important procedure in liver transplantation surgery as well as in liver volumetric assessment. Common clinical practice follows a time-consuming manual delineation of liver regions. Scanning and using Computed Tomography (CT) slices of liver cancer of different patients as input datasets aims to provide an automatic way to recognize liver tissue among other organs, to assess its volume, and to detect and measure the volume of hepatocellular carcinoma. The proposed tool for an automatic analysis of liver volumetry could be applied in several clinical cases, especially as a fast accurate indicator of healthy liver parenchyma before a major hepatectomy is performed so that the remaining healthy tissue and thus disease prognosis after tumor resection can be predicted. This timely research paper introduces a hybrid machine learning model that employs the ResUNet (Deep Residual UNET) enhanced with one additional residual block in the encoder and one additional block in the decoder (En-ResUNet). The employed ResUnet combines the advantages of deep residual learning and U-Net which is a convolutional neural network that was developed for biomedical image segmentation. The results are very promising. Panagiotis Mavrogiannis and Ilias Maglogiannis from the Department of Digital Systems, University of Piraeus, Greece, have authored the 15th paper entitled “Amateur Football Analytics using Computer Vision.” This is an interesting paper on visual sports analytics and especially in player and ball detection, action recognition, and camera pose estimation in various sports. It discusses the state of the art in this field and proposes an integrated pipeline that solves the problem of player localization in the court and extracts useful insights. Technical details concerning the proposed methods that enhance track player-ball movement and camera pose, along with post-analytic results, are presented. Finally, the 16th paper “Known and Unknown Event Detection in OTDR Traces by Deep Learning Networks” has been authored by Antonino Maria Rizzo, Luca Magri, Giacomo Boracchi, Department of Electronics, Informatics and Bioengineering, Polytechnic University of Milano, Milano, Italy, Pietro Invernizzi, Enrico Sozio, Stefano Binetti, Cisco Photonics, Cisco Systems, Vimercate, Italy, Cesare Alippi, Department of Electronics, Informatics and Bioengineering, Polytechnic University of Milano Italy, and Department, Universit`a della Svizzera Italiana, Lugano, Switzerland, Davide Rutigliano, Department of Electronics, Informatics and Bioengineering, Polytechnic University of Milano, Milano, Italy, and Cisco Photonics, Cisco Systems, Vimercate, Italy. Optical fiber links are customarily monitored by Optical Time Domain Reflectometer (OTDR), an optoelectronic instrument that measures the scattered or reflected light along the fiber and returns a signal, namely the OTDR trace. OTDR traces are typically analyzed by experts in laboratories or by hand-crafted algorithms running in embedded systems to localize critical events occurring along the fiber. This research effort addresses the problem of automatically detecting optical events in OTDR traces through a deep learning model that can be deployed in embedded systems. We wish to thank the Editor in Chief of the NCA journal Professor John Macintyre for offering the chance to edit this Issue. It was a honor and a real pleasure to work on this domain in this high-quality scientific journal. The large volume of submissions proves the interest of the international scientific community in deep learning modeling. All timely modeling efforts presented in this Issue of the Neural Computing and Applications Journal are based on robust theoretical foundations, and they have good practical applications in several diverse scientific domains. We hope that they will motivate and inspire further research for the benefit of our societies.