Publications

A deep learning based dual encoder–decoder framework for anatomical structure segmentation in chest X-ray images

Published in Scientific Reports, 2023

Automated multi-organ segmentation plays an essential part in the computer-aided diagnostic (CAD) of chest X-ray fluoroscopy. However, developing a CAD system for the anatomical structure segmentation remains challenging due to several indistinct structures, variations in the anatomical structure shape among different individuals, the presence of medical tools, such as pacemakers and catheters, and various artifacts in the chest radiographic images. In this paper, we propose a robust deep learning segmentation framework for the anatomical structure in chest radiographs that utilizes a dual encoder–decoder convolutional neural network (CNN). The first network in the dual encoder–decoder structure effectively utilizes a pre-trained VGG19 as an encoder for the segmentation task. The pre-trained encoder output is fed into the squeeze-and-excitation (SE) to boost the network’s representation power, which enables it to perform dynamic channel-wise feature calibrations. The calibrated features are efficiently passed into the first decoder to generate the mask. We integrated the generated mask with the input image and passed it through a second encoder–decoder network with the recurrent residual blocks and an attention the gate module to capture the additional contextual features and improve the segmentation of the smaller regions. Three public chest X-ray datasets are used to evaluate the proposed method for multi-organs segmentation, such as the heart, lungs, and clavicles, and single-organ segmentation, which include only lungs. The results from the experiment show that our proposed technique outperformed the existing multi-class and single-class segmentation methods.

Recommended citation: Ihsan Ullah et al. (2023). A deep learning based dual encoder–decoder framework for anatomical structure segmentation in chest X-ray images; Scientific Reports. 1(1). https://www.nature.com/articles/s41598-023-27815-w

Classification of the Confocal Microscopy Images of Colorectal Tumor and Inflammatory Colitis Mucosa Tissue Using Deep Learning.

Published in Diagnostics, 2022

Confocal microscopy image analysis is a useful method for neoplasm diagnosis. Many ambiguous cases are difficult to distinguish with the naked eye, thus leading to high inter-observer variability and significant time investments for learning this method. We aimed to develop a deep learning-based neoplasm classification model that classifies confocal microscopy images of 10× magnified colon tissues into three classes: neoplasm, inflammation, and normal tissue. ResNet50 with data augmentation and transfer learning approaches was used to efficiently train the model with limited training data. A class activation map was generated by using global average pooling to confirm which areas had a major effect on the classification. The proposed method achieved an accuracy of 81%, which was 14.05% more accurate than three machine learning-based methods and 22.6% better than the predictions made by four endoscopists. ResNet50 with data augmentation and transfer learning can be utilized to effectively identify neoplasm, inflammation, and normal tissue in confocal microscopy images. The proposed method outperformed three machine learning-based methods and identified the area that had a major influence on the results. Inter-observer variability and the time required for learning can be reduced if the proposed model is used with confocal microscopy image analysis for diagnosis.

Recommended citation: Ihsan Ullah et al. (2022). Classification of the Confocal Microscopy Images of Colorectal Tumor and Inflammatory Colitis Mucosa Tissue Using Deep Learning; Diagnostics. 1(1). https://ihsan149.github.io/files/confocal_microscopy.pdf

Machine Learning-Enabled Power Scheduling in IoT-Based Smart Cities.

Published in Computers, Materials & Continua, 2021

Recent advancements in hardware and communication technologies have enabled worldwide interconnection using the internet of things (IoT). The IoT is the backbone of smart city applications such as smart grids and green energy management. In smart cities, the IoT devices are used for linking power, price, energy, and demand information for smart homes and home energy management (HEM) in the smart grids. In complex smart grid-connected systems, power scheduling and secure dispatch of information are the main research challenge. These challenges can be resolved through various machine learning techniques and data analytics. In this paper, we have proposed a particle swarm optimization based machine learning algorithm known as a collaborative execute-before-after dependency-based requirement, for the smart grid. The proposed collaborative execute-before-after dependency-based requirement algorithm works in two phases, analysis and assessment of the requirements of end-users and power distribution companies. In the first phases, a fixed load is adjusted over a period of 24 h, and in the second phase, a randomly produced population load for 90 days is evaluated using particle swarm optimization. The simulation results demonstrate that the proposed algorithm performed better in terms of percentage cost reduction, peak to average ratio, and power variance mean ratio than particle swarm optimization and inclined block rate.

Recommended citation: Ihsan Ullah et al. (2021). Machine Learning-Enabled Power Scheduling in IoT-Based Smart Cities; Computers, Materials & Continua . 1(1). https://ihsan149.github.io/files/iot.pdf

Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture.

Published in Sensors, 2019

Make and model recognition (MMR) of vehicles plays an important role in automatic vision-based systems. This paper proposes a novel deep learning approach for MMR using the SqueezeNet architecture. The frontal views of vehicle images are first extracted and fed into a deep network for training and testing. The SqueezeNet architecture with bypass connections between the Fire modules, a variant of the vanilla SqueezeNet, is employed for this study, which makes our MMR system more efficient. The experimental results on our collected large-scale vehicle datasets indicate that the proposed model achieves 96.3% recognition rate at the rank-1 level with an economical time slice of 108.8 ms. For inference tasks, the deployed deep model requires less than 5 MB of space and thus has a great viability in real-time applications.

Recommended citation: Ihsan Ullah et al. (2019). Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture; Sensons. 1(1). https://ihsan149.github.io/files/vehicle_type.pdf

Real-Time Tracking of Guidewire Robot Tips Using Deep Convolutional Neural Networks on Successive Localized Frames.

Published in IEEE Access, 2019

Studies are proceeded to stabilize cardiac surgery using thin micro-guidewires and catheter robots. To control the robot to a desired position and pose, it is necessary to accurately track the robot tip in real time but tracking and accurately delineating the thin and small tip is challenging. To address this problem, a novel image analysis-based tracking method using deep convolutional neural networks (CNN) has been proposed in this paper. The proposed tracker consists of two parts; (1) a detection network for rough detection of the tip position and (2) a segmentation network for accurate tip delineation near the tip position. To learn a robust real-time tracker, we extract small image patches, including the tip in successive frames and then learn the informative spatial and motion features for the segmentation network. During inference, the tip bounding box is first estimated in the initial frame via the detection network, thereafter tip delineation is consecutively performed through the segmentation network in the following frames. The proposed method enables accurate delineation of the tip in real time and automatically restarts tracking via the detection network when tracking fails in challenging frames. Experimental results show that the proposed method achieves better tracking accuracy than existing methods, with a considerable real-time speed of 19ms.

Recommended citation: Ihsan Ullah et al. (2019). Real-Time Tracking of Guidewire Robot Tips Using Deep Convolutional Neural Networks on Successive Localized Frames; IEEE Access. 1(1). https://ihsan149.github.io/files/catheter_tracking.pdf

Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation.

Published in Applied Sciences, 2009

Automatic catheter and guidewire segmentation plays an important role in robot-assisted interventions that are guided by fluoroscopy. Existing learning based methods addressing the task of segmentation or tracking are often limited by the scarcity of annotated samples and difficulty in data collection. In the case of deep learning based methods, the demand for large amounts of labeled data further impedes successful application. We propose a synthesize and segment approach with plug in possibilities for segmentation to address this. We show that an adversarially learned image-to-image translation network can synthesize catheters in X-ray fluoroscopy enabling data augmentation in order to alleviate a low data regime. To make realistic synthesized images, we train the translation network via a perceptual loss coupled with similarity constraints. Then existing segmentation networks are used to learn accurate localization of catheters in a semi-supervised setting with the generated images. The empirical results on collected medical datasets show the value of our approach with significant improvements over existing translation baseline methods.

Recommended citation: Ihsan Ullah et al. (2021). Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation; Applied Sciences. 1(1). https://ihsan149.github.io/files/catheter_synthesis.pdf

Deep learning-based segmentation and classification of leaf images for detection of tomato plant disease.

Published in Frontiers in Plant Science, 2009

Plants contribute significantly to the global food supply. Various Plant diseases can result in production losses, which can be avoided by maintaining vigilance. However, manually monitoring plant diseases by agriculture experts and botanists is time-consuming, challenging and error-prone. To reduce the risk of disease severity, machine vision technology (i.e., artificial intelligence) can play a significant role. In the alternative method, the severity of the disease can be diminished through computer technologies and the cooperation of humans. These methods can also eliminate the disadvantages of manual observation. In this work, we proposed a solution to detect tomato plant disease using a deep leaning-based system utilizing the plant leaves image data. We utilized an architecture for deep learning based on a recently developed convolutional neural network that is trained over 18,161 segmented and non-segmented tomato leaf images—using a supervised learning approach to detect and recognize various tomato diseases using the Inception Net model in the research work. For the detection and segmentation of disease-affected regions, two state-of-the-art semantic segmentation models, i.e., U-Net and Modified U-Net, are utilized in this work. The plant leaf pixels are binary and classified by the model as Region of Interest (ROI) and background. There is also an examination of the presentation of binary arrangement (healthy and diseased leaves), six-level classification (healthy and other ailing leaf groups), and ten-level classification (healthy and other types of ailing leaves) models. The Modified U-net segmentation model outperforms the simple U-net segmentation model by 98.66 percent, 98.5 IoU score, and 98.73 percent on the dice. InceptionNet1 achieves 99.95% accuracy for binary classification problems and 99.12% for classifying six segmented class images; InceptionNet outperformed the Modified U-net model to achieve higher accuracy. The experimental results of our proposed method for classifying plant diseases demonstrate that it outperforms the methods currently available in the literature.

Recommended citation: Ihsan Ullah et al. (2022). Deep learning-based segmentation and classification of leaf images for detection of tomato plant disease; Frontiers in Plant Science. 1(1). https://ihsan149.github.io/files/fpls.pdf

Ihsan Ullah Khan, PhD