MaskCD: A Remote Sensing Change Detection Network Based on Mask Classification
Change detection from remote sensing images using deep learning has been widely investigated in the literature. It is typically regarded as a pixel-wise labeling task that aims to classify each pixel as changed or unchanged. Although per-pixel classification networks in encoder-decoder structures have shown dominance, they still suffer from imprecise boundaries and incomplete object delineation at various scenes. For high-resolution RS images, partly or totally changed objects are more worthy of attention rather than a single pixel. Therefore, we revisit the CD task from the mask prediction and classification perspective and propose MaskCD to detect changed areas by adaptively generating categorized masks from input image pairs.
Related Work:
Yu, W., Zhang, X., Das, S., Zhu, X. X., and Ghamisi, P., 2024. MaskCD: A Remote Sensing Change Detection Network Based on Mask Classification. arXiv preprint.
SpectralGPT: Spectral Remote Sensing Foundation Model
The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created a universal RS foundation model named SpectralGPT for the first time. It is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT).
Related Work:
Hong, D., Zhang, B., Li, X., Li, Y., Li, C., Yao, J., Yokoya, N., Li, H., Ghamisi, P., Jia, X. and Plaza, A., 2024. SpectralGPT: Spectral remote sensing foundation model. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Universal Adversarial Defense
Deep neural networks have shown remarkable success in remote sensing applications. However, their vulnerability to adversarial perturbations is a critical concern. To address this challenge, our research introduces a novel Universal Adversarial Defense approach (UAD-RS) that uses pre-trained diffusion models to protect common DNNs against unknown adversarial attacks.
Related Work:
W. Yu, Y. Xu, and P. Ghamisi, "Universal Adversarial Defense in Remote Sensing Based on Pre-trained Denoising Diffusion Models," arXiv preprint arXiv:2307.16865, 2023.
Hyperspectral Point Clouds Segmentation
We designed a fusion model for hyperspectral point clouds segmentation. Although there are numerous studies on point cloud segmentation, most of them focus on the typical point cloud data. Segmentation on hyperspectral point cloud data, however, has rarely been investigated. In this study, we present how to design a multi-stream network for multi-modal point cloud.
Related Work:
A. Rizaldy, A.J. Afifi, R. Gloaguen, P. Ghamisi, "Segmentation of Hyperspectral Point Clouds of Open-pit Mines with Multi-Stream Graph Network" under preparation, 2023.
3D Point Clouds Classification and Segmentation
In AI4RS, we also cover researchs on 3D point clouds. Unlike image classification, point cloud classification has different challenges due to the irregularity, unstructured, and unorderedness of the data. Leveraging by the advanced development of deep neural networks, we conducted researchs on various point cloud datasets.
Text-to-Image Generation
Although deep neural networks have achieved great success in many important remote sensing tasks, generating realistic remote sensing images from text descriptions is still very difficult. To address this challenge, we develop novel algorithms for high-resolution remote sensing image synthesis from text descriptions.
Related Work:
Y. Xu, W. Yu, P. Ghamisi, M. Kopp, S. Hochreiter, "Txt2Img-MHN: Remote sensing image generation from text using modern hopfield networks," arXiv preprint arXiv:2208.04441, 2022.
Image/Change Captioning
As an interdisciplinary field at the intersection of CV and NLP, image captioning has gained significant attention in recent years, with many researchers applying advanced artificial intelligence techniques to improve the state of the art. To address the challenge of generating accurate captions, we conduct interesting research for high-resolution remote sensing images.
Related Work:
S. Chang and P. Ghamisi, "Changes to Captions: An Attentive Network for Remote Sensing Change Captioning," arXiv preprint arXiv:2304.01091, 2023.
Adversarial Attacks and Defenses
While deep learning algorithms have achieved state-of-the-art performance in many remote sensing interpretation tasks, their vulnerability to adversarial examples should not be neglected. To tackle this challenge, we develop novel adversarial attack and defense methods and collect benchmark datasets for remote sensing images.
Related Work:
Y. Xu, P. Ghamisi, "Universal adversarial examples in remote sensing: Methodology and benchmark," IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1−15, 2022.
Natural Hazards Monitoring
Natural hazards are naturally occurring phenomena of geological, hydrological, or meteorological origin that might harm humans or the environment. To monitor and analyze natural hazards like landslides, we develop novel algorithms and collect benchmark datasets with multi-source remote sensing data.
Related Work:
O. Ghorbanzadeh, Y. Xu, P. Ghamisi, M. Kopp, D. Kreil, "Landslide4sense: Reference benchmark data and deep learning models for landslide detection," arXiv preprint arXiv:2206.00515, 2022.
AI for Social Good
New developments and a plethora of applications are already changing the way humanity will face the challenges of our planet. To better monitor and achieve the sustainable development goals, we develop novel AI approaches for social good.
Related Work:
C. Persello, J. Dirk Wegner, R. Hänsch, D. Tuia, P. Ghamisi, M. Koeva, G. Camps-valls, "Deep learning and earth observation to support the sustainable development goals: Current approaches, open challenges, and future opportunities," IEEE Geosci. Remote Sens. Mag., 2022.
Image Restoration
The quality of data acquired by remotely sensed imaging sensors (active and passive) is often degraded by a variety of noise types and artifacts. To tackle this challenge, we develop novel image restoration methods for remote sensing data.
Related Work:
B. Rasti, Y. Chang, E. Dalsasso, L. Denis, P. Ghamisi, "Image restoration for remote sensing: Overview and toolbox," IEEE Geosci. Remote Sens. Mag., 2021.
Spectral Unmixing
Due to the limited spatial resolution and scattering of the light, a pixel spectrum is generally a complex mixture of the pure spectra of its constituent materials, i.e., the endmember spectra. To accurately estimate the fractional abundances of those endmembers, we develop advanced deep learning-based unmixing methods.
Related Work:
B. Rasti, B. Koirala, P. Scheunders, P. Ghamisi, "Undip: Hyperspectral unmixing using deep image prior," IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1−15, 2022.
Data Clustering
Traditional clustering models essentially work in the Euclidean domain, leading to insufficient handling of graph-structured data. To tackle this challenge, we develop novel graph neural network-based approaches for high-dimensional remote sensing data.
Related Work:
Y. Cai, Z. Zhang, Z. Cai, X. Liu, Y. Ding, P. Ghamisi, "Fully linear graph convolutional networks for semi-supervised learning and clustering," arXiv preprint arXiv:2111.07942, 2021.
Learning with Sparse Annotations
Most of the existing interpretation approaches, especially deep learning-based ones, require a large number of high-quality labeled samples, which are expensive and time-consuming to be collected in practice. To tackle this challenge, we develop novel algorithms that can achieve satisfactory performance with sparse annotations.
Related Work:
Y. Xu, P. Ghamisi, "Consistency-regularized region-growing network for semantic segmentation of urban scenes with point-level annotations," IEEE Trans. Image Process., vol. 31, pp. 5038–5051, 2022.
J. Yue et al., "Optical Remote Sensing Image Understanding With Weak Supervision: Concepts, methods, and perspectives," IEEE Geoscience and Remote Sensing Magazine, vol. 10, no. 2, pp. 250-269, June 2022,
Anomaly (Change) Detection
Traditional representation-based detectors suffer from high computation burden and can hardly explore the relationship among different instances. To tackle this challenge, we develop novel collaborative representation methods for high-dimensional remote sensing data without supervision.
Related Work:
S. Chang, P. Ghamisi, "Nonnegative-constrained joint collaborative representation with union dictionary for hyperspectral anomaly detection," IEEE Trans. Geosci. Remote. Sens., vol. 60, pp. 1-13, 2022.
S. Chang, M. Kopp, and P. Ghamisi, "Sketched Multiview Subspace Learning for Hyperspectral Anomalous Change Detection," IEEE Trans. Geosci. Remote. Sens., vol. 60, pp. 1-12, 2022.
Data Fusion
Remote sensing data acquired from different sensors generally possess complementary information which is beneficial to the interpretation of ground objects. To make full use of this advantage, we develop novel data fusion approaches with deep learning for multi-source remote sensing data.
Related Work:
K. Rafiezadeh Shahi, P. Ghamisi, B. Rasti, P. Scheunders, R. Gloaguen, "Unsupervised data fusion with deeper perspective: A novel multisensor deep clustering algorithm," IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 15, pp. 284−296, 2022.
P. Ghamisi et al., "Multisource and Multitemporal Data Fusion in Remote Sensing: A Comprehensive Review of the State of the Art," IEEE Geoscience and Remote Sensing Magazine, vol. 7, no. 1, pp. 6-39, March 2019.