Dr Zhenghao Chen

Neural image compression has gained significant attention thanks to the remarkable success of deep neural networks. However, most existing neural image codecs focus solely on improving human vision perception. In this work, our objective is to enhance image compression methods for both human vision quality and machine vision tasks simultaneously. To achieve this, we introduce a novel approach to Partition, Transmit, Reconstruct, and Aggregate (PTRA) the latent representation of images to balance the optimizations for both aspects. By employing our method as a module in existing neural image codecs, we create a latent representation predictor that dynamically manages the bit-rate cost for machine vision tasks. To further improve the performance of auto-regressive-based coding techniques, we enhance our hyperprior network and predictor module with context modules, resulting in a reduction in bit-rate. The extensive experiments conducted on various machine vision benchmarks such as ILSVRC 2012, VOC 2007, VOC 2012, and COCO demonstrate the superiority of our newly proposed image compression framework. It outperforms existing neural image compression methods in multiple machine vision tasks including classification, segmentation, and detection, while maintaining high-quality image reconstruction for human vision.

In this work, we propose the first end-to-end optimized framework for compressing automotive stereo videos (i.e., stereo videos from autonomous driving applications) from both left and right views. Specifically, when compressing the current frame from each view, our framework reduces temporal redundancy by performing motion compensation using the reconstructed intra-view adjacent frame and at the same time exploits binocular redundancy by conducting disparity compensation using the latest reconstructed cross-view frame. Moreover, to effectively compress the introduced motion and disparity offsets for better compensation, we further propose two novel schemes called motion residual compression and disparity residual compression to respectively generate the predicted motion offset and disparity offset from the previously compressed motion offset and disparity offset, such that we can more effectively compress residual offset information for better bit-rate saving. Overall, the entire framework is implemented by the fully-differentiable modules and can be optimized in an end-to-end manner. Our comprehensive experiments on three automotive stereo video benchmarks Cityscapes, KITTI 2012 and KITTI 2015 demonstrate that our proposed framework outperforms the learning-based single-view video codec and the traditional hand-crafted multi-view video codec.

Exploiting the relationships between attributes is a key challenge for improving multiple facial attribute recognition. In this work, we are concerned with two types of correlations that are spatial and non-spatial relationships. For the spatial correlation, we aggregate attributes with spatial similarity into a part-based group and then introduce a Group Attention Learning to generate the group attention and the part-based group feature. On the other hand, to discover the non-spatial relationship, we model a group-based Graph Correlation Learning to explore affinities of predefined part-based groups. We utilize such affinity information to control the communication between all groups and then refine the learned group features. Overall, we propose a unified network called Multi-scale Group and Graph Network. It incorporates these two newly proposed learning strategies and produces coarse-to-fine graph-based group features for improving facial attribute recognition. Comprehensive experiments demonstrate that our approach outperforms the state-of-the-art methods.

Safety-oriented visualization is one of significant approaches to gain insights from time-spatial data while neural net currently serves as a decent way to perform machine learning in data mining industry. This paper proposes a visual analytics pipeline for trajectory data enabling better understanding movements pattern of people using Neural Network as back-end and other visualization techniques as front-end for gaining information of preferences of attractions, similarities of groups, popularities of attractions and pattern of movement flow. Such understandings help to address the management issue by extracting the outstanding features to detect abnormal pattern such as detection of crime and predicting overall movements, and so on. Successfully dealing with those issues would have significant improvements of entire management of public facility such as parks and transportation.

Diagnosis from histopathological images is the gold standard in diagnosing breast cancer. This paper investigates using transfer learning with convolutional neural networks to automatically diagnose breast cancer from patches of histopathological images. We compare the performance of using transfer learning with an off-the-shelf deep convolutional neural network architecture, VGGNet, and a shallower custom architecture. Our proposed final ensemble model, which contains three custom convolutional neural network classifiers trained using transfer learning, achieves a significantly higher image classification accuracy on the large public benchmark dataset than the current best results, for all image resolution levels.

It is usually difficult to find datasets of sufficient size to train Deep Convolutional Neural Networks (DCNNs) from scratch. In practice, a neural network is often pre-trained on a very large source dataset. Then, a target dataset is transferred onto the neural network. This approach is a form of transfer learning, and allows very deep networks to achieve outstanding performance even when a small target dataset is available. It is thought that the bottom layers of the pre-trained network contain general information, which are applicable to different datasets and tasks, while the upper layers of the pre-trained network contain abstract information relevant to a specific dataset and task. While studies have been conducted on the fine-tuning of these layers, the removal of these layers have not yet been considered. This paper explores the effect of removing the upper convolutional layers of a pre-trained network. We empirically investigated whether removing upper layers of a deep pre-trained network can improve performance for transfer learning. We found that removing upper pre-trained layers gives a significant boost in performance, but the ideal number of layers to remove depends on the dataset. We suggest removing pre-trained convolutional layers when applying transfer learning on off-the-shelf pre-trained DCNNs. The ideal number of layers to remove will depend on the dataset, and remain as a parameter to be tuned.

This paper proposes a novel Weight Adjusted Particle Swarm Optimization (WAPSO) to overcome the occlusion problem and computational cost in multiple object tracking. To this end, a new update strategy of inertia weight of the particles in WAPSO is designed to maintain particle diversity and prevent pre-mature convergence. Meanwhile, the implementation of a mechanism that enlarges the search space upon the detection of occlusion enhances WAPSO's robustness to non-linear target motion. In addition, the choice of Root Sum Squared Errors as the fitness function further increases the speed of the proposed approach. The experimental results has shown that in combination with the model feature that enables initialization of multiple independent swarms, the high-speed WAPSO algorithm can be applied to multiple non-linear object tracking for real-time applications.

For chest X-ray image (CXR) analysis, effective bone structure suppression is essential for uncovering lung abnormalities and facilitating accurate clinical diagnoses. While recent deep generative models, to some extent, improve the reconstruction quality of bone-suppressed CXRs, they often fall short in delivering substantial improvements in downstream diagnosis tasks. This limitation is attributed to a narrow focus on instance-specific details, neglecting broader domain-level knowledge, which hampers bone-suppression effectiveness. In response to these challenges, our proposed framework adopts a novel approach that integrates both instance-level and domain-level information. To capture instance information, our model employs a hybrid approach using both cross-covariance attention blocks (CABs) to underscore relevant image information and a followed Vision Transformers (ViTs) encoder for image feature embedding. To capture domain information, we introduce multi-head codebook attention (MCA) which leverages codebook structure with multi-head attention mechanism to capture global, domain-level information specific to the bone-suppressed CXR domain, thereby refining the synthesis process. During optimization, our two-stage training scheme involves a MCA learning stage that encapsulates the domain of bone-suppressed CXRs in MCA through a ViT-based GAN model, and a synthesis stage that employs the learned codebook to generate bone-suppressed CXRs from the original ones, enhancing instance synthesis through domain insights. Moreover, the incorporation of CABs further refines pixellevel instance information. Extensive experiments demonstrate the superior performance of our approach, improving PSNR by 8.36% and SSIM by 2.7% for bone suppression while boosting lung disease classification by 2.8% and 4.2% on two datasets and segmentation by 1.5%.

Recent neural networks based surface reconstruction can be roughly divided into two categories, one warping templates explicitly and the other representing 3D surfaces implicitly. To enjoy the advantages of both, we propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. This is achieved by directly predicting the displacements from surface queries and modeling shapes as Vector Fields, rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods do. In this way, our approach is capable of encoding both the distance and the direction fields so that the calculation of direction fields is differentiation-free, circumventing the non-trivial surface extraction step. Furthermore, building upon NVFs, we propose to incorporate two types of shape codebooks, i.e., NVFs (Lite or Ultra), to promote cross-category reconstruction through encoding cross-object priors. Moreover, we propose a new regularization based on analyzing the zero-curl property of NVFs, and implement this through the fully differentiable framework of our NVF (ultra). We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.