arXiv subjects

Chao Xu

Publications and source records attributed to Chao Xu.

At least 361 records · Page 20Linked to original sources

arXiv2019

Bringing Giant Neural Networks Down to Earth with Unlabeled Data

Compressing giant neural networks has gained much attention for their extensive applications on edge devices such as cellphones. During the compressing process, one of the most important procedures is to retrain the pre-trained models using the original training dataset. However, due to the consideration of security, privacy or commercial profits, in practice, only a fraction of sample training data are made available, which makes the retraining infeasible. To solve this issue, this paper proposes to resort to unlabeled data in hand that can be cheaper to acquire. Specifically, we exploit the unlabeled data to mimic the classification characteristics of giant networks, so that the original capacity can be preserved nicely. Nevertheless, there exists a dataset bias between the labeled and unlabeled data, which may disturb the training and degrade the performance. We thus fix this bias by an adversarial loss to make an alignment on the distributions of their low-level feature representations. We further provide theoretical discussions about how the unlabeled data help compressed networks to generalize better. Experimental results demonstrate that the unlabeled data can significantly improve the performance of the compressed networks.

cs.LG↗

arXiv2019

Optimizing Information Freshness in Computing enabled IoT Networks

Internet of Things (IoT) has emerged as one of the key features of the next generation wireless networks, where timely delivery of status update packets is essential for many real-time IoT applications. To provide users with context-aware services and lighten the transmission burden, the raw data usually needs to be preprocessed before being transmitted to the destination. However, the effect of computing on the overall information freshness is not well understood. In this work, we first develop an analytical framework to investigate the information freshness, in terms of peak age of information (PAoI), of a computing enabled IoT system with multiple sensors. Specifically, we model the procedure of computing and transmission as a tandem queue, and derive the analytical expressions of the average PAoI for different sensors. Based on the theoretical results, we formulate a min-max optimization problem to minimize the maximum average PAoI of different sensors. We further design a derivative-free algorithm to find the optimal updating frequency, with which the complexity for checking the convexity of the formulated problem or obtaining the derivatives of the object function can be largely reduced. The accuracy of our analysis and effectiveness of the proposed algorithm are verified with extensive simulation results.

cs.IT↗

arXiv2019

Energy Efficient Transmission Based on Grouped Spatial Modulation for upstream DSL Systems

The digital Subscriber Line (DSL) remains an important component of heterogeneous networking, especially in historic city-centers, where using optical fibre is less realistic. Recently, the power consumption has become an important performance metric in telecommunication due to the associated environmental issues. In the recent bonding model, customer sites have been equipped with two/four copper pairs, which may be exploited for designing grouped spatial modulation (SM) aiming for reducing the power consumption and mitigating the stubborn crosstalk in DSL communications. Explicitly, we view the two pair copper pairs equipped for each user as a group and propose an energy efficient transmission scheme based on grouped SM strategy for the upstream DSL systems, which is capable of reducing the power consumption of the upstream transmitters by activating a single copper line of each user. More especially, in order to compensate for the potential bit-rate reduction imposed by reducing the number of activated lines, the proposed scheme implicitly delivers ``virtual bits" via activating/deactivating the lines in addition to the classic modulation scheme. This is particularly beneficial in the DSL context, because the cross-talk imposed by activating several lines may swamp the desired signal. Furthermore, a pair of near-optimal soft turbo detection schemes are proposed for exploiting the unique properties of the DSL channel in order to eliminate the error propagation problem of SM detection routinely encountered in wireless channels. Both the attainable energy-efficiency and the achievable Bit Error Ratio (BER) are investigated. Our simulation results demonstrate that the proposed group-based SM is capable of outperforming the vectoring scheme both in terms of its energy efficiency for all the examined loop lengths and transmit powers.

eess.SP↗

arXiv2019

Exciton gas transport through nano-constrictions

An indirect exciton is a bound state of an electron and a hole in spatially separated layers. Two-dimensional indirect excitons can be created optically in heterostructures containing double quantum wells or atomically thin semiconductors. We study theoretically transmission of such bosonic quasiparticles through nano-constrictions. We show that quantum transport phenomena, e.g., conductance quantization, single-slit diffraction, two-slit interference, and the Talbot effect, are experimentally realizable in systems of indirect excitons. We discuss similarities and differences between these phenomena and their counterparts in electronic devices.

cond-mat.mes-hall↗

arXiv2019

On the computational complexity of the probabilistic label tree algorithms

Label tree-based algorithms are widely used to tackle multi-class and multi-label problems with a large number of labels. We focus on a particular subclass of these algorithms that use probabilistic classifiers in the tree nodes. Examples of such algorithms are hierarchical softmax (HSM), designed for multi-class classification, and probabilistic label trees (PLTs) that generalize HSM to multi-label problems. If the tree structure is given, learning of PLT can be solved with provable regret guaranties [Wydmuch et.al. 2018]. However, to find a tree structure that results in a PLT with a low training and prediction computational costs as well as low statistical error seems to be a very challenging problem, not well-understood yet. In this paper, we address the problem of finding a tree structure that has low computational cost. First, we show that finding a tree with optimal training cost is NP-complete, nevertheless there are some tractable special cases with either perfect approximation or exact solution that can be obtained in linear time in terms of the number of labels $m$. For the general case, we obtain $O(\log m)$ approximation in linear time too. Moreover, we prove an upper bound on the expected prediction cost expressed in terms of the expected training cost. We also show that under additional assumptions the prediction cost of a PLT is $O(\log m)$.

cs.LG↗

arXiv2019

Decomposition-Based Transfer Distance Metric Learning for Image Classification

Distance metric learning (DML) is a critical factor for image analysis and pattern recognition. To learn a robust distance metric for a target task, we need abundant side information (i.e., the similarity/dissimilarity pairwise constraints over the labeled data), which is usually unavailable in practice due to the high labeling cost. This paper considers the transfer learning setting by exploiting the large quantity of side information from certain related, but different source tasks to help with target metric learning (with only a little side information). The state-of-the-art metric learning algorithms usually fail in this setting because the data distributions of the source task and target task are often quite different. We address this problem by assuming that the target distance metric lies in the space spanned by the eigenvectors of the source metrics (or other randomly generated bases). The target metric is represented as a combination of the base metrics, which are computed using the decomposed components of the source metrics (or simply a set of random bases); we call the proposed method, decomposition-based transfer DML (DTDML). In particular, DTDML learns a sparse combination of the base metrics to construct the target metric by forcing the target metric to be close to an integration of the source metrics. The main advantage of the proposed method compared with existing transfer metric learning approaches is that we directly learn the base metric coefficients instead of the target metric. To this end, far fewer variables need to be learned. We therefore obtain more reliable solutions given the limited side information and the optimization tends to be faster. Experiments on the popular handwritten image (digit, letter) classification and challenge natural image annotation tasks demonstrate the effectiveness of the proposed method.

cs.CV↗

arXiv2019

Multi-View Matrix Completion for Multi-Label Image Classification

There is growing interest in multi-label image classification due to its critical role in web-based image analytics-based applications, such as large-scale image retrieval and browsing. Matrix completion has recently been introduced as a method for transductive (semi-supervised) multi-label classification, and has several distinct advantages, including robustness to missing data and background noise in both feature and label space. However, it is limited by only considering data represented by a single-view feature, which cannot precisely characterize images containing several semantic concepts. To utilize multiple features taken from different views, we have to concatenate the different features as a long vector. But this concatenation is prone to over-fitting and often leads to very high time complexity in MC based image classification. Therefore, we propose to weightedly combine the MC outputs of different views, and present the multi-view matrix completion (MVMC) framework for transductive multi-label image classification. To learn the view combination weights effectively, we apply a cross validation strategy on the labeled set. In the learning process, we adopt the average precision (AP) loss, which is particular suitable for multi-label image classification. A least squares loss formulation is also presented for the sake of efficiency, and the robustness of the algorithm based on the AP loss compared with the other losses is investigated. Experimental evaluation on two real world datasets (PASCAL VOC' 07 and MIR Flickr) demonstrate the effectiveness of MVMC for transductive (semi-supervised) multi-label image classification, and show that MVMC can exploit complementary properties of different features and output-consistent labels for improved multi-label image classification.

stat.ML↗

arXiv2019

Large Margin Multi-modal Multi-task Feature Extraction for Image Classification

The features used in many image analysis-based applications are frequently of very high dimension. Feature extraction offers several advantages in high-dimensional cases, and many recent studies have used multi-task feature extraction approaches, which often outperform single-task feature extraction approaches. However, most of these methods are limited in that they only consider data represented by a single type of feature, even though features usually represent images from multiple modalities. We therefore propose a novel large margin multi-modal multi-task feature extraction (LM3FE) framework for handling multi-modal features for image classification. In particular, LM3FE simultaneously learns the feature extraction matrix for each modality and the modality combination coefficients. In this way, LM3FE not only handles correlated and noisy features, but also utilizes the complementarity of different modalities to further help reduce feature redundancy in each modality. The large margin principle employed also helps to extract strongly predictive features so that they are more suitable for prediction (e.g., classification). An alternating algorithm is developed for problem optimization and each sub-problem can be efficiently solved. Experiments on two challenging real-world image datasets demonstrate the effectiveness and superiority of the proposed method.

stat.ML↗

arXiv2019

Multi-view Vector-valued Manifold Regularization for Multi-label Image Classification

In computer vision, image datasets used for classification are naturally associated with multiple labels and comprised of multiple views, because each image may contain several objects (e.g. pedestrian, bicycle and tree) and is properly characterized by multiple visual features (e.g. color, texture and shape). Currently available tools ignore either the label relationship or the view complementary. Motivated by the success of the vector-valued function that constructs matrix-valued kernels to explore the multi-label structure in the output space, we introduce multi-view vector-valued manifold regularization (MV$\mathbf{^3}$MR) to integrate multiple features. MV$\mathbf{^3}$MR exploits the complementary property of different features and discovers the intrinsic local geometry of the compact support shared by different features under the theme of manifold regularization. We conducted extensive experiments on two challenging, but popular datasets, PASCAL VOC' 07 (VOC) and MIR Flickr (MIR), and validated the effectiveness of the proposed MV$\mathbf{^3}$MR for image classification.

stat.ML↗

arXiv2019

Controllable Defect Driven Symmetry Change and Domain Structure Evolution in BiFeO3 with Enhanced Tetragonality

Defect engineering has been a powerful tool to enable the creation of exotic phases and the discovery of intriguing phenomena in ferroelectric oxides. However, accurate control the concentration of defects remains a big challenge. In this work, ion implantation, that can provide controllable point defects, allows us the ability to produce a controlled defect-driven true super-tetragonal (T) phase with enhanced tetragonality in ferroelectric BiFeO3 thin films. This point defect engineering is found to drive the phase transition from the as-grown mixed rhombohedral-like (R) and tetragonal-like (MC) phase to true tetragonal (T) symmetry. By further increasing the injected dose of He ion, we demonstrate an enhanced tetragonality super-tetragonal (super-T) phase with the largest c/a ratio (~ 1.3) that has ever been experimentally achieved in BiFeO3. A combination of morphology change and domain evolution further confirm that the mixed R/MC phase structure transforms to the single-domain-state true tetragonal phase. Moreover, the re-emergence of R phase and in-plane stripe nanodomains after heat treatment reveal the memory effect and reversible phase transition. Our findings demonstrate the control of R-Mc-T-super T symmetry changes and the creation of true T phase BiFeO3 with enhanced tetragonality through controllable defect engineering. This work also provides a pathway to generate large tetragonality (or c/a ratio) that could be extended to other ferroelectric material systems (such as PbTiO3, BaTiO3 and HfO2) which may lead to strong polarization enhancement.

cond-mat.mtrl-sci↗

arXiv2019

Cost-Sensitive Feature Selection by Optimizing F-Measures

Feature selection is beneficial for improving the performance of general machine learning tasks by extracting an informative subset from the high-dimensional features. Conventional feature selection methods usually ignore the class imbalance problem, thus the selected features will be biased towards the majority class. Considering that F-measure is a more reasonable performance measure than accuracy for imbalanced data, this paper presents an effective feature selection algorithm that explores the class imbalance issue by optimizing F-measures. Since F-measure optimization can be decomposed into a series of cost-sensitive classification problems, we investigate the cost-sensitive feature selection by generating and assigning different costs to each class with rigorous theory guidance. After solving a series of cost-sensitive feature selection problems, features corresponding to the best F-measure will be selected. In this way, the selected features will fully represent the properties of all classes. Experimental results on popular benchmarks and challenging real-world data sets demonstrate the significance of cost-sensitive feature selection for the imbalanced data setting and validate the effectiveness of the proposed method.

cs.CV↗

arXiv2019

Multi-View Intact Space Learning

It is practical to assume that an individual view is unlikely to be sufficient for effective multi-view learning. Therefore, integration of multi-view information is both valuable and necessary. In this paper, we propose the Multi-view Intact Space Learning (MISL) algorithm, which integrates the encoded complementary information in multiple views to discover a latent intact representation of the data. Even though each view on its own is insufficient, we show theoretically that by combing multiple views we can obtain abundant information for latent intact space learning. Employing the Cauchy loss (a technique used in statistical learning) as the error measurement strengthens robustness to outliers. We propose a new definition of multi-view stability and then derive the generalization error bound based on multi-view stability and Rademacher complexity, and show that the complementarity between multiple views is beneficial for the stability and generalization. MISL is efficiently optimized using a novel Iteratively Reweight Residuals (IRR) technique, whose convergence is theoretically analyzed. Experiments on synthetic data and real-world datasets demonstrate that MISL is an effective and promising algorithm for practical applications.

cs.CV↗

arXiv2019

Revisiting Perspective Information for Efficient Crowd Counting

Crowd counting is the task of estimating people numbers in crowd images. Modern crowd counting methods employ deep neural networks to estimate crowd counts via crowd density regressions. A major challenge of this task lies in the perspective distortion, which results in drastic person scale change in an image. Density regression on the small person area is in general very hard. In this work, we propose a perspective-aware convolutional neural network (PACNN) for efficient crowd counting, which integrates the perspective information into density regression to provide additional knowledge of the person scale change in an image. Ground truth perspective maps are firstly generated for training; PACNN is then specifically designed to predict multi-scale perspective maps, and encode them as perspective-aware weighting layers in the network to adaptively combine the outputs of multi-scale density maps. The weights are learned at every pixel of the maps such that the final density combination is robust to the perspective distortion. We conduct extensive experiments on the ShanghaiTech, WorldExpo'10, UCF_CC_50, and UCSD datasets, and demonstrate the effectiveness and efficiency of PACNN over the state-of-the-art.

cs.CV↗

arXiv2019

On Peak Age of Information in Data Preprocessing enabled IoT Networks

Internet of Things (IoT) has been emerging as one of the use cases permeating our daily lives in 5th Generation wireless networks, where status update packages are usually required to be timely delivered for many IoT based intelligent applications. Enabling the collected raw data to be preprocessed before transmitted to the destination can provider users with better context-aware services and lighten the transmission burden. However, the effect from data preprocessing on the overall information freshness is an essential yet unrevealed issue. In this work we study the joint effects of data preprocessing and transmission procedures on information freshness measured by peak age of information (PAoI). Particularity, we formulate the considered multi-source preprocessing and transmission enabled IoT system as a tandem queue where a priority M/G/1 queue is followed by a G/G/1 queue. Then, we respectively derive the closed-form and an information theoretic approximation of the expectation of waiting time for the formulated processing queue and transmission queue, and further get the analytical expressions of the average PAoI for packages from different sources. Finally, the accuracy of our analysis is verified with simulation results.

cs.IT↗

arXiv2018

Learning Student Networks via Feature Embedding

Deep convolutional neural networks have been widely used in numerous applications, but their demanding storage and computational resource requirements prevent their applications on mobile devices. Knowledge distillation aims to optimize a portable student network by taking the knowledge from a well-trained heavy teacher network. Traditional teacher-student based methods used to rely on additional fully-connected layers to bridge intermediate layers of teacher and student networks, which brings in a large number of auxiliary parameters. In contrast, this paper aims to propagate information from teacher to student without introducing new variables which need to be optimized. We regard the teacher-student paradigm from a new perspective of feature embedding. By introducing the locality preserving loss, the student network is encouraged to generate the low-dimensional features which could inherit intrinsic properties of their corresponding high-dimensional features from teacher network. The resulting portable network thus can naturally maintain the performance as that of the teacher network. Theoretical analysis is provided to justify the lower computation complexity of the proposed method. Experiments on benchmark datasets and well-trained networks suggest that the proposed algorithm is superior to state-of-the-art teacher-student learning methods in terms of computational and storage complexity.

cs.LG↗

arXiv2018

LP Relaxation and Tree Packing for Minimum $k$-cuts

Karger used spanning tree packings to derive a near linear-time randomized algorithm for the global minimum cut problem as well as a bound on the number of approximate minimum cuts. This is a different approach from his well-known random contraction algorithm. Thorup developed a fast deterministic algorithm for the minimum $k$-cut problem via greedy recursive tree packings. In this paper we revisit properties of an LP relaxation for $k$-cut proposed by Naor and Rabani, and analyzed by Chekuri, Guha and Naor. We show that the dual of the LP yields a tree packing, that when combined with an upper bound on the integrality gap for the LP, easily and transparently extends Karger's analysis for mincut to the $k$-cut problem. In addition to the simplicity of the algorithm and its analysis, this allows us to improve the running time of Thorup's algorithm by a factor of $n$. We also improve the bound on the number of $α$-approximate $k$-cuts. Second, we give a simple proof that the integrality gap of the LP is $2(1-1/n)$. Third, we show that an optimum solution to the LP relaxation, for all values of $k$, is fully determined by the principal sequence of partitions of the input graph. This allows us to relate the LP relaxation to the Lagrangian relaxation approach of Barahona and Ravi and Sinha; it also shows that the idealized recursive tree packing considered by Thorup gives an optimum dual solution to the LP. This work arose from an effort to understand and simplify the results of Thorup.

cs.DS↗

arXiv2018

Robust Student Network Learning

Deep neural networks bring in impressive accuracy in various applications, but the success often relies on the heavy network architecture. Taking well-trained heavy networks as teachers, classical teacher-student learning paradigm aims to learn a student network that is lightweight yet accurate. In this way, a portable student network with significantly fewer parameters can achieve a considerable accuracy which is comparable to that of teacher network. However, beyond accuracy, robustness of the learned student network against perturbation is also essential for practical uses. Existing teacher-student learning frameworks mainly focus on accuracy and compression ratios, but ignore the robustness. In this paper, we make the student network produce more confident predictions with the help of the teacher network, and analyze the lower bound of the perturbation that will destroy the confidence of the student network. Two important objectives regarding prediction scores and gradients of examples are developed to maximize this lower bound, so as to enhance the robustness of the student network without sacrificing the performance. Experiments on benchmark datasets demonstrate the efficiency of the proposed approach to learn robust student networks which have satisfying accuracy and compact sizes.

cs.LG↗

arXiv2018

Subset Sum Made Simple

Subset Sum is a classical optimization problem taught to undergraduates as an example of an NP-hard problem, which is amenable to dynamic programming, yielding polynomial running time if the input numbers are relatively small. Formally, given a set $S$ of $n$ positive integers and a target integer $t$, the Subset Sum problem is to decide if there is a subset of $S$ that sums up to $t$. Dynamic programming yields an algorithm with running time $O(nt)$. Recently, the authors [SODA '17] improved the running time to $\tilde{O}\bigl(\sqrt{n}t\bigr)$, and it was further improved to $\tilde{O}\bigl(n+t\bigr)$ by a somewhat involved randomized algorithm by Bringmann [SODA '17], where $\tilde{O}$ hides polylogarithmic factors. Here, we present a new and significantly simpler algorithm with running time $\tilde{O}\bigl(\sqrt{n}t\bigr)$. While not the fastest, we believe the new algorithm and analysis are simple enough to be presented in an algorithms class, as a striking example of a divide-and-conquer algorithm that uses FFT to a problem that seems (at first) unrelated. In particular, the algorithm and its analysis can be described in full detail in two pages (see pages 3-5).

cs.DS↗