arXiv subjects

Jonathan Taylor

Publications and source records attributed to Jonathan Taylor.

At least 19 recordsLinked to original sources

arXiv2026

AF-action groupoid models for diagonal AH-algebras

We show that an inductive AH-system with diagonal connecting maps describes an action of the canonical AF-groupoid on the spectrum of the canonical C$^*$-diagonal, and that the canonical groupoid model is given by the transformation groupoid associated to this action. This divides the groupoid structure into two distinct aspects: the acting AF-groupoid (which is well-studied) and the unit space on which it acts. We describe the unit space by enriching the Bratteli diagram with extra topological information, and apply these descriptions to examples of interest, including Villadsen algebras of the first kind.

math.OA↗

arXiv2026

VEPHand: View-Efficient Photometric Hand Performance Capture at Scale

Robust, high-fidelity 3D hand capture, while fundamental to digital human creation, remains challenging with practical multi-view systems that balance rich photometry with the geometric ambiguities of reconstruction arising from limited viewpoint density. This paper presents an end-to-end pipeline for dynamic hand performance capture and registration, specifically designed for view-efficient setups ($\sim$20 views). We address key challenges with two primary innovations. First, to overcome reconstruction difficulties like limited view overlap and background clutter, our mask-free neural method robustly extracts detailed hand geometry and appearance from unmasked images using scene parameterization and scenario-specific density regularization. Second, addressing registration challenges such as accurately capturing non-linear skin deformations and ensuring plausible results during severe self-contact, we propose a physics-inspired framework. It aligns reconstructions to a personalized hand model by optimizing intrinsic volumetric offsets within its canonical tetrahedral mesh, alongside pose parameters. This approach, supported by robust losses and optimization, captures fine surface deformations, ensures plausible results under severe articulation and self-contact, and demonstrates strong tolerance to input noise. We demonstrate the scalability and robustness of our automated pipeline on an extensive dataset of over 12,000 sequences, from which we also derive a large-scale, high-quality synthetic 2D/3D hand dataset for training downstream tasks. This showcases its effectiveness for single hands, intricate two-hand interactions, and natural hand-object manipulations. Our method achieves state-of-the-art reconstruction fidelity in view-efficient, unmasked scenarios and highly accurate registration. Our project page are available at https://vephand.github.io/.

cs.CV↗

arXiv2025

Limits in categories of \'etale groupoids and pseudogroups

We show that the category of sober \'etale groupoids and actors admits all small limits. This is achieved by computing the limits in the equivalent category of pseudogroups with pseudogroup morphisms, which we show admits a forgetful functor to the category of sets which creates limits. We give an alternative proof of the adjunction of Cockett and Garner in the specific setting of \'etale groupoids and pseudogroups which is a central tool for computing limits of sober \'etale groupoids.

math.CT↗

arXiv2025

Inference for location and height of peaks of a standardized field after selection

Peak inference concerns the use of local maxima ("peaks") of a noisy random field to detect and localize regions where underlying signal is present. We propose a peak inference method that first subjects observed peaks to a significance test of the null hypothesis that no signal is present, and then uses the peaks that are declared significant to construct post-selectively valid confidence regions for the location and height of nearby true peaks. We analyze the performance of this method in a smooth signal plus constant variance noise model under a high-curvature asymptotic assumption, and prove that it asymptotically controls both the number of false discoveries, and the number of confidence regions that do not contain a true peak, relative to the number of points at which inference is conducted. An important intermediate theoretical result uses the Kac-Rice formula to derive a novel approximation to the intensity function of a point process that counts local maxima, which is second-order accurate under the alternative, nearby high-curvature true peaks.

stat.ME↗

arXiv2024

One-Click Upgrade from 2D to 3D: Sandwiched RGB-D Video Compression for Stereoscopic Teleconferencing

Stereoscopic video conferencing is still challenging due to the need to compress stereo RGB-D video in real-time. Though hardware implementations of standard video codecs such as H.264 / AVC and HEVC are widely available, they are not designed for stereoscopic videos and suffer from reduced quality and performance. Specific multiview or 3D extensions of these codecs are complex and lack efficient implementations. In this paper, we propose a new approach to upgrade a 2D video codec to support stereo RGB-D video compression, by wrapping it with a neural pre- and post-processor pair. The neural networks are end-to-end trained with an image codec proxy, and shown to work with a more sophisticated video codec. We also propose a geometry-aware loss function to improve rendering quality. We train the neural pre- and post-processors on a synthetic 4D people dataset, and evaluate it on both synthetic and real-captured stereo RGB-D videos. Experimental results show that the neural networks generalize well to unseen data and work out-of-box with various video codecs. Our approach saves about 30% bit-rate compared to a conventional video coding scheme and MV-HEVC at the same level of rendering quality from a novel view, without the need of a task-specific hardware upgrade.

cs.CV↗

arXiv2024

Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers

We propose sandwiching standard image and video codecs between pre- and post-processing neural networks. The networks are jointly trained through a differentiable codec proxy to minimize a given rate-distortion loss. This sandwich architecture not only improves the standard codec's performance on its intended content, but more importantly, adapts the codec to other types of image/video content and to other distortion measures. The sandwich learns to transmit ``neural code images'' that optimize and improve overall rate-distortion performance, with the improvements becoming significant especially when the overall problem is well outside of the scope of the codec's design. We apply the sandwich architecture to standard codecs with mismatched sources transporting different numbers of channels, higher resolution, higher dynamic range, computer graphics, and with perceptual distortion measures. The results demonstrate substantial improvements (up to 9 dB gains or up to 30\% bitrate reductions) compared to alternative adaptations. We establish optimality properties for sandwiched compression and design differentiable codec proxies approximating current standard codecs. We further analyze model complexity, visual quality under perceptual metrics, as well as sandwich configurations that offer interesting potentials in video compression and streaming.

eess.IV↗

arXiv2024

Pretraining and the Lasso

Pretraining is a popular and powerful paradigm in machine learning to pass information from one model to another. As an example, suppose one has a modest-sized dataset of images of cats and dogs, and plans to fit a deep neural network to classify them from the pixel features. With pretraining, we start with a neural network trained on a large corpus of images, consisting of not just cats and dogs but hundreds of other image types. Then we fix all of the network weights except for the top layer(s) (which makes the final classification) and train (or "fine tune") those weights on our dataset. This often results in dramatically better performance than the network trained solely on our smaller dataset. In this paper, we ask the question "Can pretraining help the lasso?". We develop a framework for the lasso in which a model is fit to a large dataset, and then fine-tuned using a smaller dataset. This latter dataset can be a subset of the original dataset, or it can be a dataset with a different but related outcome. This framework has a wide variety of applications, including stratified models, multinomial responses, multi-response models, conditional average treatment estimation and even gradient boosting. In the stratified model setting, the pretrained lasso pipeline estimates the coefficients common to all groups at the first stage, and then estimates the group-specific coefficients at the second "fine-tuning" stage. We show that under appropriate assumptions, the support recovery rate of the common coefficients is superior to that of the usual lasso trained only on individual groups. This separate identification of common and individual coefficients can also be useful for scientific understanding.

stat.ME↗

arXiv2023

MACS: Mass Conditioned 3D Hand and Object Motion Synthesis

The physical properties of an object, such as mass, significantly affect how we manipulate it with our hands. Surprisingly, this aspect has so far been neglected in prior work on 3D motion synthesis. To improve the naturalness of the synthesized 3D hand object motions, this work proposes MACS the first MAss Conditioned 3D hand and object motion Synthesis approach. Our approach is based on cascaded diffusion models and generates interactions that plausibly adjust based on the object mass and interaction type. MACS also accepts a manually drawn 3D object trajectory as input and synthesizes the natural 3D hand motions conditioned by the object mass. This flexibility enables MACS to be used for various downstream applications, such as generating synthetic training data for ML tasks, fast animation of hands for graphics workflows, and generating character interactions for computer games. We show experimentally that a small-scale dataset is sufficient for MACS to reasonably generalize across interpolated and extrapolated object masses unseen during the training. Furthermore, MACS shows moderate generalization to unseen objects, thanks to the mass-conditioned contact labels generated by our surface contact synthesis model ConNet. Our comprehensive user study confirms that the synthesized 3D hand-object interactions are highly plausible and realistic.

cs.CV↗

arXiv2023

Functoriality for groupoid and Fell bundle $C^*$-algebras

We define a class of morphisms between \'etale groupoids and show that there is a functor from the category with these morphisms to the category of $C^*$-algebras. We show that all homomorphisms between Cartan pairs of $C^*$-algebras that preserve the Cartan structure arise from such morphisms between the underlying Weyl groupoids and twists, and attain an equivalence of categories between Cartan pairs with structure preserving homomorphisms and a their associated twists. We define analogous morphisms for Fell bundles over $C^*$-algebras and show these functorially induce ${}^*$-homomorphisms between the Fell bundle $C^*$-algebras. We also construct colimit groupoids and Fell bundles for inductive systems of such morphisms, and show that the functor to $C^*$-algebras preserves these colimits.

math.OA↗

arXiv2023

Uncertainty Intervals for Prediction Errors in Time Series Forecasting

Inference for prediction errors is critical in time series forecasting pipelines. However, providing statistically meaningful uncertainty intervals for prediction errors remains relatively under-explored. Practitioners often resort to forward cross-validation (FCV) for obtaining point estimators and constructing confidence intervals based on the Central Limit Theorem (CLT). The naive version assumes independence, a condition that is usually invalid due to time correlation. These approaches lack statistical interpretations and theoretical justifications even under stationarity. This paper systematically investigates uncertainty intervals for prediction errors in time series forecasting. We first distinguish two key inferential targets: the stochastic test error over near future data points, and the expected test error as the expectation of the former. The stochastic test error is often more relevant in applications needing to quantify uncertainty over individual time series instances. To construct prediction intervals for the stochastic test error, we propose the quantile-based forward cross-validation (QFCV) method. Under an ergodicity assumption, QFCV intervals have asymptotically valid coverage and are shorter than marginal empirical quantiles. In addition, we also illustrate why naive CLT-based FCV intervals fail to provide valid uncertainty intervals, even with certain corrections. For non-stationary time series, we further provide rolling intervals by combining QFCV with adaptive conformal prediction to give time-average coverage guarantees. Overall, we advocate the use of QFCV procedures and demonstrate their coverage and efficiency through simulations and real data examples.

stat.ME↗

arXiv2023

Discovery of Spherules of Likely Extrasolar Composition in the Pacific Ocean Site of the CNEOS 2014-01-08 (IM1) Bolide

We have conducted an extensive towed-magnetic-sled survey during the period 14-28 June, 2023, over the seafloor centered around the calculated path of the bolide CNEOS 2014-01-08 (IM1) about 85 km north of Manus Island, Papua New Guinea. We found about 700 spherules of diameter 0.05-1.3 millimeters in our samples, of which 57 were analyzed so far. The spherules were significantly concentrated along the expected meteor path. Mass spectrometry of 47 spherules near the high-yield regions along IM1's path reveals a distinct extra-solar abundance pattern for 5 of them, while background spherules have abundances consistent with a solar system origin. The unique spherules show an excess of Be, La and U, by up to three orders of magnitude relative to the solar system standard of CI chondrites. These "BeLaU"-type spherules, never seen before, also have very low refractory siderophile elements such as Re. Volatile elements, such as Mn, Zn, Pb, are depleted as expected from evaporation losses during a meteor's airburst. In addition, the mass-dependent variations in $^{57}$Fe/$^{54}$Fe and $^{56}$Fe/$^{54}$Fe are also consistent with evaporative loss of the light isotopes during the spherules' travel in the atmosphere. The "BeLaU" abundance pattern is not found in control regions outside of IM1's path and does not match commonly manufactured alloys or natural meteorites in the solar system. This evidence points towards an association of "BeLaU"-type spherules with IM1, supporting its interstellar origin independently of the high velocity and unusual material strength implied from the CNEOS data. We suggest that the "BeLaU" abundance pattern could have originated from a highly differentiated magma ocean of a planet with an iron core outside the solar system or from more exotic sources.

astro-ph.EP↗

arXiv2023

Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images

We propose a novel transformer-based framework that reconstructs two high fidelity hands from multi-view RGB images. Unlike existing hand pose estimation methods, where one typically trains a deep network to regress hand model parameters from single RGB image, we consider a more challenging problem setting where we directly regress the absolute root poses of two-hands with extended forearm at high resolution from egocentric view. As existing datasets are either infeasible for egocentric viewpoints or lack background variations, we create a large-scale synthetic dataset with diverse scenarios and collect a real dataset from multi-calibrated camera setup to verify our proposed multi-view image feature fusion strategy. To make the reconstruction physically plausible, we propose two strategies: (i) a coarse-to-fine spectral graph convolution decoder to smoothen the meshes during upsampling and (ii) an optimisation-based refinement stage at inference to prevent self-penetrations. Through extensive quantitative and qualitative evaluations, we show that our framework is able to produce realistic two-hand reconstructions and demonstrate the generalisation of synthetic-trained models to real data, as well as real-time AR/VR applications.

cs.CV↗

arXiv2023

Aperiodic dynamical inclusions of $C^*$-algebras

We define the local multiplier module of a Hilbert module in analogy to the local multiplier algebra for $C^*$-algebras. We use properties of the local multiplier module to lift non-closed actions on $C^*$-algebras by Hilbert bimodules to closed actions on local multiplier algebras, and descend known results on such closed actions down to their unclosed counterparts. We define aperiodic dynamical inclusions and characterise them as crossed products by inverse semigroup actions. We describe the slice structure for such inclusions and show that all slices arise as linear combinations of slices already present in the inverse semigroup action.

math.OA↗

arXiv2023

Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers

We propose sandwiched video compression -- a video compression system that wraps neural networks around a standard video codec. The sandwich framework consists of a neural pre- and post-processor with a standard video codec between them. The networks are trained jointly to optimize a rate-distortion loss function with the goal of significantly improving over the standard codec in various compression scenarios. End-to-end training in this setting requires a differentiable proxy for the standard video codec, which incorporates temporal processing with motion compensation, inter/intra mode decisions, and in-loop filtering. We propose differentiable approximations to key video codec components and demonstrate that, in addition to providing meaningful compression improvements over the standard codec, the neural codes of the sandwich lead to significantly better rate-distortion performance in two important scenarios.When transporting high-resolution video via low-resolution HEVC, the sandwich system obtains 6.5 dB improvements over standard HEVC. More importantly, using the well-known perceptual similarity metric, LPIPS, we observe 30% improvements in rate at the same quality over HEVC. Last but not least, we show that pre- and post-processors formed by very modestly-parameterized, light-weight networks can closely approximate these results.

eess.IV↗

arXiv2022

Exact Selective Inference with Randomization

We introduce a pivot for exact selective inference with randomization. Not only does our pivot lead to exact inference in Gaussian regression models, but it is also available in closed form. We reduce the problem of exact selective inference to a bivariate truncated Gaussian distribution. By doing so, we give up some power that is achieved with approximate maximum likelihood estimation in Panigrahi and Taylor (2022). Yet our pivot always produces narrower confidence intervals than a closely related data splitting procedure. We investigate the trade-off between power and exact selective inference on simulated datasets and an HIV drug resistance dataset.

stat.ME↗

arXiv2022

Developing a Series of AI Challenges for the United States Department of the Air Force

Through a series of federal initiatives and orders, the U.S. Government has been making a concerted effort to ensure American leadership in AI. These broad strategy documents have influenced organizations such as the United States Department of the Air Force (DAF). The DAF-MIT AI Accelerator is an initiative between the DAF and MIT to bridge the gap between AI researchers and DAF mission requirements. Several projects supported by the DAF-MIT AI Accelerator are developing public challenge problems that address numerous Federal AI research priorities. These challenges target priorities by making large, AI-ready datasets publicly available, incentivizing open-source solutions, and creating a demand signal for dual use technologies that can stimulate further research. In this article, we describe these public challenges being developed and how their application contributes to scientific advances.

cs.AI↗

arXiv2022

Essential Cartan subalgebras of $C^*$-algebras

We define essential commutative Cartan pairs of $C^*$-algebras generalising the definition of Renault and show that such pairs are given by essential twisted groupoid $C^*$-algebras as defined by Kwa\'sniewski and Meyer. We show that the underlying twisted groupoid is effective, and is unique up to isomorphism among twists over effective groupoids giving rise to the essential commutative Cartan pair. We also show that for twists over effective groupoids giving rise to such pairs, the automorphism group of the twist is isomorphic to the automorphism group of the induced essential Cartan pair via explicit constructions.

math.OA↗

arXiv2022

Inductive Limits of Noncommutative Cartan Inclusions

We prove that an inductive limit of aperiodic noncommutative Cartan inclusions is a noncommutative Cartan inclusion whenever the connecting maps are injective, preserve normalisers and entwine conditional expectations. We show that under the additional assumption that the inductive limit Cartan subalgebra is either essentially separable, essentially simple or essentially of Type I we get an aperiodic inclusion in the limit. Consequently, we subsume the case where the building block Cartan subalgebras are commutative and provide a proof of a theorem of Xin Li without passing to twisted \'etale groupoids.

math.OA↗