arXiv subjects

Matthew Yu

Publications and source records attributed to Matthew Yu.

At least 37 records · Page 2Linked to original sources

arXiv2025

FBNetV5: Neural Architecture Search for Multiple Tasks in One Run

Neural Architecture Search (NAS) has been widely adopted to design accurate and efficient image classification models. However, applying NAS to a new computer vision task still requires a huge amount of effort. This is because 1) previous NAS research has been over-prioritized on image classification while largely ignoring other tasks; 2) many NAS works focus on optimizing task-specific components that cannot be favorably transferred to other tasks; and 3) existing NAS methods are typically designed to be "proxyless" and require significant effort to be integrated with each new task's training pipelines. To tackle these challenges, we propose FBNetV5, a NAS framework that can search for neural architectures for a variety of vision tasks with much reduced computational cost and human effort. Specifically, we design 1) a search space that is simple yet inclusive and transferable; 2) a multitask search process that is disentangled with target tasks' training pipeline; and 3) an algorithm to simultaneously search for architectures for multiple tasks with a computational cost agnostic to the number of tasks. We evaluate the proposed FBNetV5 targeting three fundamental vision tasks -- image classification, object detection, and semantic segmentation. Models searched by FBNetV5 in a single run of search have outperformed the previous stateof-the-art in all the three tasks: image classification (e.g., +1.3% ImageNet top-1 accuracy under the same FLOPs as compared to FBNetV3), semantic segmentation (e.g., +1.8% higher ADE20K val. mIoU than SegFormer with 3.6x fewer FLOPs), and object detection (e.g., +1.1% COCO val. mAP with 1.2x fewer FLOPs as compared to YOLOX).

cs.CV↗

arXiv2025

Movie Gen: A Cast of Media Foundation Models

We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.

cs.CV↗

arXiv2024

Fiber 2-Functors and Tambara-Yamagami Fusion 2-Categories

We introduce group-theoretical fusion 2-categories, a strong categorification of the notion of a group-theoretical fusion 1-category. Physically speaking, such fusion 2-categories arise by gauging subgroups of a global symmetry. We show that group-theoretical fusion 2-categories are completely characterized by the property that the braided fusion 1-category of endomorphisms of the monoidal unit is Tannakian. Then, we describe the underlying finite semisimple 2-category of group-theoretical fusion 2-categories, and, more generally, of certain 2-categories of bimodules. We also partially describe the fusion rules of group-theoretical fusion 2-categories, and investigate the group gradings of such fusion 2-categories. Using our previous results, we classify fusion 2-categories admitting a fiber 2-functor. Next, we study fusion 2-categories with a Tambara-Yamagami defect, that is $\mathbb{Z}/2$-graded fusion 2-categories whose non-trivially graded factor is $\mathbf{2Vect}$. We classify these fusion 2-categories, and examine more closely the more restrictive notion of Tambara-Yamagami fusion 2-categories. Throughout, we give many examples to illustrate our various results.

math.CT↗

arXiv2024

The Classification of Fusion 2-Categories

We classify (multi)fusion 2-categories in terms of braided fusion categories and group cohomological data. This classification is homotopy coherent -- we provide an equivalence between the 3-groupoid of (multi)fusion 2-categories up to monoidal equivalences and a certain 3-groupoid of commuting squares of $\mathrm{B}\mathbb{Z}/2$-equivariant spaces. Rank finiteness and Ocneanu rigidity for fusion 2-categories are immediate corollaries of our classification.

math.CT↗

arXiv2024

Pixel-Space Post-Training of Latent Diffusion Models

Latent diffusion models (LDMs) have made significant advancements in the field of image generation in recent years. One major advantage of LDMs is their ability to operate in a compressed latent space, allowing for more efficient training and deployment. However, despite these advantages, challenges with LDMs still remain. For example, it has been observed that LDMs often generate high-frequency details and complex compositions imperfectly. We hypothesize that one reason for these flaws is due to the fact that all pre- and post-training of LDMs are done in latent space, which is typically $8 \times 8$ lower spatial-resolution than the output images. To address this issue, we propose adding pixel-space supervision in the post-training process to better preserve high-frequency details. Experimentally, we show that adding a pixel-space objective significantly improves both supervised quality fine-tuning and preference-based post-training by a large margin on a state-of-the-art DiT transformer and U-Net diffusion models in both visual quality and visual flaw metrics, while maintaining the same text alignment quality.

cs.CV↗

arXiv2024

Semi-Chiral Operators in 4d ${\cal N}=1$ Gauge Theories

We discuss the properties of quarter-BPS local operators in four dimensional ${\cal N}=1$ supersymmetric Yang-Mills theory using the formalism of holomorphic twists. We study loop corrections both to the space of local operators and to algebraic operations which endow the twisted theory with an infinite symmetry algebra. We classify all single-trace quarter-BPS operators in the planar approximation for $SU(N)$ gauge theory and propose a holographic dual description for the twisted theory. We classify perturbative quarter-BPS operators in $SU(2)$ and $SU(3)$ gauge theories with sufficiently small quantum numbers and discuss possible non-perturbative corrections to the answer. We set up analogous calculations for some theories with matter.

hep-th↗

arXiv2023

Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack

Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text. However, these pre-trained models often face challenges when it comes to generating highly aesthetic images. This creates the need for aesthetic alignment post pre-training. In this paper, we propose quality-tuning to effectively guide a pre-trained model to exclusively generate highly visually appealing images, while maintaining generality across visual concepts. Our key insight is that supervised fine-tuning with a set of surprisingly small but extremely visually appealing images can significantly improve the generation quality. We pre-train a latent diffusion model on $1.1$ billion image-text pairs and fine-tune it with only a few thousand carefully selected high-quality images. The resulting model, Emu, achieves a win rate of $82.9\%$ compared with its pre-trained only counterpart. Compared to the state-of-the-art SDXLv1.0, Emu is preferred $68.4\%$ and $71.3\%$ of the time on visual appeal on the standard PartiPrompts and our Open User Input benchmark based on the real-world usage of text-to-image models. In addition, we show that quality-tuning is a generic approach that is also effective for other architectures, including pixel diffusion and masked generative transformer models.

cs.CV↗

arXiv2023

Gauging Categorical Symmetries in 3d Topological Orders and Bulk Reconstruction

We use the language of categorical condensation to give a procedure for gauging nonabelian anyons, which are the manifestations of categorical symmetries in three spacetime dimensions. We also describe how the condensation procedure can be used in other contexts such as for topological cosets and constructing modular invariants. By studying a generalization of which anyons are condensable, we arrive at representations of congruence subgroups of the modular group. We finally present an analysis for ungauging anyons, which is related to the problem of constructing a Drinfeld center for a fusion category; this procedure we refer to as bulk reconstruction. We introduce a set of consistency relations regarding lines in the parent theory and wall category. Through use of these relations along with the $S$-matrix elements of the child theory, we construct $S$-matrix elements of a parent theory in a number of examples.

hep-th↗

arXiv2023

A Practical Stereo Depth System for Smart Glasses

We present the design of a productionized end-to-end stereo depth sensing system that does pre-processing, online stereo rectification, and stereo depth estimation with a fallback to monocular depth estimation when rectification is unreliable. The output of our depth sensing system is then used in a novel view generation pipeline to create 3D computational photography effects using point-of-view images captured by smart glasses. All these steps are executed on-device on the stringent compute budget of a mobile phone, and because we expect the users can use a wide range of smartphones, our design needs to be general and cannot be dependent on a particular hardware or ML accelerator such as a smartphone GPU. Although each of these steps is well studied, a description of a practical system is still lacking. For such a system, all these steps need to work in tandem with one another and fallback gracefully on failures within the system or less than ideal input data. We show how we handle unforeseen changes to calibration, e.g., due to heat, robustly support depth estimation in the wild, and still abide by the memory and latency constraints required for a smooth user experience. We show that our trained models are fast, and run in less than 1s on a six-year-old Samsung Galaxy S8 phone's CPU. Our models generalize well to unseen data and achieve good results on Middlebury and in-the-wild images captured from the smart glasses.

cs.CV↗

arXiv2023

Infrared phases of 2d QCD

We derive the necessary and sufficient conditions for a 2d QCD theory of massless gluons and left and right chiral quarks in arbitrary representations of a gauge group G to develop a mass gap. These results are obtained from spectral properties of the lightcone and temporal QCD Hamiltonians. The conditions can be explicitly solved, and we provide the complete list of all 2d QCD theories that have a quantum mechanical gap in the spectrum, while any other theory not in the list is gapless. The list of gapped theories includes QCD models with quarks in vector-like as well as chiral representations. The gapped theories consist of several infinite families of classical gauge groups with quarks in rank 1 and 2 representations, plus a finite number of isolated cases. We also put forward and analyze the effective infrared description of QCD - TQFTs for gapped theories and CFTs for gapless theories - and exhibit several interesting features in the infrared, such as the existence of non-trivial global 't Hooft anomalies and emergent supersymmetry. We identify 2d QCD theories that flow in the infrared to celebrated CFTs such as minimal models, bosonic and supersymmetric, and Wess-Zumino-Witten and Kazama-Suzuki models.

hep-th↗

arXiv2022

Gauging Noninvertible Defects: A 2-Categorical Perspective

We generalize the notion of an anomaly for a symmetry to a noninvertible symmetry enacted by surface operators using the framework of condensation in 2-categories. Given a multifusion 2-category, potentially with some additional levels of monoidality, we prove theorems about the structure of the 2-category obtained by condensing a suitable algebra object. We give examples where the resulting category displays grouplike fusion rules and through a cohomology computation, find the obstruction to condensing further to the vacuum theory.

math.CT↗

arXiv2022

Feynman Diagrams in Four-Dimensional Holomorphic Theories and the Operatope

We study a class of universal Feynman integrals which appear in four-dimensional holomorphic theories. We recast the integrals as the Fourier transform of a certain polytope in the space of loop momenta (aka the ``Operatope''). We derive a set of quadratic recursion relations which appear to fully determine the final answer. Our strategy can be applied to a very general class of twisted supersymmetric quantum field theories.

hep-th↗

arXiv2022

Topological Orders in (4+1)-Dimensions

We investigate the Morita equivalences of (4+1)-dimensional topological orders. We show that any (4+1)-dimensional super (fermionic) topological order admits a gapped boundary condition -- in other words, all (4+1)-dimensional super topological orders are Morita trivial. As a result, there are no inherently gapless super (3+1)-dimensional theories. On the other hand, we show that there are infinitely many algebraically Morita-inequivalent bosonic (4+1)-dimensional topological orders.

hep-th↗

arXiv2022

KLT Factorization of Nonrelativistic String Amplitudes

We continue our study of the Kawai-Lewellen-Tye (KLT) factorization of winding string amplitudes. In a toroidal compactification, amplitudes for winding closed string states factorize into products of amplitudes for open strings ending on an array of D-branes localized in the compactified directions; the specific D-brane configuration is determined by the closed string data. In this paper, we study a zero Regge slope limit of the KLT relations between winding string amplitudes. Such a limit of string theory requires a critically tuned Kalb-Ramond field in a compact direction, and leads to a self-contained corner called nonrelativistic string theory. This theory is unitary, ultraviolet complete, and its string spectrum and spacetime S-matrix satisfy nonrelativistic symmetry. Moreover, the asymptotic closed string states in nonrelativistic string theory necessarily carry nonzero windings. First, starting with relativistic string theory, we construct a KLT factorization of amplitudes for winding closed strings in the presence of a critical Kalb-Ramond field. Then, in the zero Regge limit, we uncover a KLT relation for amplitudes in nonrelativistic string theory. Finally, we show how such a relation can be reproduced from first principles in a purely nonrelativistic string theory setting. We will also discuss connections to the amplitudes of string theory in the discrete light cone quantization (DLCQ), a method that is relevant for Matrix theory.

hep-th↗

arXiv2021

Filtering cohomology of ordinary and Lagrangian Grassmannians

This paper studies, for a positive integer $m$, the subalgebra of the cohomology ring of the complex Grassmannians generated by the elements of degree at most $m$. We build in two ways upon a conjecture for the Hilbert series of this subalgebra due to Reiner and Tudose. The first reinterprets it in terms of the operation of $k$-conjugation, suggesting two conjectural bases for the subalgebras that would imply their conjecture. The second introduces an analogous conjecture for the cohomology of Lagrangian Grassmannians.

math.CO↗

arXiv2021

KLT Factorization of Winding String Amplitudes

We uncover a Kawai-Lewellen-Tye (KLT)-type factorization of closed string amplitudes into open string amplitudes for closed string states carrying winding and momentum in toroidal compactifications. The winding and momentum closed string quantum numbers map respectively to the integer and fractional winding quantum numbers of open strings ending on a D-brane array localized in the compactified directions. The closed string amplitudes factorize into products of open string scattering amplitudes with the open strings ending on a D-brane configuration determined by closed string data.

hep-th↗

arXiv2021

Genus-One data and Anomaly Detection

We introduce the notion of genus-one data for theories in (1+1)-dimensions with an anomalous finite group global symmetry. We outline the groups for which genus-one data is effective in detecting the anomaly, and also show that genus-one data is insufficient to detect the anomaly for dicyclic groups.

hep-th↗

arXiv2021

Nonrelativistic Open String and Yang-Mills Theory

The classical and quantum worldsheet theory describing nonrelativistic open string theory in an arbitrary nonrelativistic open and closed string background is constructed. We show that the low energy dynamics of open strings ending on n coincident D-branes in flat spacetime is described by a Galilean invariant U(n) Yang-Mills theory. We also study nonrelativistic open string excitations with winding number and demonstrate that their dynamics can be encoded into a local gauge theory in one higher dimension. By demanding conformal invariance of the boundary couplings, the nonlinear equations of motion that govern the consistent open string backgrounds coupled to an arbitrary closed background (described by a string Newton-Cartan geometry, Kalb-Ramond, and dilaton field) are derived and shown to emerge from a Galilean invariant Dirac-Born-Infeld type action.

hep-th↗