# Neural Architecture Search

Blockwise distillation architecture search picks child networks by summing each replacement block's isolated score, an additive assumption rarely tested directly. On DeiT-S and DeiT-B, as the block-latency budget drops from 60% to 55%, the additive integer linear program begins to trail a pairwise integer quadratic program that models block interactions, by 3.3–16.6 top-1 points on ImageNet-100 after fine-tuning. The error splits into a metric part from the score's nonlinearity and a functional interaction part, dominant on DeiT-S and DeiT-B respectively; a hybrid search that ranks with single-block logits and measures only a few candidates needs no dedicated pair measurements and beats the pairwise search in 13 of 14 full ImageNet-1k settings.

Blockwise distillation architecture search picks child networks by summing each replacement block's isolated score, an additive assumption rarely tested directly. On DeiT-S and DeiT-B, as the block-latency budget drops from 60% to 55%, the additive integer linear program begins to trail a pairwise integer quadratic program that models block interactions, by 3.3–16.6 top-1 points on ImageNet-100 after fine-tuning. The error splits into a metric part from the score's nonlinearity and a functional interaction part, dominant on DeiT-S and DeiT-B respectively; a hybrid search that ranks with single-block logits and measures only a few candidates needs no dedicated pair measurements and beats the pairwise search in 13 of 14 full ImageNet-1k settings.

Graph neural networks learn structure–property relationships from graphs of atoms and bonds, but those relationships stay in a latent space and are hard to read as scientific rules. Graph symbolic distillation (GSD) first uses a constrained message-passing architecture search to fix how information is transformed, propagated and aggregated, then fits each layer's learned transformation with a sparse symbolic expression while keeping graph connectivity and aggregation. Across halide-perovskite band gaps, molecular hydration free energies and multidentate adsorption on transition-metal surfaces, it rebuilds expert descriptors from elementary node and edge features and yields testable local-geometry, polarity–size and electronic–connectivity relationships.

Graph neural networks learn structure–property relationships from graphs of atoms and bonds, but those relationships stay in a latent space and are hard to read as scientific rules. Graph symbolic distillation (GSD) first uses a constrained message-passing architecture search to fix how information is transformed, propagated and aggregated, then fits each layer's learned transformation with a sparse symbolic expression while keeping graph connectivity and aggregation. Across halide-perovskite band gaps, molecular hydration free energies and multidentate adsorption on transition-metal surfaces, it rebuilds expert descriptors from elementary node and edge features and yields testable local-geometry, polarity–size and electronic–connectivity relationships.

The work replaces one-shot quantile estimation with online feedback control (Adaptive Conformal Inference with tuning-free, locally-adaptive, and group-conditional variants) to address non-exchangeability of candidates during reinforcement-learning search, tracking requested coverage to within about 10^-3 across three architecture families and both single-step and sequential search (three seeds) while pruning 25-50% of evaluations at no measured accuracy cost, whereas static calibration loses coverage control and a Gaussian-process baseline stays conservative.

The work replaces one-shot quantile estimation with online feedback control (Adaptive Conformal Inference with tuning-free, locally-adaptive, and group-conditional variants) to address non-exchangeability of candidates during reinforcement-learning search, tracking requested coverage to within about 10^-3 across three architecture families and both single-step and sequential search (three seeds) while pruning 25-50% of evaluations at no measured accuracy cost, whereas static calibration loses coverage control and a Gaussian-process baseline stays conservative.

EvoCast is a fully autonomous research-agent system for iterative time-series forecasting architecture evolution: it first establishes and diagnoses a task-specific baseline, then generates evidence-grounded research directions from dataset characteristics, diagnostic results, prior rounds, and failure records, with LLM agents handling hypothesis generation and code implementation while deterministic program authorities control source-edit boundaries, canonical evaluation, and model promotion; it reaches 96.7% success on 30 repository-level implementation tasks and the lowest MSE on three real forecasting cases (Air Quality, Melbourne Pedestrian T33, Steel Industry) while using fewer tokens and lower API cost than R&D-Agent.

EvoCast is a fully autonomous research-agent system for iterative time-series forecasting architecture evolution: it first establishes and diagnoses a task-specific baseline, then generates evidence-grounded research directions from dataset characteristics, diagnostic results, prior rounds, and failure records, with LLM agents handling hypothesis generation and code implementation while deterministic program authorities control source-edit boundaries, canonical evaluation, and model promotion; it reaches 96.7% success on 30 repository-level implementation tasks and the lowest MSE on three real forecasting cases (Air Quality, Melbourne Pedestrian T33, Steel Industry) while using fewer tokens and lower API cost than R&D-Agent.