# IntBMoE

The work proposes IntBMoE, a block-conditioned Mixture-of-Experts architecture in which a small learned codebook defines reusable multi-layer blocks, a shared hypernetwork merges all expert bases of each layer into one composed expert, and a router executes only a few blocks per token, thereby keeping pool-wide participation while retaining sparse execution and bounded parameter materialization; it reaches 73.76% Top-1 and 91.48% Top-5 on ImageNet-1K, outperforms the compared sparse and dense MoE baselines on MiniPile language modeling and IntTravel sequential recommendation, and is fully deployed in AMap's generative recommendation system with a 2.4% relative UVCTR gain in online A/B testing.

The work proposes IntBMoE, a block-conditioned Mixture-of-Experts architecture in which a small learned codebook defines reusable multi-layer blocks, a shared hypernetwork merges all expert bases of each layer into one composed expert, and a router executes only a few blocks per token, thereby keeping pool-wide participation while retaining sparse execution and bounded parameter materialization; it reaches 73.76% Top-1 and 91.48% Top-5 on ImageNet-1K, outperforms the compared sparse and dense MoE baselines on MiniPile language modeling and IntTravel sequential recommendation, and is fully deployed in AMap's generative recommendation system with a 2.4% relative UVCTR gain in online A/B testing.