# Protein Sequence-Structure Generation

Continuous bitstream diffusion writes images, speech and proteins as binary codes and, with one training and sampling protocol, reaches 3.5% word error rate in joint speech-text generation, while the fold-level agreement of generated protein sequence-structure pairs is reported as 75% in the results section and 69% in the conclusion.

Continuous bitstream diffusion writes images, speech and proteins as binary codes and, with one training and sampling protocol, reaches 3.5% word error rate in joint speech-text generation, while the fold-level agreement of generated protein sequence-structure pairs is reported as 75% in the results section and 69% in the conclusion.

This work introduces two learned discrete latent protein languages: PLL, a 4,096-state contextual alphabet built on a frozen ESM-2 encoder on the sequence side, and SLL, which adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision while remaining decodable to backbone coordinates; under matched downstream training, PLLM has a fitted compute-scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model and reduces the fraction of samples below a 1.5-bit residue-composition entropy threshold by 54%, while replacing the original tokenizer with SLL reduces best validation perplexity by 34% for sequence-to-structure prediction.

This work introduces two learned discrete latent protein languages: PLL, a 4,096-state contextual alphabet built on a frozen ESM-2 encoder on the sequence side, and SLL, which adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision while remaining decodable to backbone coordinates; under matched downstream training, PLLM has a fitted compute-scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model and reduces the fraction of samples below a 1.5-bit residue-composition entropy threshold by 54%, while replacing the original tokenizer with SLL reduces best validation perplexity by 34% for sequence-to-structure prediction.