MultiHashFormer: Hash-based Generative Language Models
MultiHashFormer employs a multi-hash signature mechanism supporting causal language modeling, outperforming standard Transformers across 100M-3B parameters, with zero-parameter multilingual vocabulary expansion.
Huiyin Xue, Atsuki Yamaguchi, Nikolaos Aletras