Reverse Synthetic Neural Networks (RSN): Training-Free Model Construction by Closed-Form Statistical Synthesis
Modern neural networks are trained: a fixed architecture is initialized at random and fit by gradient descent and backpropagation. We study the opposite construction. In a Reverse Synthetic Network the data synthesizes the model directly — every weight, from embeddings and attention projections to feed-forward maps and output heads, is computed in closed form from corpus statistics, with no loss function, no gradient, and no GPU. We report two results, deliberately separated. Synthesis is competitive with trained baselines on discriminative tasks, reaching 87.0% on 20 Newsgroups in tens of seconds on a CPU with zero training. For generation the same paradigm hits a hard ceiling: it collapses to an n-gram model, which we explain mechanistically — PPMI–SVD representations encode distributional similarity, not predictive composition. Measured in bits-per-byte against a locally-run GPT-2, RSN is best understood as a cheap, strong zero-training floor, not a path to the trained frontier.