7th April 26

Update: Image classification by evolving bytecode

1. Introduction

We are developing a novel machine learning approach that reimagines genetic programming, a family of methods based on the principles of evolution by natural selection. Like genetic programming broadly, our method generates computer programs capable of solving a given task by iteratively selecting, mutating, and recombining best-performing candidates from a population. We extend this foundation with Zyme, our evolvable programming language and virtual machine, built specifically to support open-ended algorithm discovery.

In a previous post, we described this approach in detail and demonstrated consistent accuracy improvements on a four-digit subset of the MNIST image classification dataset. At that stage, however, the best programs reached only around 50% accuracy - a promising start, but well short of anything that could be called competitive. Since then, we have continued refining the method, with a focus on both accuracy and computational efficiency, and have recently succeeded in evolving programs that reach ~75% accuracy.

Beyond the accuracy gain itself, these results are significant for how they were achieved. Rather than relying on parameter estimation within the confines of predefined model structures, we treat machine learning as a problem of algorithm discovery: the autonomous identification of computational procedures (sequences of byte-level operations) capable of solving a task, with no fixed architecture or prior assumptions about what form a solution should take. Here, the algorithms take the form of the bytecode programs we are evolving. This framing - machine learning as algorithm discovery - represents a reorientation in the prevailing paradigm of how artificial intelligence is pursued.

2. Results

To evolve computer programs capable of classifying images, we followed the same two step-process as in the previous experiments: (1) reproduction with random mutation followed by (2) selection, in which the top-performing programs are retained to form the population for the next generation (see Previous Methods 3.4). This time, we scaled the process by significantly increasing in training dataset size and the population parameters controlling how many programs are spawned each generation¹.

We also changed the composition of the initial programs. As before, we used the Zyme compiler to generate the starting population (see Previous Methods 3.3). In earlier experiments, we seeded the population directly with raw compiler output: programs in their most minimal form, consisting only of the functional strands defined in the source code.

Here, we introduced ‘junk’ bytes into these initial programs. These are extra bytes that do not affect immediate program behavior, analogous to the non-coding DNA found in biological genomes². We hypothesised that minimal programs are brittle: because nearly every byte encodes an active instruction, most mutations disrupt existing logic rather than expanding it. By adding ‘junk’ bytes, we provide space where point mutations (that change individual bytes) can effectively activate new instructions from inactive bytes, rather than overwriting functional code.

This ‘junk’ was introduced to programs in two ways. First, random no-op bytes were inserted into existing strands, increasing their length without altering their behaviour. Second, strands were duplicated with a high error rate, producing inactive copies that resembled the functional strands in structure but not in behaviour.

Using these junk-seeded populations, we ran two independent replicates for 100 generations. To measure generalization, we evaluated every program’s accuracy against the full test dataset at each generation (see Figure 1).

As observed in previous experiments, accuracy increases consistently across generations. However, these runs substantially exceeded our prior ceiling of ~50%, reaching mean population accuracies of ~73% and ~63% across the two replicates. Notably, individual programs in replicate A reached as high as 75% accuracy - a milestone that felt like a significant breakthrough.

The evolutionary trajectory in both replicates differed from our earlier experiments. Instead of the immediate gains seen previously, we observed a ‘lag phase’ of approximately five generations before accuracy improved meaningfully. In replicate B, while some individuals show above-random performance (>25% accuracy) from early on, the population as a whole does not begin to shift until around generation five; replicate A shows even less early activity. Once improvements do take hold, however, accuracy rises sharply and quickly surpasses the ceiling observed in previous experiments. Perhaps the 'junk' added to the initial programs makes it difficult to find early adaptive mutations, but paradoxically enhances long-term evolvability?

The two replicates converged on distinct peak accuracy scores, with Replicate A outperforming Replicate B by ~10%. This divergence reinforces our previous observation that independent runs often discover potentially entirely different mechanisms for solving the task. This difference is also visible in their stability: as replicate A approaches its peak, intergeneration variability increases as the population struggles to maintain its top-performing programs. In contrast, replicate B shows considerably greater stability. Yet, when examining batch accuracy - measured against the random training samples drawn each generation - performance became highly volatile across both replicates (see Figure 2).

The high batch-level variability likely reflects sampling bias, where randomly drawn training batches may coincidentally favor specific programs and artificially inflate their batch-level accuracy. This concern is compounded by the narrow range of variation in batch accuracy, suggesting that selection pressure may be overly aggressive. This can lead to stochastic overfitting, where the population oscillates between accuracy levels as it chases the characteristics of individual batches. While this is a clear limitation for future research, it does not fully explain the intergenerational instability in test accuracy, which was largely unique to Replicate A.

3. Discussion

The experiments presented here represent some of our best results to date, reaching ~75% accuracy. Although this falls well short of state-of-the-art image classification methods, it illustrates how a broader paradigm - machine learning as algorithm discovery - can work in practice.

Traditional machine learning treats the problem as one of parameter estimation: given a model with a fixed, predetermined structure - a neural network, for instance - find the set of parameters, such as its weights, that best fits the data. We take a different view, framing the problem as one of algorithm discovery: we search for a sequence of rudimentary instructions - unconstrained by any particular structure - that can learn to make sense of data on its own terms.

For such a search to be truly open-ended, the primitive instructions being composed must be low-level enough to carry no implicit knowledge of the problem domain, while still including control flow operations that enable loops and conditional behavior. In principle, this leaves open the possibility that any algorithm capable of solving the given task could be discovered - including, conceivably, something resembling a neural network architecture.

This is, in essence, the aspiration of genetic programming: using the evolutionary process to search for algorithms, embodied as computer programs, that solve a given problem.

We pursue algorithm discovery by evolving Zyme programs. While built on an unconventional model of computation, they are composed entirely of low-level, byte-level instructions. Any operations on larger complex collections of input bytes - such as those needed for image processing - must be assembled autonomously from these primitive building blocks.

Achieving 75% accuracy on this subset of the MNIST image classification task therefore carries a precise meaning: through evolution alone, we have generated an algorithm that manipulates raw bytes and functions as an image classifier - without any implicit prior knowledge of the problem, without hand-crafted representations, and without any form of a priori feature extraction⁴. The resulting algorithm is not yet highly accurate, but we believe that solving the problem in this way - discovering an algorithm composed of byte-level instructions - is without precedent.

The resulting image processing algorithm is obviously embodied in Zyme bytecode, as this is the evolvable representation utilised in the discovery process. But since the programs are ultimately composed of byte-level instructions, there is no reason the underlying algorithm couldn't be extracted and transpiled into other, potentially more useful, representations: maybe Python if interpretability is the priority, or machine code if performance is. To date, our focus has been on optimizing the evolvability of programs - discovering algorithms worth extracting - rather than building the analysis tooling that extraction would require. In principle, though, this opens up the products of algorithm discovery to a much broader range of applications - from standalone synthesized programs to components embedded directly within larger systems.

But before getting caught up in any grand visions, we should be honest about the result at hand — and whether 75% accuracy on this task marks the ceiling of this approach. Over the past year we have run many experiments to identify the limiting factors, and we feel we are making steady progress. Many avenues for optimization also remain unexplored, including the core Zyme architecture itself: its instruction set and underlying design assumptions. We are confident this is only the beginning.