Background

Language models are known to exhibit preference for their own model outputs while evaluating the quality of task completion. This happens across many different models, even when objective criteria are given to inform judgment. Self-refinement of LLM outputs tends to amplify this self-preference bias.

Some work suggests that lower perplexity from self-generated work may be a driver behind this self-preference. The perplexity of a given text to a model can be loosely understood to indicate how similar the text is to samples that the model saw during training. Text generated by a given language model will naturally be lower-perplexity than text generated by a human or by a different language model trained on different data.

AI Assisted Coding and Self-Preference

This self-preference presents an opportunity for improving AI-assisted coding. If a model generates, reviews, and refines its own output, the effects of self-preference may interfere with a model’s ability to effectively complete coding tasks. In a previous experiment, I saw clear self-preference effects resulting in lower recall rates of bugs detected by model self-generated outputs.

AI assisted coding is unique because it is fairly easy to manipulate perplexity while retaining code functionality and correctness. Within a function, the spelling of local variable names and commented-out code can be changed while leaving core functionality (e.g. whether it completes a given task) intact. If self-preference can be manipulated by merely changing code perplexity, LLM-assisted coding harnesses can programmatically manipulate pieces of code that don’t impact code logic to improve code review quality and self-improvement.

The current experiment

In this experiment, I will manipulate perplexity of given code outputs by generating high-perplexity strings from the hash of variable names and other identifiers, and either appending those high-perplexity strings to the end of the identifiers or replacing those identifiers fully with high-perplexity strings.

I will use an open-source language model (Qwen2.5-Coder-7B) to validate that the hash-appending strategy actually increases code perplexity. Then, I will use both gemini-3.5-flash-lite and gpt-5.6-luna (which had high self-preference bias in our previous experiment) to evaluate whether increasing model perplexity reduces self-preference and makes a model more able to detect errors in its own logic.

Perplexity-increasing transformations

To demonstrate what the transformations I’m using are, and why they would work to increase code perplexity, I’ll share an example of transformations for one of the problems:

problem_id=468
----------------------------------------------------------------------
Write a function to find the maximum product formed by multiplying numbers of an increasing subsequence of that array.

Unit tests:
  assert max_product([3, 100, 4, 5, 150, 6]) == 3000
  assert max_product([4, 42, 55, 68, 80]) == 50265600
  assert max_product([10, 22, 9, 33, 21, 50, 41, 60]) == 2460

Here’s what the response from gemini-3.5-flash-lite looks like without any hashing applied:

def max_product(arr):
    """Finds the maximum product formed by multiplying numbers of 
    an increasing subsequence (contiguous) of the array.
    """
    if not arr:
        return 0

    max_prod = curr_prod = arr[0]

    for i in range(1, len(arr)):
        if arr[i] > arr[i - 1]:
            curr_prod *= arr[i]
        else:
            curr_prod = arr[i]
        max_prod = max(max_prod, curr_prod)

    return max_prod

Here’s what the response looks like with hashing appended. Note that this definitely increases perplexity in the code (I doubt that the variable name arr_2e6839f2 appears in many python programs).

def max_product_cc95865c(arr_2e6839f2):
    """Finds the maximum product formed by multiplying numbers of 
    an increasing subsequence (contiguous) of the array.
    """
    if not arr_2e6839f2:
        return 0

    max_prod_76e4731a = curr_prod_1b5849b8 = arr_2e6839f2[0]

    for i in range(1, len(arr_2e6839f2)):
        if arr_2e6839f2[i] > arr_2e6839f2[i - 1]:
            curr_prod_1b5849b8 *= arr_2e6839f2[i]
        else:
            curr_prod_1b5849b8 = arr_2e6839f2[i]
        max_prod_76e4731a = max(max_prod_76e4731a, curr_prod_1b5849b8)

    return max_prod_76e4731a

And here is the maximally high-perplexity transformation, with a full replacement of all variable names.

def _cc95865c(_2e6839f2):
    """Finds the maximum product formed by multiplying numbers of 
    an increasing subsequence (contiguous) of the array.
    """
    if not _2e6839f2:
        return 0

    _76e4731a = _1b5849b8 = _2e6839f2[0]

    for i in range(1, len(_2e6839f2)):
        if _2e6839f2[i] > _2e6839f2[i - 1]:
            _1b5849b8 *= _2e6839f2[i]
        else:
            _1b5849b8 = _2e6839f2[i]
        _76e4731a = max(_76e4731a, _1b5849b8)

    return _76e4731a

Note that all three programs return the same outputs for identical inputs. You can define the function _cc95865c and pass inputs and it will work fine. So, if a model is sufficiently able to “reason” about variable names, it should be able to detect bugs in a similar manner to the original code. However, because of the high perplexity from these hashed strings, any self-preference effects arising from low perplexity in self-generated code will be diminished.

Also note that semantic information contained in variable names is destroyed or obfuscated as well by these transformations. If an identifier is useful at defining what a variable is supposed to do, getting rid of that information can harm a model’s ability to review.

Results

Manipulation check: Does our hashed variable intervention increase perplexity?

First, I verified that my proposed mechanism actually worked to increase code chunk perplexity. For both problem IDs I chose, and across all model sources, perplexity increased with hash-appended and hash-replaced variable names vs the baseline of the raw code chunk.

Note that the sample size here is pretty small; I haven’t quite talked myself into buying a GPU yet. But, all examples showed higher perplexity after my transformations were applied than the original.

Also note that to really understand the causal pathway here, I should probably get access to logprobs for gemini-3.5-flash-lite and gpt-5.6-luna. But this will have to do for now.

Obfuscation and bug detection in self-generated code.

Here’s a chart showing bug detection rates by code transformation. There is no clear reversal of the self-preference trend with the transformations applied.

Here are some logistic regression outputs confirming the results. Neither transformation had a significant interaction with reviewer/generator concordance, so we don’t have evidence to believe that this transformation is useful based on the current dataset.

	odds_ratio	ci_low	ci_high	p_value
Intercept	0.511	0.309	0.847	0.009
C(transform, Treatment(‘original’))[T.hash_append]	0.899	0.728	1.109	0.321
C(transform, Treatment(‘original’))[T.hash_replace]	0.867	0.676	1.111	0.258
C(source)[T.gpt]	0.797	0.491	1.293	0.357
C(reviewer_model)[T.gpt]	0.993	0.664	1.486	0.974
concordant_model	0.752	0.556	1.016	0.064
C(transform, Treatment(‘original’))[T.hash_append]:concordant_model	1.015	0.723	1.425	0.931
C(transform, Treatment(‘original’))[T.hash_replace]:concordant_model	1.141	0.803	1.622	0.461

A more useful framing: does the reviewer model “understand” the concepts involved in a given problem?

Given the results above, I don’t think that a perplexity-derived self-preference effect is the main limiting factor for automated code review. While doing manual review of coding failures, I didn’t find that many instances of a generator LLM not “understanding” what the code was doing, but I did find cases where the generator LLM did not have a sufficient understanding of concepts involved to solve the problem. For example, one problem asked LLMs to identify undulating numbers and both LLM generators created a function that identified non-trivial undulating numbers, which makes sense given that these models are likely distilled from larger models, and the difference between trivial and non-trivial undulating numbers was probably not sufficiently important to make it through the distillation process.

In my experiments on LLM-enabled bug detection, there was a clear pattern where a model’s bug-detection capabilities were much higher on problems that it solved correctly itself. So, if a model “understood” some concept well enough to generate correct code in the first place, it was likely to identify errors in code attempting to solve the same problem. Note that this isn’t really a clean experiment (yep, still trying to talk myself into buying a GPU, and I’ll need a few of those to run a clean experiment there) but it does make intuitive sense.

To conclude, I’ll share some somewhat relevant ideas for good performance on LLM-assisted coding.

1. Have a stronger model make a plan for exactly what to do, including edge cases for unit tests. This can be done in relatively few turns, requiring less token usage overall. The stronger model can include relevant details (like considerations on non-trivial undulating numbers) to the weaker model if needed.
2. Have a weaker model implement the code as well as the tests (this may take relatively more turns to get everything working correctly)
3. Have a stronger model review the weaker model’s outputs