What historical knowledge do language models encode?

  • Lenguador@kbin.social
    link
    fedilink
    arrow-up
    1
    ·
    1 year ago

    GPT-4 was fine-tuned on English and Chinese instruction examples only (source). There’s clearly some western bias in the historic events, but it would have been interesting to also discuss if there was a bias towards Chinese events as well. And, if so, what other languages or prompts may elicit that bias.
    As an example, could you get the model to have an English bias with “I’m from America…” and a Chinese bias with “I’m from China…” even when using English?