Does AI-Encoded Meaning Align with Human Meaning?

2.1 Zipf's law modelling

Let $f$ and $m$ denote the occurrence frequency of a word and the number of its meanings, respectively. Zipf's meaning–frequency law can be formulated as: $m\propto f^{\alpha }$.

Following representative studies[1], this relationship is examined through regression analysis, where $f$ and $m$ serve as the explanatory and dependent variables, respectively. To support robust linear estimation, we have ${\log }_{10}\left(m\right)=\alpha {\log }_{10}\left(f\right)+\beta$, where β is a constant that depends, in part, on the size and composition of the corpus.

Consistent with common practice in previous studies, the regression is conducted using the mean values of $m$ and $f$ within frequency-based bins. Specifically, all word types are sorted in descending order by frequency, and a fixed bin size λ is adopted, such as λ=100. For the $j$-th bin, word types satisfy: $\lambda (j-1)+1\le i\le \lambda j$, where $j=1,2,3,\cdots ,\mathrm{round}\left(\frac{n}{\lambda }\right)$. The coefficient of determination, $R^2$, is used to evaluate the extent to which the meaning–frequency law holds. In addition, a Student's t-test is applied to examine whether the exponent α is significantly different from zero.

2.2 Meaning-Zipf Deviation

For the human-measured meaning distribution $H$ and the AI-encoded meaning distribution $\mathrm{AI}$, we fit Zipf's meaning law: ${\hat{m}}_S\left(f\right)={10}^{{\beta }_S}{f}^{{\alpha }_S},S\in \{H,\mathrm{AI}\}$.

To transform the law into a continuous meaning distribution, let $x={\log }_{10}\left(f\right)$, and the fitted Zipf curve is rewritten as: ${\hat{m}}_S\left(x\right)={10}^{{\alpha }_{S}x+{\beta }_S}$. On the common domain $[x_{\min },x_{\max }]$, we normalize the fitted curve into a continuous probability density: $p_S\left(x\right)=\frac{{\hat{m}}_S\left(x\right)}{{\int }_{x_{\min }}^{x_{\max }}{\hat{m}}_S\left(t\right)dt}$.

Thus, ${\int }_{x_{\min }}^{x_{\max }}{p}_S\left(x\right)dx=1$. Here, $p_S\left(x\right)$ represents how the Zipf meaning law allocates meaning mass over the logarithmic frequency space. After normalization, the scale effect of β is removed; the comparison therefore focuses mainly on the distributional shape of the two Zipf laws, especially the allocation tendency determined by the α.

To quantify the discrepancy between the two distributions, we present calculus-based Jensen–Shannon distance. Let $q\left(x\right)=\frac{p_H\left(x\right)+p_{\mathrm{AI}}\left(x\right)}{2}$, and we have:

We normalize it to the [0,1]: $D_{\mathrm{JS}}(H,\mathrm{AI})=\sqrt{\frac{\mathrm{JS}(p_H,p_{\mathrm{AI}})}{\log 2}}$. $D_{\mathrm{JS}}=0$ → distributions are identical; $D_{\mathrm{JS}}=1$ → maximum possible divergence between them.

If either side has a low $R^2$, it suggests that the corresponding distribution substantially deviates from Zipf's meaning law. Therefore, $R^2$ is introduced as a reliability penalty. Let ε be a small positive constant to avoid division by zero, and let $\gamma \in [0,1]$ control the strength of the penalty. The reliability weight is: $W_{R^2}(H,\mathrm{AI})={\left(\max \right(\epsilon ,R_H^2\left)\max \right(\epsilon ,R_{\mathrm{AI}}^2\left)\right)}^{\gamma /2}$.

The Meaning-Zipf Deviation is then defined as:

MZD lies in the interval [0,1]. If the two meaning distributions are identical and both fitted Zipf laws have perfect goodness of fit, then MZD = 0. If the two distributions are highly divergent, or if either fitted law has a low $R^2$, then MZD → 1. Therefore, MZD jointly captures two aspects: whether the Zipfian structure encoded by the AI resembles the human-measured distribution, and whether the two fitted laws themselves are statistically reliable.

3.1 Data and Setting

We use four representative corpora covering distinct language settings: Project Gutenberg for literary texts[5], WikiText for encyclopedic writing[6], Brown Corpus for balanced general English[7], and arXiv for academic papers[4]. These corpora differ in style, topical structure, domain specificity, and lexical distribution, providing a heterogeneous basis for examining meaning–frequency relationships. All texts are lowercased, and English word types are extracted using regular expressions by retaining only alphabetic tokens. Special symbols are removed, and spaCy lemmatization is applied to reduce inflectional variants to lemma forms. Table 1 reports the statistics of the cleaned corpora, where Hapax type share denotes the proportion of word types that occur only once.

To examine the effect of model choice on AI-based meaning estimation, we evaluate 3 model families: BERT (small, base, large), GPT-2 (small, medium, large), and Qwen3 models ranging from 0.6B to 8B parameters. For each model, contextualized word representations are obtained by averaging hidden states from the final several layers. We use the same occurrence-level extraction strategy across models, with batch size and maximum input length adjusted according to model scale. Corpus frequency is computed at the word level rather than the token or subword level, so frequency and meaning count share the same unit. Each experiment is repeated 5 times, and mean values are reported.

3.2 Human-measured meaning and Zipf's meaning law

We first examine whether human-measured meaning conforms to Zipf's meaning law. Figure B1 (Appendix B) shows that human lexical meanings consistently exhibit a Zipfian meaning–frequency pattern. The estimated scaling exponent αis positive in all corpora, with all p < 10⁻⁴. The goodness of fit is also high: R² = 0.870 for arXiv, 0.916 for Brown, 0.931 for Gutenberg, and 0.926 for Wiki. These results confirm the robustness of Zipf's meaning law in human lexical systems.

The α varies across corpora, suggesting that meaning allocation is shaped by corpus type. To interpret this variation, we divide each corpus into 500 fixed-length chunks and compute lexical indicators including MSTTR, normalized lexical entropy, top-100 word share, low-frequency word share, Gini coefficient, long-word proportion, top-50 bigram share, corpus-specific log-odds, and cross-entropy relative to Brown. Kruskal–Wallis tests show significant cross-corpus differences for all indicators, with all p < 10⁻⁴ (Figure B2 in Appendix B), confirming that the corpora differ substantially in lexical organization.

arXiv has the lowest exponent, α = 0.173, together with the highest domain log-odds, cross-entropy to Brown, long-word proportion, top-100 word share, and Gini coefficient. This indicates a highly specialized and concentrated vocabulary structure. Brown has the highest exponent, α = 0.343, along with the highest MSTTR and normalized lexical entropy, and the lowest concentration and domain-shift indicators, reflecting balanced and dispersed general-language usage. Gutenberg shows an intermediate exponent, α = 0.283, with high lexical diversity but the lowest long-word proportion, suggesting that literary diversity mainly arises from stylistic and narrative variation rather than technical terminology. Wiki has a lower exponent, α = 0.229, and high cross-entropy to Brown and top-50 bigram share, consistent with standardized encyclopedic writing and recurrent knowledge expressions.

Overall, human-measured meanings robustly follow Zipf's meaning law, but the strength and interpretation of the scaling relationship vary with corpus type. This establishes a necessary human baseline for comparing AI-encoded meaning distributions.

3.3 AI-encoded meaning and Zipf's meaning law

Figure B3–Figure B5 (Appendix B) show that AI-based meaning estimates can also exhibit Zipfian meaning–frequency patterns: more frequent words generally receive higher estimated meaning quantities. However, the strength of this relationship varies substantially across measurement methods, which is expected given that different operationalizations capture different aspects of AI-encoded meaning.

Clear differences emerge among the three AI-based meaning measures. The EVU- and CRC-based measures are relatively stable, often reaching good (R² > 0.8) or even excellent (R² > 0.9) goodness of fit, and the CDD-based measure shows weaker fitting performance. This suggests that language models, after being trained on large-scale human-generated corpora, can inherit certain structural regularities of human lexical meaning. Nevertheless, the degree to which such regularities are expressed depends strongly on how model-based number of meanings is defined.

Although larger AIs are often associated with stronger general intelligence and broader semantic understanding[8], our results indicate that this assumption does not straightforwardly hold for number of word meanings. Larger models do not necessarily conform more closely to Zipf's meaning law. In the BERT series, some corpora and measurement settings show improved fitting performance as model size increases. In the GPT-2 series, GPT-2-large tends to be more stable than the small and medium variants. However, in the Qwen series, larger models do not consistently produce higher slopes or better R² values; in some cases, both even declines. These results indicate that greater model scale does not automatically translate into a meaning structure that more closely resembles the human lexical system.

Corpus type also plays an important role. AI's Zipfian behaviour is not constant across corpora. In knowledge-oriented or literary corpora, the relationship between number of AI-estimated meanings and word frequency is more likely to display a stable Zipfian pattern. By contrast, balanced general corpora, such as Brown, can be more difficult for some models to capture consistently.

3.4 Alignment between AI-encoded and human-measured meaning

Figure B6 (Appendix B) visualizes the Meaning-Zipf Deviation (MZD) between AI-encoded and human-measured meaning distributions. Across the 120 model–corpus combinations, the mean MZD is 0.331 and the median is 0.246, indicating a substantial deviation between AI and human meaning structures.

At the corpus level, Wiki has the lowest average MZD, 0.284, suggesting that AI-encoded meaning structures are relatively closer to human-measured meaning in encyclopedic texts. Brown has the highest average MZD, 0.365, indicating that balanced general-language corpora place stronger demands on AI–human meaning alignment. The average MZD values for arXiv and Gutenberg are 0.341 and 0.336, respectively, falling between Wiki and Brown. These results suggest that the alignment between AI and human meaning systems is shaped by the way words are used across different textual environments.

A.1 AI-encoded meaning

Word meaning is central to AI language understanding, since tasks such as retrieval, question answering, knowledge extraction, and text generation all require models to determine how words refer, associate, and shift across contexts[8, 21, 25]. Early work relied on the distributional hypothesis, representing words as static vectors in models such as Word2Vec and GloVe. This made word meaning computationally measurable, but compressed all senses of a word into a single representation[30].

Pretrained language models changed this view by introducing contextualized representations. Models such as BERT, ELMo, and GPT-2 assign different representations to the same word in different contexts, with upper layers becoming more context-dependent. Subsequent studies show that such representations can partially encode polysemy, sense relatedness, and sense separability[12–14]. Recent work further derives explicit sense representations through word sense induction and word sense embeddings, or measures semantic breadth through non-point embeddings and contextual vector dispersion[1–3].

Overall, existing research shows that AI can represent word meaning, differentiate polysemy, and reveal meaning-related statistical patterns. Yet a fundamental question remains unresolved: whether the meanings encoded by AI can reach the human word meanings.

A.2 Zipf's meaning law

Zipf's meaning law links word use with word meaning. It states that more frequent words tend to have more meanings or broader semantic applicability, while this semantic expansion grows sublinearly. The law reflects the principle of linguistic economy: human language balances expressive efficiency, cognitive cost, and communicative demand. High-frequency forms tend to carry richer semantic functions, whereas low-frequency forms are usually more specific. This pattern has been related to the tension between unification and diversification in communication[18], and provides an empirical entry point for measuring how semantic load is distributed across the lexicon[17].

Empirical studies have provided cross-linguistic evidence. Casas et al. define the meaning–frequency law as a positive association between word frequency and WordNet polysemy, and test it in English, Spanish, and Dutch[16]. Bond et al. further examine the law with multilingual wordnets across languages including English, Spanish, Portuguese, French, Polish, Japanese, Indonesian, and Chinese[15]. These studies show that the law is more stable at the aggregate level, especially when average meaning counts and frequency ranks are used, but weaker for predicting the number of senses of individual words.

Accordingly, Zipf's meaning law should be treated as a macro-level regularity rather than a word-level rule. Individual sense counts may vary with lexicographic conventions, part of speech, register, corpus source, and semantic change[22]. This is crucial for our study: when comparing human and AI meaning distributions, the central question is not whether each individual word has the same sense count, but whether systematic deviations emerge in the overall meaning–frequency structure[23]. Communication-optimization models further support this view by showing that dependencies between form frequency and number of meanings can emerge from constraints on expressive economy, information transmission, and form–meaning mapping[24].