Construction of Paired Knowledge Graph – Text Datasets Informed by Cyclic Evaluation

Datasets that pair Data Graphs (KG) and textual content collectively (KG-T) can be utilized to coach ahead and reverse neural fashions that generate textual content from KG and vice versa. Nonetheless fashions skilled on datasets the place KG and textual content pairs are usually not equal can undergo from extra hallucination and poorer recall. On this paper, we confirm this empirically by producing datasets with completely different ranges of noise and discover that noisier datasets do certainly result in extra hallucination. We argue that the power of ahead and reverse fashions skilled on a dataset to cyclically regenerate supply KG or textual content is a proxy for the equivalence between the KG and the textual content within the dataset. Utilizing cyclic analysis we discover that manually created WebNLG is significantly better than mechanically created TeKGen and T-REx. Knowledgeable by these observations, we assemble a brand new, improved dataset referred to as LAGRANGE utilizing heuristics meant to enhance equivalence between KG and textual content and present the influence of every of the heuristics on cyclic analysis. We additionally assemble two artificial datasets utilizing giant language fashions (LLMs), and observe that these are conducive to fashions that carry out considerably nicely on cyclic technology of textual content, however much less so on cyclic technology of KGs, in all probability due to a scarcity of a constant underlying ontology.