Enhancing Knowledge Graph Embedding's via Logic Regularized Graph Neural Networks
DOI:
https://doi.org/10.65204/djes.v3i3.688Keywords:
Neuro-Symbolic AI, Knowledge Graph , Graph Neural Networks(GNN), Logic Regularization , Semantic ConsistencyAbstract
In (high)-dimensional data, deep learning-based Knowledge Graph Embedding (KGE) models have shown better capacity in learning the true pattern of latent features. However, they are typically used as "black boxes", where statistical correlation is preferred over explicit logical inference. These are geometrically feasible but semantically invalid predictions, do not satisfy self-explaining constraints. In this paper, we present an Logic Regularized Graph Neural Network (LRGNN) as a neuro-symbolic framework to bridge the gap between the expressive power of deep learning and formal logic's semantic-accuracy. In contrast to traditional non-differentiable logic based approaches where logic and learning are separated, we introduce a new child-sum product T-norm loss that is differentiable into the learning objectives. We wish to extract high confidences compositional rules (e.g. triangle patterns) explicitly from data, and use them as constraints to control the optimization landscape – this is because it seems impossible for humans to learn all patterns and effects purely by observation of training data. As one can see from the experiments on the FB15k-237 benchmark, even after removing all previous feature information for shallow logic inference, LR-GNN still keeps a competitive training convergence (Loss: 0.0349) and sufficiently testified that the logic with confidence (99.56%). these findings affirm that neuro symbolic regularization in practice prunes the embedding space of logically invalid relations, yielding a robust and interpretable alternative to fully data driven models.
References
References
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems (NeurIPS), 26, 2787–2795.
Galárraga, L., Bazan, C., Hosseini, K., & Suchanek, F. M. (2015). Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal, 24(6), 707-730.
Guo, S., Wang, Q., Wang, L., Wang, B., & Guo, L. (2016). Jointly embedding knowledge graphs and logical rules. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 192–202.
Hamilton, W., Ying, Z., & Leskovec, J. (2017). Inductive representation learning on large graphs. Advances in Neural Information Processing Systems (NeurIPS), 30.
Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G. D., Gutierrez, C., ... & Zimmermann, A. (2021). Knowledge graphs. ACM Computing Surveys (CSUR), 54(4), 1-37.
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2021). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33(2), 494-514.
Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR).
Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015). Learning entity and relation embeddings for knowledge graph completion. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1).
Nathani, D., Chauhan, J., Sharma, C., & Kaul, M. (2019). Learning attention-based embeddings for relation prediction in knowledge graphs. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 4710-4723.
Nickel, M., Rosasco, L., & Poggio, T. (2016). Holographic embeddings of knowledge graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
Qu, M., & Tang, J. (2019). Probabilistic logic neural networks for reasoning. Advances in Neural Information Processing Systems (NeurIPS), 32.
Rocktäschel, T., Singh, S., & Riedel, S. (2015). Injecting logical background knowledge into embeddings for relation extraction. Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 1119–1129.
Schlichtkrull, M., Kipf, T. N., Bloem, P., Van Den Berg, R., Titov, I., & Welling, M. (2018). Modeling relational data with graph convolutional networks. European Semantic Web Conference (ESWC), 593-607.
Sun, Z., Deng, Z. H., Nie, J. Y., & Tang, J. (2019). RotatE: Knowledge graph embedding by relational rotation in complex space. International Conference on Learning Representations (ICLR).
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., & Bouchard, G. (2016). Complex embeddings for simple link prediction. International Conference on Machine Learning (ICML), 2071-2080.
Tunyasuvunakool, S., Mocz, A., Kohli, P., & Gilmer, J. (2020). Logic integration for graph neural networks. arXiv preprint arXiv:2003.01602.
Vashishth, S., Sanyal, S., Nitin, V., & Talukdar, P. (2020). Composition-based multi-relational graph convolutional networks. International Conference on Learning Representations (ICLR).
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2018). Graph attention networks. International Conference on Learning Representations (ICLR).
Wang, H., Ren, H., & Leskovec, J. (2021). Relational message passing for knowledge graph completion. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 1697-1707.
Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014). Knowledge graph embedding by translating on hyperplanes. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1).
Wang, X., He, X., Cao, Y., Liu, M., & Chua, T. S. (2019). KGAT: Knowledge graph attention network for recommendation. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 950-958.
Wang, P., Ren, Z., & Xiao, C. (2022). Logic-attention based neighborhood aggregation for inductive knowledge graph reasoning. IEEE Transactions on Knowledge and Data Engineering (TKDE).
Yang, B., Yih, W. T., He, X., Gao, J., & Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. International Conference on Learning Representations (ICLR).
Zhang, J., Chen, B., Wang, X., & Chen, H. (2021). MEGALR: Multi-view embedding for graph analytic logic reasoning. Information Sciences, 571, 158-176.
Zhu, Q., Wei, X., & Wang, B. (2023). A neuro-symbolic approach for interpretable knowledge graph reasoning. Knowledge-Based Systems, 260, 110156.
