Associate professor Jianfeng Li and Prof. Hongdong Zhang of Department of Macromolecular Science, Fudan University, and the Professor Jeff Z.-Y. Chen from the University of Waterloo have made new progress in the field of deep learning and physics. They proposed a novel method, other than vector representation, to represent concepts in deep learning. This work has been published in the Physical Review Letters (Structural Prediction and Inverse Design by a Strongly Correlated Neural Network, Phys. Rev. Lett. 123, 2019, 108002).

     When applying neural networks to natural language processing (NLP) or other research areas, it is first necessary to properly and adequately represent the concepts (words or physical entities) involved. Previously, people normally used one-hot encoding or vector representation. Although these two representations are able to describe the features of the concept, they cannot account for the influence of the environment on its attributes, nor the function of the concept.

    “So I am often wondering: Does the brain really use a static vector to represent concepts and physical entities? I don’t think so. For example, when we speak of bicycles, the concept itself should have its own 'construction', and it has something to do with the environment. The image of the bike can be different viewing in different angles. Therefore, we want to upgrade the representation of the concept from vector representation to operator representation." Jianfeng Li said.
      Based on this idea, the team creatively proposed the idea of representing entities in terms of operators. Specifically, a sub-neural network is used to represent a physical entity: the input of the network can be seen as the influence of the environment in which the physical entity is located, and the output of the network represents the physical attributes exhibited by the physical entity in the environment.
      The environment is determined by the physical attributes of other entities. This self-consistent iteration forms a strong correlation between the physical attributes and the environment, thereby greatly improving the accuracy of neural network prediction. This method is currently used in the structural prediction and reverse design of HP protein models, and has achieved high precisions. They found that the new method is particularly suitable for solving problems involving degeneracy.
 

Article link:

https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.123.108002