Hi there! 👋 My name is Robert Morain. I am currently working on my PhD in Computer Science at Brigham Young University. I research knowledge integration in pre-trained language models.

Here, I will share my most recent work! If you would like to get in touch, feel free to reach out at: robert.morain@gmail.com.

Recent Work

Symbolic Semantic Memory in Transformer Language Models

This paper demonstrates how transformer language models can be improved by giving them access to relevant structured data extracted from a knowledge base. The knowledge base preparation process and modifications to transformer models are explained. We evaluate these methods on language modeling and question answering tasks. These results show that even simple additional knowledge augmentation leads to a reduction in validation loss by 73%. These methods also significantly outperform common ways of improving language models such as increasing the model size or adding more data.

Multi-agent Story-based Settlement Generation

Video game content creation is a creative task that has typically been performed by 3D artists. While procedurally generated worlds provide the opportunity to create arbitrarily large cohesive environments, if the generated content lacks an integrated narrative, the environment may begin to feel generic and auto-generated; human artists can use story narratives to help them create 3D worlds which feel more "alive". This paper describes the StoryViz system which uses a short story as inspiration for generating a 3D settlement in Minecraft, leveraging swarm intelligence to optimize a set of rule-based interest functions. A user survey evaluating the system’s generated settlements provides a baseline for further development of the story visualization task.