I’m working with OpenNMT -py on a multiple modality to sequence task. Essentially summarization, but from text as well as numerical attributes. I want to use a mean encoder over the attribute section of the source but a bi-directional rnn over the text segment of the source.
Is anyone familiar with using more than one encoder in a model? Are there any resources I can look at?