EACL 2022

• German
• Italian
• Chinese
• Japanese

TLDRs

• Unsupervised Sentence-embeddings by Manifold Approximation and Projection
  • Subhradeep Kayal
  • TLDR: We propose a novel technique to generate sentence embeddings in an unsupervised fashion by projecting the sentences onto a fixed-dimensional manifold with the objective of preserving local neighbourhoods in the original space.
• Contrastive Multi-document Question Generation
  • Woon Sang Cho, Yizhe Zhang, Sudha Rao, Asli Celikyilmaz, Chenyan Xiong, Jianfeng Gao, Mengdi Wang, Bill Dolan
  • TLDR: We propose a novel framework for multi-document question generation that uses contrastive learning to generate more specific and related questions.
• Disambiguatory Signals are Stronger in Word-initial Positions
  • Tiago Pimentel, Ryan Cotterell, Brian Roark
  • TLDR: We show that languages have evolved to provide more information earlier in words than later in the word, and present several new measures that avoid these confounds.
• On the (In)Effectiveness of Images for Text Classification
  • Chunpeng Ma, Aili Shen, Hiyori Yoshikawa, Tomoya Iwakura, Daniel Beck, Timothy Baldwin
  • TLDR: We show that images complement NLP models (including BERT) trained without external pre-training, but when combined with BERT models pre-trained on large-scale external data, images contribute nothing.
• If you’ve got it, flaunt it: Making the most of fine-grained sentiment annotations
  • Jeremy Barnes, Lilja Øvrelid, Erik Velldal
  • TLDR: We propose to incorporate holder and expression information into the prediction of target and polarity BIO labels for fine-grained sentiment analysis and show that this information improves target extraction and classification.
• Keep Learning: Self-supervised Meta-learning for Learning from Inference
  • Akhil Kedia, Sai Chetan Chinthakindi
  • TLDR: We propose a new method for fine-tuning a language modelling model during inference using pseudo-labels, which improves performance on a wide variety of tasks.
• ResPer: Computationally Modelling Resisting Strategies in Persuasive Conversations
  • Ritam Dutt, Sayan Sinha, Rishabh Joshi, Surya Shekhar Chakraborty, Meredith Riggs, Xinru Yan, Haogang Bao, Carolyn Rose
  • TLDR: We propose a generalised framework for identifying resisting strategies in persuasive conversations and show the benefits of incorporating resisting strategies on the final conversation outcome.
• BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression
  • Ji Xin, Raphael Tang, Yaoliang Yu, Jimmy Lin
  • TLDR: We propose a new fine-tuning strategy for early exiting models that can take full advantage of BERT.
• Telling BERT’s Full Story: from Local Attention to Global Aggregation
  • Damian Pascual, Gino Brunner, Roger Wattenhofer
  • TLDR: We take a deep look into the behaviour of self-attention heads in the transformer architecture.
• Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection
  • Jens Kaiser, Sinan Kurtyigit, Serge Kotchourko, Dominik Schlechtweg
  • TLDR: We optimize existing models by pre-training on large corpora and refining on diachronic target corpora tackling the notorious small data problem, and applying post-processing transformations that have been shown to improve performance on synchronic tasks.
• The Gutenberg Dialogue Dataset
  • Richard Csaky, Gábor Recski
  • TLDR: We present a dialogue extraction pipeline for public-domain dialogue datasets and show that it can be extended to further languages with little additional effort.
• On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search
  • Gustavo Penha, Claudia Hauff
  • TLDR: We propose a new method for probabilizing neural rankers for conversational search problems by estimating uncertainty of neural ranker.
• Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples
  • Maximilian Mozes, Pontus Stenetorp, Bennett Kleinberg, Lewis Griffin
  • TLDR: We show that adversarial attacks against CNN, LSTM and Transformer-based classification models perform word substitutions that are identifiable through frequency differences between replaced words and their corresponding substitutions.
• Maximal Multiverse Learning for Promoting Cross-Task Generalization of Fine-Tuned Language Models
  • Itzik Malkiel, Lior Wolf
  • TLDR: We present a method that leverages the second phase of language modeling with BERT to its fullest, by applying an extensive number of parallel classifier heads, which are enforced to be orthogonal, while adaptively eliminating the weaker heads during training.
• Unification-based Reconstruction of Multi-hop Explanations for Science Questions
  • Marco Valentino, Mokanarangan Thayaparan, André Freitas
  • TLDR: We propose a novel framework for reconstructing multi-hop explanations in science Question Answering using the corpus of scientific explanations.
• Dictionary-based Debiasing of Pre-trained Word Embeddings
  • Masahiro Kaneko, Danushka Bollegala
  • TLDR: We propose a method for debiasing pre-trained word embeddings using dictionaries, without requiring access to the original training resources or any knowledge regarding the word embedding algorithms used.
• Belief-based Generation of Argumentative Claims
  • Milad Alshomary, Wei-Fan Chen, Timon Gurcke, Henning Wachsmuth
  • TLDR: We propose a new method of belief-based claim generation and show empirical evidence of its applicability to argument generation.
• Non-Autoregressive Text Generation with Pre-trained Language Models
  • Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier
  • TLDR: BERT can be employed as the backbone of a NAG model for a greatly improved performance.
• Multi-split Reversible Transformers Can Enhance Neural Machine Translation
  • Yuekai Zhao, Shuchang Zhou, Zhihua Zhang
  • TLDR: We present a new multi-split based reversible transformer for large-scale translation that achieves very strong translation accuracy and memory efficiency.
• Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference
  • Timo Schick, Hinrich Schütze
  • TLDR: We propose Pattern-Exploiting Training, a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task.
• CDˆ2CR: Co-reference resolution across documents and domains
  • James Ravenscroft, Amanda Clare, Arie Cattan, Ido Dagan, Maria Liakata
  • TLDR: We propose a new task and English language dataset for cross-document cross-domain co-reference resolution (CDˆ2CR) and a new model for cross document cross-word co-referencing.
• AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization
  • Keping Bi, Rahul Jha, Bruce Croft, Asli Celikyilmaz
  • TLDR: We present a novel adaptive learning model for extractive summarization that learns to balance salience and redundancy in a novel way.
• “Talk to me with left, right, and angles”: Lexical entrainment in spoken Hebrew dialogue
  • Andreas Weise, Vered Silber-Varod, Anat Lerner, Julia Hirschberg, Rivka Levitan
  • TLDR: We show that lexical entrainment in Hebrew is not only a phenomenon in terms of generalization, but also affects lexical choice.
• Recipes for Building an Open-Domain Chatbot
  • Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, Jason Weston
  • TLDR: We present a new approach to building open-domain chatbots that learns conversational skills by scaling up the number of parameters and generation strategy.
• Evaluating the Evaluation of Diversity in Natural Language Generation
  • Guy Tevet, Jonathan Berant
  • TLDR: We propose a framework for evaluating diversity of natural language generation models.
• Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering
  • Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang
  • TLDR: We propose a new method for question answering over knowledge bases that uses a simple search algorithm to identify topic entities and relations.
• Implicitly Abusive Comparisons – A New Dataset and Linguistic Analysis
  • Michael Wiegand, Maja Geulig, Josef Ruppenhofer
  • TLDR: We present a novel dataset for detecting implicitly abusive comparisons and present a range of linguistic features that help us better understand the mechanisms underlying abusive comparisons.
• Exploiting Emojis for Abusive Language Detection
  • Michael Wiegand, Josef Ruppenhofer
  • TLDR: We propose to use abusive emojis, such as the “middle finger” or “face vomiting”, as a proxy for learning a lexicon of abusive words.
• A Systematic Review of Reproducibility Research in Natural Language Processing
  • Anya Belz, Shubham Agarwal, Anastasia Shimorina, Ehud Reiter
  • TLDR: We provide a wide-angle, and as near as possible complete, snapshot of current work on reproducibility in NLP, and provide a framework for future work.
• Bootstrapping Multilingual AMR with Contextual Word Alignments
  • Janaki Sheth, Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Radu Florian, Salim Roukos, Todd Ward
  • TLDR: We develop high performance multilingual abstract meaning representation systems by projecting English AMR annotations to other languages with weak supervision.
• Semantic Oppositeness Assisted Deep Contextual Modeling for Automatic Rumor Detection in Social Networks
  • Nisansa de Silva, Dejing Dou
  • TLDR: We show that semantic oppositeness captures elements of discord, which are not properly covered by previous efforts, which only utilize semantic similarity or reply structure.
• Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation
  • Vikash Balasubramanian, Ivan Kobyzev, Hareesh Bahuleyan, Ilya Shapiro, Olga Vechtomova
  • TLDR: We propose polarized-VAE, an approach that disentangles select attributes in the latent space based on proximity measures reflecting the similarity between data points with respect to these attributes.
• ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation
  • Qingxiu Dong, Xiaojun Wan, Yue Cao
  • TLDR: We propose ParaSCI, the first large-scale paraphrase dataset in the scientific field, including 33,981 paraphrase pairs from ACL (ParaSCI-ACL) and 316,063 pairs from arXiv (PARASCI).
• Discourse Understanding and Factual Consistency in Abstractive Summarization
  • Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
  • TLDR: We introduce a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary.
• Knowledge Base Question Answering through Recursive Hypergraphs
  • Naganand Yadati, Dayanidhi R S, Vaishnavi S, Indira K M, Srinidhi G
  • TLDR: We propose a new method for KBQA based on hypergraphs to answer knowledge base question answering.
• FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary
  • Terra Blevins, Mandar Joshi, Luke Zettlemoyer
  • TLDR: We present a new low-shot WSD dataset for rare senses and show that models trained with FEWS better capture rare senses in existing WSD datasets.
• MONAH: Multi-Modal Narratives for Humans to analyze conversations
  • Joshua Y. Kim, Kalina Yacef, Greyson Kim, Chunfeng Liu, Rafael Calvo, Silas Taylor
  • TLDR: We propose a system that automatically expands the verbatim transcripts of video-recorded conversations using multimodal data streams and show that the expansion leads to statistically significant improvements in detecting rapport-building.
• Does Typological Blinding Impede Cross-Lingual Sharing?
  • Johannes Bjerva, Isabelle Augenstein
  • TLDR: We show that a model trained in a cross-lingual setting will pick up on typological cues from the input data, thus overshadowing the utility of explicitly using such features.
• AdapterFusion: Non-Destructive Task Composition for Transfer Learning
  • Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, Iryna Gurevych
  • TLDR: We propose AdapterFusion, a new two stage learning algorithm that leverages knowledge from multiple tasks.
• CHOLAN: A Modular Approach for Neural Entity Linking on Wikipedia and Wikidata
  • Manoj Prabhakar Kannan Ravi, Kuldeep Singh, Isaiah Onando Mulang’, Saeedeh Shekarpour, Johannes Hoffart, Jens Lehmann
  • TLDR: We propose CHOLAN, a modular approach to target end-to-end entity linking (EL) over knowledge bases.
• Grounding as a Collaborative Process
  • Luciana Benotti, Patrick Blackburn
  • TLDR: We argue that making mistakes and being able to recover from them collaboratively is a key ingredient in grounding meaning.
• Does She Wink or Does She Nod? A Challenging Benchmark for Evaluating Word Understanding of Language Models
  • Lutfi Kerem Senel, Hinrich Schütze
  • TLDR: We present a new probing task to evaluate word understanding in language models by querying dictionary definitions of words.
• Joint Coreference Resolution and Character Linking for Multiparty Conversation
  • Jiaxin Bai, Hongming Zhang, Yangqiu Song, Kun Xu
  • TLDR: We propose a novel model for character linking and coreference resolution, which can significantly outperform previous works on both tasks.
• Improving Factual Consistency Between a Response and Persona Facts
  • Mohsen Mesgar, Edwin Simpson, Iryna Gurevych
  • TLDR: We propose to fine-tune neural models for response generation by reinforcement learning and an efficient reward function that explicitly captures the consistency between a response and persona facts as well as semantic plausibility.
• PolyLM: Learning about Polysemy through Language Modeling
  • Alan Ansell, Felipe Bravo-Marquez, Bernhard Pfahringer
  • TLDR: We present PolyLM, a language modeling method for learning sense embeddings which matches the current state-of-the-art specialized WSI method despite having six times fewer parameters.
• Predicting Treatment Outcome from Patient Texts:The Case of Internet-Based Cognitive Behavioural Therapy
  • Evangelia Gogoulou, Magnus Boman, Fehmi Ben Abdesslem, Nils Hentati Isacsson, Viktor Kaldo, Magnus Sahlgren
  • TLDR: We investigate the feasibility of applying standard text categorisation methods to patient text in order to predict treatment outcome in Internet-based cognitive behavioural therapy.
• Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning
  • Alon Jacovi, Gang Niu, Yoav Goldberg, Masashi Sugiyama
  • TLDR: We propose a novel approach to retrieve documents on cohesive topics using positive-unlabeled PU learning.
• The Role of Syntactic Planning in Compositional Image Captioning
  • Emanuele Bugliarello, Desmond Elliott
  • TLDR: We propose a new method to improve compositional generalization in image captioning by planning the syntactic structure of a caption.
• Is “hot pizza” Positive or Negative? Mining Target-aware Sentiment Lexicons
  • Jie Zhou, Yuanbin Wu, Changzhi Sun, Liang He
  • TLDR: We propose a new method for extracting sentiment lexicons that can be used to model a word’s polarity in different contexts.
• Quality Estimation without Human-labeled Data
  • Yi-Lin Tuan, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Francisco Guzmán, Lucia Specia
  • TLDR: We propose a new method for quality estimation using synthetic training data and show that it is comparable to human-annotated data.
• How Fast can BERT Learn Simple Natural Language Inference?
  • Yi-Chung Lin, Keh-Yih Su
  • TLDR: We empirically study whether BERT can really learn to conduct natural language inference without utilizing hidden dataset bias; and how efficiently it can learn if it could.
• GRIT: Generative Role-filler Transformers for Document-level Event Entity Extraction
  • Xinya Du, Alexander Rush, Claire Cardie
  • TLDR: We introduce a generative transformer-based encoder-decoder framework for document-level role-filler entity extraction and show that it performs substantially better than prior work.
• Cross-lingual Entity Alignment with Incidental Supervision
  • Muhao Chen, Weijia Shi, Ben Zhou, Dan Roth
  • TLDR: We propose a new model, JEANS, which jointly represents multilingual KGs and text corpora in a shared embedding scheme, and seeks to improve entity alignment with incidental supervision signals from text.
• Query Generation for Multimodal Documents
  • Kyungho Kim, Kyungjae Lee, Seung-won Hwang, Young-In Song, Seungwook Lee
  • TLDR: We propose a novel method for first-stage retrieval of relevant doc-uments with multimodal images.
• End-to-End Argument Mining as Biaffine Dependency Parsing
  • Yuxiao Ye, Simone Teufel
  • TLDR: We propose a neural end-to-end approach to argument mining which is based on dependency parsing, in contrast to the current state-of-the-art which relies on relation extraction.
• FakeFlow: Fake News Detection by Modeling the Flow of Affective Information
  • Bilal Ghanem, Simone Paolo Ponzetto, Paolo Rosso, Francisco Rangel
  • TLDR: We propose a novel neural architecture for capturing the flow of affective information in fake news articles.
• CTC-based Compression for Direct Speech Translation
  • Marco Gaido, Mauro Cettolo, Matteo Negri, Marco Turchi
  • TLDR: We propose a dynamic compression algorithm for speech translation in direct ST models.
• A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline
  • Yerbolat Khassanov, Saida Mussakhojayeva, Almas Mirzakhmetov, Alen Adiyev, Mukhamet Nurpeiissov, Huseyin Atakan Varol
  • TLDR: We present an open-source speech corpus for the Kazakh language.
• TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics
  • Yufang Hou, Charles Jochim, Martin Gleize, Francesca Bonin, Debasis Ganguly
  • TLDR: We present a new corpus that contains domain expert annotations for Task (T), Datasets and Evaluation Metrics extracted from NLP papers.
• Top-down Discourse Parsing via Sequence Labelling
  • Fajri Koto, Jey Han Lau, Timothy Baldwin
  • TLDR: We introduce a top-down approach to discourse parsing that is conceptually simpler than its predecessors (Kobayashi et al., 2020; Zhang et al, 2020).
• Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning
  • Ernie Chang, Hui-Syuan Yeh, Vera Demberg
  • TLDR: We propose a novel curriculum-based approach for data-to-text generation that improves both the performance and convergence speed.
• TrNews: Heterogeneous User-Interest Transfer Learning for News Recommendation
  • Guangneng Hu, Qiang Yang
  • TLDR: We propose a transfer learning model for cross-corpus news recommendation which is better than various existing transfer learning methods.
• Dialogue Act-based Breakdown Detection in Negotiation Dialogues
  • Atsuki Yamaguchi, Kosui Iwasa, Katsuhide Fujita
  • TLDR: We present a new dialogue dialogue corpus for human-human negotiation support and a dialogue act-based breakdown detection method for human negotiation dialogue.
• Neural Data-to-Text Generation with LM-based Text Augmentation
  • Ernie Chang, Xiaoyu Shen, Dawei Zhu, Vera Demberg, Hui Su
  • TLDR: We propose a novel few-shot approach for data-to-text generation that can improve the performance of sequence-to sequence models by over 5 BLEU points, and also improves the reconstruction accuracy of data samples.
• Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling
  • Muhammad Khalifa, Muhammad Abdul-Mageed, Khaled Shaalan
  • TLDR: We propose to self-train pre-trained language models in zero- and few-shot scenarios to improve performance on data-scarce varieties using only resources from data-rich ones.
• Multiple Tasks Integration: Tagging, Syntactic and Semantic Parsing as a Single Task
  • Timothée Bernard
  • TLDR: We propose Multiple Tasks Integration, a multitask paradigm orthogonal to weight sharing, which is based on all of the structures that are already inferred and free from usual ordering constraints.
• Coordinate Constructions in English Enhanced Universal Dependencies: Analysis and Computational Modeling
  • Stefan Grünewald, Prisca Piccirilli, Annemarie Friedrich
  • TLDR: We propose to learn propagation rules for conjunction propagation in Enhanced Universal Dependencies, and show that they are more effective than hand-designing heuristic rules.
• Ellipsis Resolution as Question Answering: An Evaluation
  • Rahul Aralikatte, Matthew Lamm, Daniel Hardt, Anders Søgaard
  • TLDR: We present a new approach for English ellipsis resolution based on question answering and coreference resolution.
• Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling
  • Ernie Chang, Vera Demberg, Alex Marin
  • TLDR: We propose a novel weakly supervised training paradigm for neural natural language generation and understanding models that outperforms benchmark systems on both datasets.
• Continuous Learning in Neural Machine Translation using Bilingual Dictionaries
  • Jan Niehues
  • TLDR: We propose an evaluation framework to assess the ability of neural machine translation to continuously learn new phrases and words and show that it is important to address both in order to successfully make use of bilingual dictionaries.
• Adv-OLM: Generating Textual Adversaries via OLM
  • Vijit Malik, Ashwani Bhat, Ashutosh Modi
  • TLDR: We present Adv-OLM, a black-box attack method that adapts the idea of Occlusion and Language Models (OLM) to the current state of the art attack methods.
• Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks
  • Endri Kacupaj, Joan Plepi, Kuldeep Singh, Harsh Thakkar, Jens Lehmann, Maria Maleshkova
  • TLDR: We propose LASAGNE, a transformer architecture extended with Graph Attention Networks for multi-task neural semantic parsing over a knowledge graph.
• DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewriting
  • Hrituraj Singh, Gaurav Verma, Aparna Garimella, Balaji Vasan Srinivasan
  • TLDR: We propose a Director-Generator framework to rewrite input text in the target author’s style, specifically focusing on certain target attributes.
• Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
  • Gautier Izacard, Edouard Grave
  • TLDR: We show that sequence-to-sequence models can benefit from retrieving text passages, potentially containing evidence.
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration
  • Betty van Aken, Jens-Michalis Papaioannou, Manuel Mayrdorfer, Klemens Budde, Felix Gers, Alexander Loeser
  • TLDR: We propose a novel outcome prediction task for admission to discharge and a novel ICD code hierarchy task for clinical text.
• Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification
  • Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen
  • TLDR: We combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task.
• Multi-facet Universal Schema
  • Rohan Paul, Haw-Shiuan Chang, Andrew McCallum
  • TLDR: We propose multi-facet universal schema that uses a neural model to represent each sentence pattern as multiple facet embeddings and encourage one of these facet embeds to be close to that of another sentence pattern if they co-occur with the same entity pair.
• Exploring Transitivity in Neural NLI Models through Veridicality
  • Hitomi Yanaka, Koji Mineshima, Kentaro Inui
  • TLDR: We show that current NLI models do not perform consistently well on transitivity inference tasks, suggesting that they lack the generalization capacity for drawing composite inferences from provided training examples.
• A Neural Few-Shot Text Classification Reality Check
  • Thomas Dopierre, Christophe Gravier, Wilfried Logerais
  • TLDR: We compare several neural few-shot classification models in Computer Vision and NLP, and show that most of them perform worse than older and simpler ones.
• Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders
  • Carlos Escolano, Marta R. Costa-jussà, José A. R. Fonollosa, Mikel Artetxe
  • TLDR: Language-specific encoder-decoders for multilingual machine translation.
• Clustering Word Embeddings with Self-Organizing Maps. Application on LaRoSeDa - A Large Romanian Sentiment Data Set
  • Anca Tache, Gaman Mihaela, Radu Tudor Ionescu
  • TLDR: We present a new large Romanian sentiment data set, which is used for sentiment classification and self-organizing maps.
• Elastic weight consolidation for better bias inoculation
  • James Thorne, Andreas Vlachos
  • TLDR: We show that elastic weight consolidation allows fine-tuning of models to mitigate biases while being less susceptible to catastrophic forgetting.
• Hierarchical Multi-head Attentive Network for Evidence-aware Fake News Detection
  • Nguyen Vo, Kyumin Lee
  • TLDR: We propose a Hierarchical Multi-head Attentive Network to fact-check textual claims.
• Identifying Named Entities as they are Typed
  • Ravneet Arora, Chen-Tse Tsai, Daniel Preotiuc-Pietro
  • TLDR: We present a novel experimental setup for evaluating Named Entity Recognition systems for text editors and text processing applications where decisions about named entity boundaries need to be made in an online fashion.
• SANDI: Story-and-Images Alignment
  • Sreyasi Nag Chowdhury, Simon Razniewski, Gerhard Weikum
  • TLDR: We present SANDI, a methodology for automatically selecting images from an image collection and aligning them with text paragraphs of a story.
• Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
  • Patrick Lewis, Pontus Stenetorp, Sebastian Riedel
  • TLDR: We perform a detailed study of the test sets of three popular open-domain question answering benchmarks and show that most models can generalize to completely novel questions with novel answers.
• El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing
  • Arash Einolghozati, Abhinav Arora, Lorena Sainz-Maza Lecanda, Anuj Kumar, Sonal Gupta
  • TLDR: We propose a dataset for Spanglish utterances and show the generalizability of pre-trained language models when data for only one language is present.
• Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs
  • Kuan-Hao Huang, Kai-Wei Chang
  • TLDR: Syntactically controlled paraphrase generation without the need for annotated paraphrase pairs.
• Data Augmentation for Hypernymy Detection
  • Thomas Kober, Julie Weeds, Lorenzo Bertolini, David Weir
  • TLDR: We present two novel data augmentation techniques for hypernymy detection and dataset extension and demonstrate that both of the proposed methods substantially improve classifier performance.
• Few-shot learning through contextual data augmentation
  • Farid Arthaud, Rachel Bawden, Alexandra Birch
  • TLDR: We propose a novel data augmentation approach for machine translation models that adapt to new data by simulating novel vocabulary appearing in human-submitted translations.
• Zero-shot Generalization in Dialog State Tracking through Generative Question Answering
  • Shuyang Li, Jin Cao, Mukund Sridhar, Henghui Zhu, Shang-Wen Li, Wael Hamza, Julian McAuley
  • TLDR: We present a novel ontology-free framework for dialog state tracking that generalizes to unseen constraints and slots in multi-domain task-oriented dialogs.
• Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
  • Ji Ma, Ivan Korotkov, Yinfei Yang, Keith Hall, Ryan McDonald
  • TLDR: We propose a novel approach to zero-shot learning for passage retrieval that uses synthetic question generation to close the gap between supervised and synthetic neural retrieval models.
• Discourse-Aware Unsupervised Summarization for Long Scientific Documents
  • Yue Dong, Andrei Mircea, Jackie Chi Kit Cheung
  • TLDR: We propose an unsupervised graph-based ranking model for extractive summarization of long scientific documents.
• MIDAS: A Dialog Act Annotation Scheme for Open Domain HumanMachine Spoken Conversations
  • Dian Yu, Zhou Yu
  • TLDR: We present a dialog act annotation scheme for open-domain human-machine conversations and show that it can improve human-human communication.
• Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
  • Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James Glass, Fuchun Peng
  • TLDR: We study how the finetuning stage in the pretrain-finetune framework changes the behavior of a pretrained neural language generator.
• Leveraging End-to-End ASR for Endangered Language Documentation: An Empirical Study on Yolóxochitl Mixtec
  • Jiatong Shi, Jonathan D. Amith, Rey Castillo García, Esteban Guadalupe Sierra, Kevin Duh, Shinji Watanabe
  • TLDR: We propose a combinatory transcription bottleneck and transcriber shortage approach to endangered language documentation.
• Mode Effects’ Challenge to Authorship Attribution
  • Haining Wang, Allen Riddell, Patrick Juola
  • TLDR: We measure the effect of writing mode on writing style in the context of authorship attribution research using a corpus of documents composed online (in a web browser) and documents composed offline using a traditional word processor.
• Generative Text Modeling through Short Run Inference
  • Bo Pang, Erik Nijkamp, Tian Han, Ying Nian Wu
  • TLDR: We propose a new method for inference of latent variables for text models that is based on a short run of Langevin dynamics and show that it is able to generate coherent sentences with smooth transition and demonstrate improved classification over strong baselines with latent features from unsupervised pretraining.
• Detecting Extraneous Content in Podcasts
  • Sravana Reddy, Yongze Yu, Aasish Pappu, Aswin Sivaraman, Rezvaneh Rezapour, Rosie Jones
  • TLDR: We present classifiers that leverage both textual and listening patterns in order to detect extraneous content in podcast descriptions and audio transcripts.
• Randomized Deep Structured Prediction for Discourse-Level Processing
  • Manuel Widmoser, Maria Leonor Pacheco, Jean Honorio, Dan Goldwasser
  • TLDR: We show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.
• Automatic Data Acquisition for Event Coreference Resolution
  • Prafulla Kumar Choubey, Ruihong Huang
  • TLDR: We propose to leverage lexical paraphrases and high precision rules informed by news discourse structure to automatically collect coreferential and non-coreferential event pairs from unlabeled English news articles.
• Joint Learning of Representations for Web-tables, Entities and Types using Graph Convolutional Network
  • Aniket Pramanick, Indrajit Bhattacharya
  • TLDR: Graph Convolutional Network for table annotation with entities and types.
• Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
  • Wanrong Zhu, Xin Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang
  • TLDR: We introduce a Multimodal Text Style Transfer (MTST) learning approach and leverage external multimodal resources to mitigate data scarcity in outdoor navigation tasks.
• ECOL-R: Encouraging Copying in Novel Object Captioning with Reinforcement Learning
  • Yufei Wang, Ian Wood, Stephen Wan, Mark Johnson
  • TLDR: We propose a novel object captioning model that is encouraged to describe novel objects in the input images and a novel novel object label selector that helps to improve the caption quality.
• Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation
  • Ye Liu, Yao Wan, Jianguo Zhang, Wenting Zhao, Philip Yu
  • TLDR: We propose to incorporate syntactic and semantic structures of natural language into a non-autoregressive Transformer, for the task of neural machine translation.
• NLQuAD: A Non-Factoid Long Question Answering Data Set
  • Amir Soleimani, Christof Monz, Marcel Worring
  • TLDR: We introduce NLQuAD, the first data set with baseline methods for non-factoid long question answering, a task requiring document-level language understanding.
• Debiasing Pre-trained Contextualised Embeddings
  • Masahiro Kaneko, Danushka Bollegala
  • TLDR: We propose a fine-tuning method that can be applied at token- or sentence-levels to debias pre-trained contextualised embeddings.
• Language Models for Lexical Inference in Context
  • Martin Schmitt, Hinrich Schütze
  • TLDR: We present three approaches based on pretrained language models for lexical inference in context and show the potential of these models for this task.
• Few-Shot Semantic Parsing for New Predicates
  • Zhuang Li, Lizhen Qu, Shuo Huang, Gholamreza Haffari
  • TLDR: We propose a new meta-learning method for semantic parsing in few-shot learning setting.
• Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models
  • Qingyang Wu, Yichi Zhang, Yu Li, Zhou Yu
  • TLDR: We propose a simple, general, and effective framework for dialog system models that can generalize to more challenging, non-collaborative tasks such as persuasion.
• On the Evaluation of Vision-and-Language Navigation Instructions
  • Ming Zhao, Peter Anderson, Vihan Jain, Su Wang, Alexander Ku, Jason Baldridge, Eugene Ie
  • TLDR: We present an instruction-trajectory compatibility model that operates without reference instructions and show that it correlates well with human wayfinding outcomes.
• Cross-lingual Visual Pre-training for Multimodal Machine Translation
  • Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Madhyastha, Erkut Erdem, Aykut Erdem, Lucia Specia
  • TLDR: We combine the translation language modelling and visual pre-training methods to learn visually-grounded cross-lingual representations for multimodal machine translation.
• Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation
  • Aparna Elangovan, Jiayuan He, Karin Verspoor
  • TLDR: We identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction, and study the impact of that leakage on the model’s ability to memorize versus generalize.
• An Expert Annotated Dataset for the Detection of Online Misogyny
  • Ella Guest, Bertie Vidgen, Alexandros Mittos, Nishanth Sastry, Gareth Tyson, Helen Margetts
  • TLDR: We present a hierarchical taxonomy for online misogyny, as well as an expert labelled dataset to enable automatic classification of misogynistic content.
• WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia
  • Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, Francisco Guzmán
  • TLDR: We present an approach based on multilingual sentence embeddings to automatically extract parallel sentences from the content of Wikipedia articles in 96 languages, including several dialects or low-resource languages.
• ChEMU-Ref: A Corpus for Modeling Anaphora Resolution in the Chemical Domain
  • Biaoyan Fang, Christian Druckenbrodt, Saber A Akhondi, Jiayuan He, Timothy Baldwin, Karin Verspoor
  • TLDR: We propose a novel annotation scheme for chemical patents, based on which we create the ChEMU-Ref dataset from reaction description snippets in English-language chemical patents.
• Syntactic Nuclei in Dependency Parsing – A Multilingual Exploration
  • Ali Basirat, Joakim Nivre
  • TLDR: We show that composition functions for transition-based dependency parsing can improve parsing accuracy and improve the quality of syntactic dependency parsing.
• Searching for Search Errors in Neural Morphological Inflection
  • Martina Forster, Clara Meister, Ryan Cotterell
  • TLDR: We show that the empty string is almost never the most probable solution under neural sequence-to-sequence models for morphological inflection.
• Quantifying Appropriateness of Summarization Data for Curriculum Learning
  • Ryuji Kano, Takumi Takahashi, Toru Nishino, Motoki Taniguchi, Tomoki Taniguchi, Tomoko Ohkuma
  • TLDR: We propose a method of curriculum learning to train summarization models from noisy data.
• Evaluating language models for the retrieval and categorization of lexical collocations
  • Luis Espinosa Anke, Joan Codina-Filba, Leo Wanner
  • TLDR: We perform an exhaustive analysis of current language models for lexical collocation understanding and show that most models fail to distinguish, first, different syntactic structures within the same semantic category, and second, fine-grained semantic categories which restrict the use of small sets of valid collocates for a given base.
• BART-TL: Weakly-Supervised Topic Label Generation
  • Cristian Popa, Traian Rebedea
  • TLDR: We propose a novel solution for assigning labels to topic models by using multiple weak labelers.
• Dynamic Graph Transformer for Implicit Tag Recognition
  • Yi-Ting Liou, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen
  • TLDR: We present a novel dynamic graph transformer for implicit tag recognition and relation extraction.
• Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learning
  • Evgeny Lagutin, Daniil Gavrilov, Pavel Kalaidin
  • TLDR: We propose a new language model optimization method that improves generation output by minimizing repetition in generated text.
• Civil Rephrases Of Toxic Texts With Self-Supervised Transformers
  • Léo Laugier, John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon
  • TLDR: We present a novel text style transfer system that can help to suggest rephrasings of toxic comments in a more civil manner.
• Generating Weather Comments from Meteorological Simulations
  • Soichiro Murakami, Sora Tanaka, Masatsugu Hangyo, Hidetaka Kamigaito, Kotaro Funakoshi, Hiroya Takamura, Manabu Okumura
  • TLDR: We propose a data-to-text model for generating weather-forecast comments from meteorological simulations that is based on prediction data, observation data, and meta-data.
• SICK-NL: A Dataset for Dutch Natural Language Inference
  • Gijs Wijnholds, Michael Moortgat
  • TLDR: We present SICK-NL (read: signal), a dataset targeting Natural Language Inference in Dutch.
• A phonetic model of non-native spoken word processing
  • Yevgen Matusevych, Herman Kamper, Thomas Schatz, Naomi Feldman, Sharon Goldwater
  • TLDR: We show that phonetic perception of the phonetic representation of words in the lexical memory may not be necessary to explain some of the word processing effects observed in non-native speakers.
• Bootstrapping Relation Extractors using Syntactic Search by Examples
  • Matan Eyal, Asaf Amrami, Hillel Taub-Tabib, Yoav Goldberg
  • TLDR: We propose a method for bootstrapping neural-networks based on search-based relation extraction using syntactic-graphs and a by-example syntax.
• Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR
  • Juri Opitz, Anette Frank
  • TLDR: We propose a new metric for evaluating abstract meaning representations that can be used to evaluate text generation from abstract meaning representation.
• The Source-Target Domain Mismatch Problem in Machine Translation
  • Jiajun Shen, Peng-Jen Chen, Matthew Le, Junxian He, Jiatao Gu, Myle Ott, Michael Auli, Marc’Aurelio Ranzato
  • TLDR: We study the effect of local context in machine translation and postulate that this causes the domains of the source and target language to greatly mismatch.
• Cross-Topic Rumor Detection using Topic-Mixtures
  • Xiaoying Ren, Jing Jiang, Ling Min Serena Khoo, Hai Leong Chieu
  • TLDR: We propose a method for rumor detection using deep learning models that uses the mixture of experts to predict rumor class labels.
• Understanding Pre-Editing for Black-Box Neural Machine Translation
  • Rei Miyata, Atsushi Fujita
  • TLDR: We investigate human pre-editing of black-box neural MT and show that it is more important to enhance the explicitness of the meaning of an ST and its syntactic structure than making the ST shorter and simpler.
• RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding
  • Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi
  • TLDR: We propose a theoretical analysis of Knowledge Graph Embeddings and a learning objective for knowledge graph embedding.
• Few-shot Learning for Slot Tagging with Attentive Relational Network
  • Cennet Oguz, Ngoc Thang Vu
  • TLDR: We propose a novel metric-based learning architecture for slot tagging and show that it outperforms other state of the art metric- based learning methods.
• SpanEmo: Casting Multi-label Emotion Classification as Span-prediction
  • Hassan Alhuzali, Sophia Ananiadou
  • TLDR: SpanEmo is a novel multi-label emotion classification model that can help ER models to learn meaningful associations between labels and words in a sentence.
• Exploiting Position and Contextual Word Embeddings for Keyphrase Extraction from Scientific Papers
  • Krutarth Patel, Cornelia Caragea
  • TLDR: We present a graph-based algorithm for keyphrase extraction that exploits both positional information and contextual word embeddings into a biased PageRank.
• Benchmarking Machine Reading Comprehension: A Psychological Perspective
  • Saku Sugawara, Pontus Stenetorp, Akiko Aizawa
  • TLDR: We provide theoretical basis for the design of machine reading comprehension datasets based on psychology and psychometrics and provide a theoretical framework for benchmarking MRC.
• Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders
  • Xiang Kong, Adithya Renduchintala, James Cross, Yuqing Tang, Jiatao Gu, Xian Li
  • TLDR: We propose a deep encoder with multiple shallow decoders for multilingual translation that can improve translation quality without sacrificing speed.
• With Measured Words: Simple Sentence Selection for Black-Box Optimization of Sentence Compression Algorithms
  • Yotam Shichel, Meir Kalech, Oren Tsur
  • TLDR: We propose a black-box optimizer for sentence compression that learns to find the best candidates for compression in order to maximize both compression rate and quality.
• WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context
  • Anna Breit, Artem Revenko, Kiamehr Rezaee, Mohammad Taher Pilehvar, Jose Camacho-Collados
  • TLDR: We present WiC-TSV, a new multi-domain evaluation benchmark for Word Sense Disambiguation.
• Self-Supervised and Controlled Multi-Document Opinion Summarization
  • Hady Elsahar, Maximin Coavoux, Jos Rozen, Matthias Gallé
  • TLDR: We address the problem of unsupervised abstractive summarization of collections of user generated reviews through self-supervision and control.
• NewsMTSC: A Dataset for (Multi-)Target-dependent Sentiment Classification in Political News Articles
  • Felix Hamborg, Karsten Donnay
  • TLDR: We propose a new dataset for target-dependent sentiment classification in news articles and a novel model for multi-target sentences.
• Cross-lingual Contextualized Topic Models with Zero-shot Learning
  • Federico Bianchi, Silvia Terragni, Dirk Hovy, Debora Nozza, Elisabetta Fersini
  • TLDR: We introduce a zero-shot cross-lingual topic model that learns topics on one language and predicts them for unseen documents in different languages.
• Dependency parsing with structure preserving embeddings
  • Ákos Kádár, Lan Xiao, Mete Kemertas, Federico Fancellu, Allan Jepson, Afsaneh Fazly
  • TLDR: We propose a new approach to parse dependency trees by embedding tree embedding losses within a graph-based parser.
• Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates
  • Artem Shelmanov, Dmitri Puzyrev, Lyubov Kupriyanova, Denis Belyakov, Daniil Larionov, Nikita Khromov, Olga Kozlova, Ekaterina Artemova, Dmitry V. Dylov, Alexander Panchenko
  • TLDR: We show that transfer learning and active learning can be used to reduce annotation budget for sequence tagging tasks.
• MultiHumES: Multilingual Humanitarian Dataset for Extractive Summarization
  • Jenny Paola Yela-Bello, Ewan Oglethorpe, Navid Rekabsaz
  • TLDR: We present a new multilingual extractive summarization model for humanitarian response data and provide a new data collection for this purpose.
• Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale
  • Gabriella Skitalinskaya, Jonas Klaff, Henning Wachsmuth
  • TLDR: We study claim quality assessment irrespective of discussed aspects by comparing different revisions of the same claim.
• Few Shot Dialogue State Tracking using Meta-learning
  • Saket Dingliwal, Shuyang Gao, Sanchit Agarwal, Chien-Wei Lin, Tagyoung Chung, Dilek Hakkani-Tur
  • TLDR: Meta-Learning for dialogue state tracking.
• BERT Prescriptions to Avoid Unwanted Headaches: A Comparison of Transformer Architectures for Adverse Drug Event Detection
  • Beatrice Portelli, Edoardo Lenzi, Emmanuele Chersoni, Giuseppe Serra, Enrico Santus
  • TLDR: We present a systematic evaluation of transformer-based models in biomedical text classification and show that span-based pretraining gives a decisive advantage in the precise recognition of ADEs.
• Semantic Parsing of Disfluent Speech
  • Priyanka Sen, Isabel Groves
  • TLDR: We investigate semantic parsing of disfluent speech with the ATIS dataset.
• Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models
  • Tianxing He, Bryan McCann, Caiming Xiong, Ehsan Hosseini-Asl
  • TLDR: Joint energy-based model training during finetuning of pretrained text encoders for NLU tasks.
• What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology
  • Qinlan Shen, Carolyn Rose
  • TLDR: We present empirical evidence that the role of experiential factors in how humans perceive and interpret political ideology is not inherently built into text.
• Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
  • Benjamin Heinzerling, Kentaro Inui
  • TLDR: We present a proof-of-concept that language models can indeed serve as knowledge bases.
• Globalizing BERT-based Transformer Architectures for Long Document Summarization
  • Quentin Grail, Julien Perez, Eric Gaussier
  • TLDR: We propose a hierarchical propagation layer for efficient language model summarization and show state-of-the-art results for long document summarization.
• Through the Looking Glass: Learning to Attribute Synthetic Text Generated by Language Models
  • Shaoor Munir, Brishna Batool, Zubair Shafiq, Padmini Srinivasan, Fareed Zaffar
  • TLDR: We propose and test several methods for attributing authorship of synthetic text generated by language models.
• We Need To Talk About Random Splits
  • Anders Søgaard, Sebastian Ebert, Jasmijn Bastings, Katja Filippova
  • TLDR: We argue that random splits, like standard splits, lead to overly optimistic performance estimates.
• How Certain is Your Transformer?
  • Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanin, Alexander Panchenko, Maxim Panov
  • TLDR: We investigate the applicability of uncertainty estimates based on dropout usage at the inference stage (Monte Carlo dropout) for Transformer-based models.
• Alignment verification to improve NMT translation towards highly inflectional languages with limited resources
  • George Tambouratzis
  • TLDR: We propose a novel method for improving translation quality using neural neural networks trained with different metrics.
• Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase
  • Akhila Yerukola, Mason Bretan, Hongxia Jin
  • TLDR: We introduce a data augmentation technique based on byte pair encoding and a BERT-like self-attention model to boost performance on spoken language understanding tasks.
• How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation
  • Julius Steen, Katja Markert
  • TLDR: We show that the total number of annotators can have a strong impact on study power and that current statistical analysis methods can inflate type I error rates up to eight-fold.
• Open-Mindedness and Style Coordination in Argumentative Discussions
  • Aviv Ben-Haim, Oren Tsur
  • TLDR: We show that open-mindedness correlates with linguistic accommodation, rather than social status, in Reddit’s Change My View subreddit.
• Error Analysis and the Role of Morphology
  • Marcel Bollmann, Anders Søgaard
  • TLDR: We evaluate two common conjectures in error analysis of NLP models: (i) Morphology is predictive of errors; and (ii) the importance of morphology increases with the morphological complexity of a language.
• Applying the Transformer to Character-level Transduction
  • Shijie Wu, Ryan Cotterell, Mans Hulden
  • TLDR: We show that the transformer outperforms recurrent neural network-based sequence-to-sequence models on character-level NLP tasks, and we show that with a large enough batch size, the transformer does indeed outperform recurrent models.
• Exploring Supervised and Unsupervised Rewards in Machine Translation
  • Julia Ive, Zixu Wang, Marina Fomicheva, Lucia Specia
  • TLDR: We propose to address the mismatch between the mismatch in reward function and evaluation metrics in neural Machine Translation by exploring a dynamic unsupervised reward function.
• Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions
  • Pere-Lluís Huguet Cabot, David Abadi, Agneta Fischer, Ekaterina Shutova
  • TLDR: We present the first large-scale computational models of populist attitudes and the first empirical datasets for this phenomenon.
• Multilingual Entity and Relation Extraction Dataset and Model
  • Alessandro Seganti, Klaudia Firląg, Helena Skowronska, Michał Satława, Piotr Andruszkiewicz
  • TLDR: We present a novel dataset and model for a multilingual setting to approach the task of Joint Entity and Relation Extraction.
• A New View of Multi-modal Language Analysis: Audio and Video Features as Text “Styles”
  • Zhongkai Sun, Prathusha K Sarma, Yingyu Liang, William Sethares
  • TLDR: We propose a style-transferred language semantics model that learns richer representations for multi-modal utterances using style-Transferred multi-Modal features.
• Multilingual and cross-lingual document classification: A meta-learning approach
  • Niels van der Heijden, Helen Yannakoudakis, Pushkar Mishra, Ekaterina Shutova
  • TLDR: Meta-learning for document classification in low-resource languages.
• Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies
  • Gabriele Pergola, Elena Kochkina, Lin Gui, Maria Liakata, Yulan He
  • TLDR: We propose biomedical entity-aware masking (BEM) strategy, encouraging masked language models to learn entity-centric knowledge based on the pivotal entities characterizing the domain at hand, and employ those entities to drive the LM fine-tuning.
• Attention-based Relational Graph Convolutional Network for Target-Oriented Opinion Words Extraction
  • Junfeng Jiang, An Wang, Akiko Aizawa
  • TLDR: We propose a novel attention-based relational graph convolutional neural network (ARGCN) to exploit syntactic information over dependency graphs.
• “Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space
  • Debanjan Ghosh, Ritvik Shrivastava, Smaranda Muresan
  • TLDR: We present a thorough experimental setup using a corpus annotated with both argumentative moves (agree/disagree) and sarcasm.
• Learning Relatedness between Types with Prototypes for Relation Extraction
  • Lisheng Fu, Ralph Grishman
  • TLDR: We propose to use prototypical examples to represent each relation type and use these examples to augment related types from a different dataset.
• I Beg to Differ: A study of constructive disagreement in online conversations
  • Christine De Kock, Andreas Vlachos
  • TLDR: We develop a variety of neural models for WikiDisputes and show that taking into account the structure of the conversation improves predictive accuracy, exceeding that of feature-based models.
• Acquiring a Formality-Informed Lexical Resource for Style Analysis
  • Elisabeth Eder, Ulrike Krieg-Holz, Udo Hahn
  • TLDR: We introduce a novel lexicon for the German language, with entries ordered by their degree of (in)formality.
• Probing into the Root: A Dataset for Reason Extraction of Structural Events from Financial Documents
  • Pei Chen, Kang Liu, Yubo Chen, Taifeng Wang, Jun Zhao
  • TLDR: We propose a new causality detection task for document-level texts, which provides structural event descriptions and reason spans for Financial events in company announcements.
• Language Modelling as a Multi-Task Problem
  • Lucas Weber, Jaap Jumelet, Elia Bruni, Dieuwke Hupkes
  • TLDR: We propose to study language modelling as a multi-task problem, bringing together three strands of research: multi-Task learning, linguistics, and interpretability.
• ChainCQG: Flow-Aware Conversational Question Generation
  • Jing Gu, Mostafa Mirshekari, Zhou Yu, Aaron Sisto
  • TLDR: We present a novel approach to question-answering in conversational systems that learns question-answer representations across multiple dialogue turns using a flow propagation training strategy.
• The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues
  • Alberto Testoni, Raffaella Bernardi
  • TLDR: We show that language quality could improve the accuracy of referential dialogue guessing games, but not the language quality of the model.
• “Are you kidding me?”: Detecting Unpalatable Questions on Reddit
  • Sunyam Bagga, Andrew Piper, Derek Ruths
  • TLDR: We propose a novel task of detecting unpalatable questions in online discourse and show that it is feasible to model subtle forms of abuse.
• Neural-Driven Search-Based Paraphrase Generation
  • Betty Fabre, Tanguy Urvoy, Jonathan Chevelu, Damien Lolive
  • TLDR: We study a search-based paraphrase generation scheme where candidate paraphrases are generated by iterated transformations from the original sentence and evaluated in terms of syntax quality, semantic distance, and lexical distance.
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora
  • Zi-Yi Dou, Graham Neubig
  • TLDR: We propose methods to marry pre-trained contextualized word embeddings with pre-tuned language models for word alignment over parallel corpora.
• Paraphrases do not explain word analogies
  • Louis Fournier, Ewan Dunbar
  • TLDR: We show that word embeddings encode linguistic regularities as directions, not as paraphrase relations.
• An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games
  • Alessandro Suglia, Yonatan Bisk, Ioannis Konstas, Antonio Vergari, Emanuele Bastianelli, Andrea Vanzo, Oliver Lemon
  • TLDR: We propose two ways to exploit playing guessing games to improve performance on visual question answering tasks.
• A Unified Feature Representation for Lexical Connotations
  • Emily Allaway, Kathleen McKeown
  • TLDR: We present a lexical resource for capturing connotations of nouns and adjectives and show that it aligns well with human judgments.
• FAST: Financial News and Tweet Based Time Aware Network for Stock Trading
  • Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah
  • TLDR: We propose a novel hierarchical, learning to rank approach that uses textual data to make time-aware predictions for ranking stocks based on expected profit.
• Building Representative Corpora from Illiterate Communities: A Reviewof Challenges and Mitigation Strategies for Developing Countries
  • Stephanie Hirmer, Alycia Leonard, Josephine Tumwesige, Costanza Conforti
  • TLDR: We address the under-representation of illiterate communities in NLP corpora and propose a set of practical mitigation strategies to help future work.
• Process-Level Representation of Scientific Protocols with Interactive Annotation
  • Ronen Tamari, Fan Bai, Alan Ritter, Gabriel Stanovsky
  • TLDR: We develop Process Execution Graphs (PEG), a document-level representation of real-world wet lab biochemistry protocols, addressing challenges such as cross-sentence relations, long-range coreference, grounding, and implicit arguments.
• Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation
  • Eva Vanmassenhove, Dimitar Shterionov, Matthew Gwilliam
  • TLDR: We propose that existing machine translation models amplify biases observed in the training data and this can lead to an artificially impoverished language.
• First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
  • Benjamin Muller, Yanai Elazar, Benoît Sagot, Djamé Seddah
  • TLDR: We show that multilingual BERT is a stacking of two sub-networks: a multilingual encoder followed by a task-specific language-agnostic predictor.
• Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models
  • Daniel de Vassimon Manela, David Errington, Thomas Fisher, Boris van Breugel, Pasquale Minervini
  • TLDR: We propose two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task.
• On the evolution of syntactic information encoded by BERT’s contextualized representations
  • Laura Pérez-Mayos, Roberto Carlini, Miguel Ballesteros, Leo Wanner
  • TLDR: We analyze the evolution of the embedded syntax trees along the fine-tuning process of BERT for six different tasks, covering all levels of the linguistic structure.
• Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Tasks
  • Lisa Bauer, Mohit Bansal
  • TLDR: We present an approach to assess how well a candidate KG can correctly identify and accurately fill in gaps of reasoning for a task, which we call KG-to-task match.
• Calculating the optimal step of arc-eager parsing for non-projective trees
  • Mark-Jan Nederhof
  • TLDR: We show that the optimal next step of an arc-eager parser relative to a non-projective dependency structure can be calculated in cubic time, solving an open problem in parsing theory.
• Subword Pooling Makes a Difference
  • Judit Ács, Ákos Kádár, Andras Kornai
  • TLDR: We show that the choice of subword pooling affects downstream performance on three tasks: morphological probing, POS tagging and NER, in 9 typologically diverse languages.
• Content-based Models of Quotation
  • Ansel MacLaughlin, David Smith
  • TLDR: We explore the task of quotability identification, in which, given a document, we aim to identify which of its passages are the most quotable, i.e. the most likely to be directly quoted by later derived documents.
• L2C: Describing Visual Differences Needs Semantic Understanding of Individuals
  • An Yan, Xin Wang, Tsu-Jui Fu, William Yang Wang
  • TLDR: We introduce a new model for captioning images that learns to compare them and compare them while learning to describe each one.
• VoiSeR: A New Benchmark for Voice-Based Search Refinement
  • Simone Filice, Giuseppe Castellucci, Marcus Collins, Eugene Agichtein, Oleg Rokhlenko
  • TLDR: We present a large-scale dataset of voice-based search refinements, VoiSeR, consisting of about 10,000 search refinement utterances, collected using a novel crowdsourcing task.
• Event-Driven News Stream Clustering using Entity-Aware Contextual Embeddings
  • Kailash Karthik Saravanakumar, Miguel Ballesteros, Muthu Kumar Chandrasekaran, Kathleen McKeown
  • TLDR: We propose a novel method for online news stream clustering that is a variant of the non-parametric streaming K-means algorithm.
• Adversarial Learning of Poisson Factorisation Model for Gauging Brand Sentiment in User Reviews
  • Runcong Zhao, Lin Gui, Gabriele Pergola, Yulan He
  • TLDR: We propose a new model for sentiment-topic extraction based on adversarial learning and show that it outperforms existing models in terms of topic coherence and unique-ness.
• Lexical Normalization for Code-switched Data and its Effect on POS Tagging
  • Rob van der Goot, Özlem Çetinoğlu
  • TLDR: We propose three different normalization layers for code-switched data and show that they significantly improve the performance of social media normalization systems.
• Structural Encoding and Pre-training Matter: Adapting BERT for Table-Based Fact Verification
  • Rui Dong, David Smith
  • TLDR: We propose a novel method for verifying textual statements given evidence in tables.
• A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
  • Miruna-Adriana Clinciu, Arash Eshghi, Helen Hastie
  • TLDR: We investigate which of the NLG evaluation measures map well to explanations.
• Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling
  • Chris Emmery, Ákos Kádár, Grzegorz Chrupała
  • TLDR: We propose a new adversarial stylometry attack that can be used to attack language models by rewriting an author’s text.
• Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks
  • Jimin Sun, Hwijeen Ahn, Chan Young Park, Yulia Tsvetkov, David R. Mortensen
  • TLDR: We propose three linguistic features that capture cross-cultural similarities that manifest in linguistic patterns and quantify distinct aspects of language pragmatics.
• PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media
  • Ramit Sawhney, Harshit Joshi, Lucie Flek, Rajiv Ratn Shah
  • TLDR: We propose PHASE, a time-and phase-aware framework that adaptively learns features from a user’s historical emotional spectrum on Twitter for preliminary screening of suicidal risk.
• Exploiting Definitions for Frame Identification
  • Tianyu Jiang, Ellen Riloff
  • TLDR: We present a new model for frame identification that uses a pre-trained transformer model to generate representations for frames and lexical units (senses) using their formal definitions in FrameNet.
• ADePT: Auto-encoder based Differentially Private Text Transformation
  • Satyapriya Krishna, Rahul Gupta, Christophe Dupuy
  • TLDR: We propose a differentially private text transformation algorithm that offers robustness against membership inference attacks and produces transformations with high semantic quality that perform well on downstream NLP tasks.
• Conceptual Grounding Constraints for Truly Robust Biomedical Name Representations
  • Pieter Fivez, Simon Suster, Walter Daelemans
  • TLDR: We propose a novel neural neural encoding architecture for biomedical names that captures both lexical and semantic relatedness of biomedical names.
• Adaptive Mixed Component LDA for Low Resource Topic Modeling
  • Suzanna Sia, Kevin Duh
  • TLDR: We propose a mixture model for probabilistic topic models that interpolates between discrete and continuous topic-word distributions that utilise pre-trained embeddings to improve topic coherence.
• Evaluating Neural Model Robustness for Machine Comprehension
  • Winston Wu, Dustin Arendt, Svitlana Volkova
  • TLDR: We evaluate neural model robustness to adversarial attacks using different types of linguistic unit perturbations and propose a new method for strategic sentence-level perturbation.
• Hidden Biases in Unreliable News Detection Datasets
  • Xiang Zhou, Heba Elfardy, Christos Christodoulopoulos, Thomas Butler, Mohit Bansal
  • TLDR: We show that selection bias during data collection leads to undesired artifacts in the datasets.
• Annealing Knowledge Distillation
  • Aref Jafari, Mehdi Rezagholizadeh, Pranav Sharma, Ali Ghodsi
  • TLDR: We propose an improved knowledge distillation method (called Annealing-KD) by feeding the rich information provided by teacher’s soft-targets incrementally and more efficiently.
• Unsupervised Extractive Summarization using Pointwise Mutual Information
  • Vishakh Padmakumar, He He
  • TLDR: We propose new metrics of relevance and redundancy using pointwise mutual information between sentences, which can be easily computed by a pre-trained language model.
• Context-aware Neural Machine Translation with Mini-batch Embedding
  • Makoto Morishita, Jun Suzuki, Tomoharu Iwata, Masaaki Nagata
  • TLDR: We propose mini-batch embedding (MBE) as a way to represent the features of sentences in a mini-batch.
• Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT
  • Isabel Papadimitriou, Ethan A. Chi, Richard Futrell, Kyle Mahowald
  • TLDR: We investigate how Multilingual BERT (mBERT) encodes grammar by examining how the high-order grammatical feature of morphosyntactic alignment (how different languages define what counts as a “subject”) is manifested across the embedding spaces of different languages.
• Streaming Models for Joint Speech Recognition and Translation
  • Orion Weller, Matthias Sperber, Christian Gollan, Joris Kluivers
  • TLDR: We present an end-to-end streaming ST model based on a re-translation approach and compare against standard cascading approaches.
• DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections
  • Yury Zemlyanskiy, Sudeep Gandhe, Ruining He, Bhargav Kanagal, Anirudh Ravula, Juraj Gottweis, Fei Sha, Ilya Eckstein
  • TLDR: We present several training strategies that learn to jointly predict words and entities from large text corpora, and show that these models can outperform competitive baselines on downstream tasks in the TV-Movies domain.
• Scientific Discourse Tagging for Evidence Extraction
  • Xiangci Li, Gully Burns, Nanyun Peng
  • TLDR: We present the capability of automatically extracting text fragments from primary research papers that describe the evidence presented in that paper’s figures, which arguably provides the raw material of any scientific argument made within the paper.
• Incremental Beam Manipulation for Natural Language Generation
  • James Hargreaves, Andreas Vlachos, Guy Emerson
  • TLDR: We propose incremental beam manipulation, a method for optimizing the output of neural network beam search by reranking the hypotheses in the beam during decoding instead of only at the end.
• StructSum: Summarization via Structured Representations
  • Vidhisha Balachandran, Artidoro Pagnoni, Jay Yoon Lee, Dheeraj Rajagopal, Jaime Carbonell, Yulia Tsvetkov
  • TLDR: We propose a novel framework based on document-level structure induction for abstractive text summarization that improves the coverage of content in the source documents, generates more abstractive summaries by generating more novel n-grams, and incorporates interpretable sentence-level structures, while performing on par with standard baselines.
• Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing
  • Hiroaki Ozaki, Gaku Morio, Terufumi Morishita, Toshinori Miyoshi
  • TLDR: We propose a graph-based semantic dependency parser that can transfer cross-linguals between pre-trained language models and pre-learned language models.
• LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction
  • Jacob Solawetz, Stefan Larson
  • TLDR: We present a large-scale OIE dataset for natural language processing and provide benchmarks for future improvements.
• Changing the Mind of Transformers for Topically-Controllable Language Generation
  • Haw-Shiuan Chang, Jiaming Yuan, Mohit Iyyer, Andrew McCallum
  • TLDR: We present a novel framework for generating text that learns to generate fluent sentences related to a set of candidate topics.
• Unsupervised Abstractive Summarization of Bengali Text Documents
  • Radia Rayan Chowdhury, Mir Tafseer Nayeem, Tahsin Tasnim Mim, Md. Saifur Rahman Chowdhury, Taufiqul Jannat
  • TLDR: Graph-based unsupervised abstractive summarization system for Bengali text documents.
• From Toxicity in Online Comments to Incivility in American News: Proceed with Caution
  • Anushree Hede, Oshin Agarwal, Linda Lu, Diana C. Mutz, Ani Nenkova
  • TLDR: We show that toxicity models, as exemplified by Perspective, are inadequate for the analysis of incivility in news.
• On the Computational Modelling of Michif Verbal Morphology
  • Fineen Davis, Eddie Antonio Santos, Heather Souter
  • TLDR: This paper presents a finite-state computational model of the verbal morphology of Michif.
• A Few Topical Tweets are Enough for Effective User Stance Detection
  • Younes Samih, Kareem Darwish
  • TLDR: We improve user-level stance detection on vocal Twitter users by representing tweets using contextualized embeddings, which capture latent meanings of words in context, and then we perform unsupervised classification of the user, which entails clustering a user with other users in the training set.
• Do Syntax Trees Help Pre-trained Transformers Extract Information?
  • Devendra Sachan, Yuhao Zhang, Peng Qi, William L. Hamilton
  • TLDR: We propose and investigate syntax-augmented transformers that obtain state-of-the-art results on SRL and relation extraction tasks.
• Informative and Controllable Opinion Summarization
  • Reinald Kim Amplayo, Mirella Lapata
  • TLDR: We propose a novel approach to summarization of opinions which uses all input reviews and allows to take user preferences into account when customizing summaries.
• Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings
  • Katharina Kann, Mauro M. Monsalve-Mercado
  • TLDR: We study the similarity of English character embeddings to synesthetes and show that LSTMs are more similar to humans than transformers.
• How Good (really) are Grammatical Error Correction Systems?
  • Alla Rozovskaya, Dan Roth
  • TLDR: We show that GEC systems are 20-40 points better than standard evaluations, even when considering any of the top-10 hypotheses produced by a system.
• BERTective: Language Models and Contextual Information for Deception Detection
  • Tommaso Fornaciari, Federico Bianchi, Massimo Poesio, Dirk Hovy
  • TLDR: We present a new state-of-the-art neural model for detecting deception in dialogues containing deceptive statements and show that not all context is equally useful to the task.
• Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning
  • Philip Arthur, Trevor Cohn, Gholamreza Haffari
  • TLDR: We present a novel approach to efficiently learn a simultaneous translation model with coupled programmer-interpreter policies.
• Complementary Evidence Identification in Open-Domain Question Answering
  • Xiangyang Mou, Mo Yu, Shiyu Chang, Yufei Feng, Li Zhang, Hui Su
  • TLDR: We propose a method that learns vector representations of passages and models the sufficiency and diversity within the selected set, in addition to the relevance between the question and passages.
• Entity-level Factual Consistency of Abstractive Text Summarization
  • Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, Bing Xiang
  • TLDR: We propose a set of new metrics to quantify the entity-level factual consistency of generated summaries and show that the entity hallucination problem can be alleviated by simply filtering the training data.
• On Hallucination and Predictive Uncertainty in Conditional Language Generation
  • Yijun Xiao, William Yang Wang
  • TLDR: We show that hallucinations are more likely to occur in neural models that are more uncertain than the predictions.
• Fine-Grained Event Trigger Detection
  • Duong Le, Thien Huu Nguyen
  • TLDR: We present the first study on fine-grained ED (FED) where the evaluation dataset involves much more fine-gained event types (i.e., 449 types).
• Extremely Small BERT Models from Mixed-Vocabulary Training
  • Sanqiang Zhao, Raghav Gupta, Yang Song, Denny Zhou
  • TLDR: We propose a distillation method to align the teacher and student embeddings via mixed-vocabulary training to compress BERT-LARGE to a task-agnostic model with smaller vocabulary and hidden dimensions.
• Diverse Adversaries for Mitigating Bias in Training
  • Xudong Han, Timothy Baldwin, Trevor Cohn
  • TLDR: We propose a novel approach to adversarial learning based on the use of multiple diverse discriminators, whereby discriminators are encouraged to learn orthogonal hidden representations from one another.
• ‘Just because you are right, doesn’t mean I am wrong’: Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks
  • Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral
  • TLDR: We propose Alternative Answer Sets (AAS) of ground-truth answers to address this limitation, which is created automatically using off-the-shelf NLP tools.
• Better Neural Machine Translation by Extracting Linguistic Information from BERT
  • Hassan S. Shavarani, Anoop Sarkar
  • TLDR: We augment neural machine translation with dense fine-tuned vector-based linguistic information from BERT instead of using point estimates.
• CLiMP: A Benchmark for Chinese Language Model Evaluation
  • Beilei Xiang, Changbing Yang, Yu Li, Alex Warstadt, Katharina Kann
  • TLDR: We present a corpus of Chinese linguistic minimal pairs for syntactic contrasts and show that models generally perform better on the syntactic dependencies than the models.
• Measuring and Improving Faithfulness of Attention in Neural Machine Translation
  • Pooya Moradi, Nishant Kambhatla, Anoop Sarkar
  • TLDR: We propose a new objective for neural machine translation models that rewards faithful behaviour by the model through probability divergence and show that it improves translation quality and improves faithfulness.
• Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering
  • Wenhan Xiong, Hong Wang, William Yang Wang
  • TLDR: We propose a new approach to information retrieval using dense question and paragraph representations for deep semantic matching.
• Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs
  • Dora Jambor, Komal Teru, Joelle Pineau, William L. Hamilton
  • TLDR: We present a systematic study of few-shot link prediction methods and show that they are surprisingly strong in the few-shots setting.
• ProFormer: Towards On-Device LSH Projection Based Transformers
  • Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva
  • TLDR: We present a novel projection based transformer architecture for text based neural models that is faster and lighter than BERT and outperforms BERT on multiple text classification tasks.
• Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification
  • Soumya Chatterjee, Ayush Maheshwari, Ganesh Ramakrishnan, Saketha Nath Jagarlapudi
  • TLDR: We propose a novel formulation for multi-label classification that leverages the prior knowledge of existence of a hierarchy over the labels, and show that the label-embedding obtained by joint learning is more faithful to the label hierarchy.
• Segmenting Subtitles for Correcting ASR Segmentation Errors
  • David Wan, Chris Kedzie, Faisal Ladhak, Elsbeth Turcan, Petra Galuscakova, Elena Zotkina, Zhengping Jiang, Peter Bell, Kathleen McKeown
  • TLDR: We propose a model for correcting the acoustic segmentation of ASR models for low-resource languages to improve performance on downstream tasks.
• Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO
  • Zarana Parekh, Jason Baldridge, Daniel Cer, Austin Waters, Yinfei Yang
  • TLDR: We present Crisscrossed Captions, a new dataset for image captioning that measures the influence of intra- and inter-modality learning.
• On-Device Text Representations Robust To Misspellings via Projections
  • Chinnadhurai Sankar, Sujith Ravi, Zornitsa Kozareva
  • TLDR: We show that the projection based neural classifiers are inherently robust to misspellings and perturbations of the input text.
• ENPAR:Enhancing Entity and Entity Pair Representations for Joint Entity Relation Extraction
  • Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei Li, Junchi Yan
  • TLDR: We propose a novel pre-training method for entity relation extraction that uses the entity pair information to improve the joint extraction performance.
• Text Augmentation in a Multi-Task View
  • Jason Wei, Chengyu Huang, Shiqi Xu, Soroush Vosoughi
  • TLDR: We propose a multi-task view of data augmentation in which both original and augmented examples are weighted substantively during training, relaxing the constraint that augmented examples must resemble original data and thereby allowing us to apply stronger augmentation functions.
• Representations for Question Answering from Documents with Tables and Text
  • Vicky Zayats, Kristina Toutanova, Mari Ostendorf
  • TLDR: We propose a novel method to improve question answering from tables by refining table representations based on information from surrounding text.
• PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation
  • Kemal Kurniawan, Lea Frermann, Philip Schulz, Trevor Cohn
  • TLDR: We present a method for unsupervised cross-lingual transfer that improves over direct transfer systems by using their output as implicit supervision as part of self-training on unlabelled text in the target language.
• Modelling Context Emotions using Multi-task Learning for Emotion Controlled Dialog Generation
  • Deeksha Varshney, Asif Ekbal, Pushpak Bhattacharyya
  • TLDR: We propose a novel multi-task learning based response generation module that generates a viable response to a user’s utterance in an empathetic manner.
• Gender and Racial Fairness in Depression Research using Social Media
  • Carlos Aguirre, Keith Harrigian, Mark Dredze
  • TLDR: We analyze the fairness of depression classifiers trained on Twitter data with respect to gender and racial demographic groups.
• MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark
  • Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, Yashar Mehdad
  • TLDR: We present a new multilingual dataset for task-oriented semantic parsing and benchmarking of state-of-the-art multilingual pre-trained models for task oriented dialog systems.
• Adapting Event Extractors to Medical Data: Bridging the Covariate Shift
  • Aakanksha Naik, Jill Fain Lehman, Carolyn Rose
  • TLDR: We tackle the task of adapting event extractors to new domains without labeled data, by aligning the marginal distributions of source and target domains.
• NoiseQA: Challenge Set Evaluation for User-Centric Question Answering
  • Abhilasha Ravichander, Siddharth Dalmia, Maria Ryskina, Florian Metze, Eduard Hovy, Alan W Black
  • TLDR: We show that components in the pipeline that precede an answering engine can introduce varied and considerable sources of error, and performance can degrade substantially based on these upstream noise sources even for powerful pre-trained QA models.
• Co-evolution of language and agents in referential games
  • Gautier Dagan, Dieuwke Hupkes, Elia Bruni
  • TLDR: We show that the optimal situation is to take into account also the learning biases of the language learners and thus let language and agents co-evolve.
• Modeling Context in Answer Sentence Selection Systems on a Latency Budget
  • Rujun Han, Luca Soldaini, Alessandro Moschitti
  • TLDR: We present a novel approach to incorporate contextual information into answer sentence selection in question-answer answering systems.
• Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees
  • Jiangang Bai, Yujing Wang, Yiren Chen, Yaming Yang, Jing Bai, Jing Yu, Yunhai Tong
  • TLDR: Syntactic trees can be incorporated into pre-trained Transformer models without explicit consideration of syntactic information.
• DISK-CSV: Distilling Interpretable Semantic Knowledge with a Class Semantic Vector
  • Housam Khalifa Bashier, Mi-Young Kim, Randy Goebel
  • TLDR: We propose a new technique (DISK-CSV) to distill knowledge concurrently from any neural network architecture for text classification, captured as a lightweight interpretable/explainable classifier.
• Attention Can Reflect Syntactic Structure (If You Let It)
  • Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou, Anders Søgaard, Joakim Nivre
  • TLDR: We present decoding experiments for multilingual BERT across 18 languages in order to test the generalizability of the claim that dependency syntax is reflected in attention patterns.
• Extractive Summarization Considering Discourse and Coreference Relations based on Heterogeneous Graph
  • Yin Jou Huang, Sadao Kurohashi
  • TLDR: We propose a heterogeneous graph based model for extractive summarization that incorporates both discourse and coreference relations.
• CDA: a Cost Efficient Content-based Multilingual Web Document Aligner
  • Thuy Vu, Alessandro Moschitti
  • TLDR: We introduce a Content-based Document Alignment approach (CDA), an efficient method to align multilingual web documents based on content in creating parallel training data for machine translation (MT) systems operating at the industrial level.
• Metric-Type Identification for Multi-Level Header Numerical Tables in Scientific Papers
  • Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Manabu Okumura, Hiroya Takamura
  • TLDR: We propose a new metric-type identification task for numerical tables and propose a novel neural classification and generation scheme.
• EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction
  • Bhanu Prakash Reddy Guda, Aparna Garimella, Niyati Chhaya
  • TLDR: We propose EmpathBERT, a demographic-aware framework for empathy prediction based on BERT.
• Are Neural Networks Extracting Linguistic Properties or Memorizing Training Data? An Observation with a Multilingual Probe for Predicting Tense
  • Bingzhi Li, Guillaume Wisniewski
  • TLDR: We evaluate the ability of Bert embeddings to represent tense information, taking French and Chinese as a case study.
• Is Supervised Syntactic Parsing Beneficial for Language Understanding Tasks? An Empirical Investigation
  • Goran Glavaš, Ivan Vulić
  • TLDR: We empirically investigate the usefulness of supervised parsing for semantic language understanding in the context of LM-pretrained transformer networks.
• Facilitating Terminology Translation with Target Lemma Annotations
  • Toms Bergmanis, Mārcis Pinnis
  • TLDR: We propose to train machine translation systems using a source-side data augmentation method that annotates randomly selected source language words with their target language lemmas.
• Enhancing Sequence-to-Sequence Neural Lemmatization with External Resources
  • Kirill Milintsevich, Kairit Sirts
  • TLDR: We propose a novel hybrid approach to lemmatization that enhances the seq2seq neural model with additional lemmas extracted from an external lexicon or a rule-based system.
• Summarising Historical Text in Modern Languages
  • Xutan Peng, Yi Zheng, Chenghua Lin, Advaith Siddharthan
  • TLDR: We introduce the task of historical text summarisation, where documents in historical forms of a language are summarised in the corresponding modern language.
• Challenges in Automated Debiasing for Toxic Language Detection
  • Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, Noah Smith
  • TLDR: We propose a novel method for debiasing toxic language classifiers that reduces dialectal associations with toxicity.
• Adaptive Fusion Techniques for Multimodal Data
  • Gaurav Sahu, Olga Vechtomova
  • TLDR: We propose adaptive fusion techniques that aim to model context from different modalities effectively.
• Detecting Scenes in Fiction: A new Segmentation Task
  • Albin Zehe, Leonard Konle, Lea Katharina Dümpelmann, Evelyn Gius, Andreas Hotho, Fotis Jannidis, Lucas Kaufmann, Markus Krug, Frank Puppe, Nils Reiter, Annekea Schreiber, Nathalie Wiedmer
  • TLDR: An annotated corpus for scene segmentation on narrative texts and a baseline analysis of the task.
• LESA: Linguistic Encapsulation and Semantic Amalgamation Based Generalised Claim Detection from Online Content
  • Shreya Gupta, Parantak Singh, Megha Sundriyal, Md. Shad Akhtar, Tanmoy Chakraborty
  • TLDR: We propose LESA, a novel approach to annotate unstructured unstructural claims by combining syntactic features and contextual features.
• Interpretability for Morphological Inflection: from Character-level Predictions to Subword-level Rules
  • Tatyana Ruzsics, Olga Sozinova, Ximena Gutierrez-Vasques, Tanja Samardzic
  • TLDR: We propose a novel approach for pattern extraction from attention weights to interpret what the model learn.
• Expanding, Retrieving and Infilling: Diversifying Cross-Domain Question Generation with Flexible Templates
  • Xiaojing Yu, Anxiao Jiang
  • TLDR: We propose a novel framework for question generation over SQL database that incorporates flexible templates with a neural-based model to generate diverse expressions of questions with sentence structure guidance.
• Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings
  • Ohjoon Kwon, Dohyun Kim, Soo-Ryeon Lee, Junyoung Choi, SangKeun Lee
  • TLDR: We propose a robust Hangeul word embedding model against typos, while maintaining high performance.
• Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation
  • Julia Ive, Andy Mingren Li, Yishu Miao, Ozan Caglayan, Pranava Madhyastha, Lucia Specia
  • TLDR: We propose a multimodal approach to simultaneous machine translation using reinforcement learning, with strategies to integrate visual and textual information in both the agent and the environment.
• STAR: Cross-modal [STA]tement [R]epresentation for selecting relevant mathematical premises
  • Deborah Ferreira, André Freitas
  • TLDR: We propose STAR, a model that uses cross-modal attention to learn how to represent mathematical text for the task of Natural Language Premise Selection.
• Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?
  • Yixuan Tang, Hwee Tou Ng, Anthony Tung
  • TLDR: We propose a new sub-question evaluation method for multi-hop question answering, which provides a new insight into the reasoning process of multi-hypothesis QA systems.
• Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
  • Nora Kassner, Philipp Dufter, Hinrich Schütze
  • TLDR: We investigate the ability of multilingual models to learn knowledge bases.
• Variational Weakly Supervised Sentiment Analysis with Posterior Regularization
  • Ziqian Zeng, Yangqiu Song
  • TLDR: We propose a new variational approach to the weakly supervised sentiment analysis to better control the posterior distribution of the label assignment.
• Framing Word Sense Disambiguation as a Multi-Label Problem for Model-Agnostic Knowledge Integration
  • Simone Conia, Roberto Navigli
  • TLDR: We present a novel multi-label word sense disambiguation problem in which multiple senses can be assigned to each target word.
• Graph-based Fake News Detection using a Summarization Technique
  • Gihwan Kim, Youngjoong Ko
  • TLDR: We propose a novel graph-based fake news detection method using a summarization technique that uses only the document internal information.
• Cognition-aware Cognate Detection
  • Diptesh Kanojia, Prashant Sharma, Sayali Ghodekar, Pushpak Bhattacharyya, Gholamreza Haffari, Malhar Kulkarni
  • TLDR: We propose a novel method for enriching the feature sets of cognate detection using human readers’ gaze behaviour.
• A Simple Three-Step Approach for the Automatic Detection of Exaggerated Statements in Health Science News
  • Jasabanta Patro, Sabyasachee Baruah
  • TLDR: We present a simple yet rational three-step approach to identify whether a scientific statement is an exaggerated version of another.
• Modeling Coreference Relations in Visual Dialog
  • Mingxiao Li, Marie-Francine Moens
  • TLDR: We propose two novel and linguistically inspired soft constraints that can improve the model’s ability of resolving coreferences in dialog in an unsupervised way.
• Increasing Robustness to Spurious Correlations using Forgettable Examples
  • Yadollah Yaghoobzadeh, Soroush Mehri, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
  • TLDR: We propose a new approach to robustify neural NLP models by finding minority examples without prior knowledge of the spurious correlations present in the dataset.
• On Robustness of Neural Semantic Parsers
  • Shuo Huang, Zhuang Li, Lizhen Qu, Lei Pan
  • TLDR: We provide the first empirical study on the robustness of semantic parsers in the presence of adversarial attacks.
• Benchmarking a transformer-FREE model for ad-hoc retrieval
  • Tiago Almeida, Sérgio Matos
  • TLDR: We empirically assess the feasibility of applying transformer-based models in real-world ad-hoc retrieval applications by comparison to a “greener and more sustainable” alternative, comprising only 620 trainable parameters.
• Reanalyzing the Most Probable Sentence Problem: A Case Study in Explicating the Role of Entropy in Algorithmic Complexity
  • Eric Corlett, Gerald Penn
  • TLDR: We use statistical measures such as entropy to give an updated analysis of the complexity of the NP-complete Most Probable Sentence problem for pCFGs, which can then be applied to word sense disambiguation and inference tasks.
• Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
  • Abhilasha Ravichander, Yonatan Belinkov, Eduard Hovy
  • TLDR: We show that neural models can learn to encode linguistic properties even if they are not needed for the task on which the model was trained.
• One-class Text Classification with Multi-modal Deep Support Vector Data Description
  • Chenlong Hu, Yukun Feng, Hidetaka Kamigaito, Hiroya Takamura, Manabu Okumura
  • TLDR: We present multi-modal deep SVDD for one-class text classification and show that it outperforms uni-modality and can get further improvements when negative supervision is incorporated.
• Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings
  • Christos Xypolopoulos, Antoine Tixier, Michalis Vazirgiannis
  • TLDR: We propose a novel algorithm to estimate polysemy based on simple geometry in the contextual embedding space.
• Mega-COV: A Billion-Scale Dataset of 100+ Languages for COVID-19
  • Muhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi, Dinesh Pabbi, Kunal Verma, Rannie Lin
  • TLDR: We describe Mega-COV, a billion-scale dataset from Twitter for studying COVID-19.
• Disfluency Correction using Unsupervised and Semi-supervised Learning
  • Nikhil Saini, Drumil Trivedi, Shreya Khare, Tejas Dhamecha, Preethi Jyothi, Samarth Bharadwaj, Pushpak Bhattacharyya
  • TLDR: We present a novel unsupervised style-transfer model for speech that converts disfluent to fluent text.
• Complex Question Answering on knowledge graphs using machine translation and multi-task learning
  • Saurabh Srivastava, Mayur Patidar, Sudip Chowdhury, Puneet Agarwal, Indrajit Bhattacharya, Gautam Shroff
  • TLDR: We propose a multi-task BERT based Neural Machine Translation (NMT) model to address multiple questions answering over a knowledge graph over a Knowledge Graph.
• Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation
  • Asa Cooper Stickland, Xian Li, Marjan Ghazvininejad
  • TLDR: We show that freezing parameters and adding new ones improve performance on machine translation tasks.
• From characters to words: the turning point of BPE merges
  • Ximena Gutierrez-Vasques, Christian Bentz, Olga Sozinova, Tanja Samardzic
  • TLDR: We show that text entropy values tend to converge at specific subword levels: relatively few BPE merges (around 350) lead to the most similar distributions across languages.
• A Large-scale Evaluation of Neural Machine Transliteration for Indic Languages
  • Anoop Kunchukuttan, Siddharth Jain, Rahul Kejriwal
  • TLDR: We take up the task of large-scale evaluation of neural machine transliteration between English and Indic languages, with a focus on multilingual transliterations to utilize orthographic similarity between Indian languages.
• Communicative-Function-Based Sentence Classification for Construction of an Academic Formulaic Expression Database
  • Kenichi Iwatsuki, Akiko Aizawa
  • TLDR: We propose a fully automated construction of a CF-labelled FE database using the top–down approach, in which the CF labels are first assigned to sentences, and then the FEs are extracted.
• Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations
  • Ilias Chalkidis, Manos Fergadiotis, Nikolaos Manginas, Eva Katakalou, Prodromos Malakasiotis
  • TLDR: We present a novel document-to-document information retrieval method for regulatory compliance, which is based on document-based queries and a novel neural re-ranker.
• The Chinese Remainder Theorem for Compact, Task-Precise, Efficient and Secure Word Embeddings
  • Patricia Thaine, Gerald Penn
  • TLDR: We propose a new method for compressing word vector embeddings into integers using the Chinese Reminder Theorem that speeds up addition by up to 48.27% and compresses GloVe word embedding libraries by up by 25.86%.
• Don’t Change Me! User-Controllable Selective Paraphrase Generation
  • Mohan Zhang, Luchen Tan, Zihang Fu, Kun Xiong, Jimmy Lin, Ming Li, Zhengkai Tu
  • TLDR: We provide the user with explicit tags that can be placed around any arbitrary segment of text to mean “don’t change me!” when generating a paraphrase; the model learns to explicitly copy these phrases to the output.
• Rethinking Coherence Modeling: Synthetic vs. Downstream Tasks
  • Tasnim Mohiuddin, Prathyusha Jwalapuram, Xiang Lin, Shafiq Joty
  • TLDR: We benchmark the performance of well-known traditional and neural coherence models on synthetic sentence ordering tasks and compare it with their performance on three downstream applications.
• From the Stage to the Audience: Propaganda on Reddit
  • Oana Balalau, Roxana Horincar
  • TLDR: Propaganda on Reddit: Who is posting propaganda and how is propaganda received?
• Probing for idiomaticity in vector space models
  • Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, Aline Villavicencio
  • TLDR: We propose probing measures to assess if some of the expected linguistic properties of noun compounds, especially those related to idiomatic meanings, and their dependence on context and sensitivity to lexical choice, are readily available in some standard and widely used representations.
• Is the Understanding of Explicit Discourse Relations Required in Machine Reading Comprehension?
  • Yulong Wu, Viktor Schlegel, Riza Batista-Navarro
  • TLDR: We propose a new methodology to measure the extent to which MRC datasets evaluate the understanding of explicit discourse relations.
• Why Is MBTI Personality Detection from Texts a Difficult Task?
  • Sanja Stajner, Seren Yenikent
  • TLDR: We present theoretical and empirical insights into the low accuracy of automatic MBTI personality detection on Twitter.
• Enconter: Entity Constrained Progressive Sequence Generation via Insertion-based Transformer
  • Lee Hsun Hsieh, Yang-Yin Lee, Ee-Peng Lim
  • TLDR: We propose a new insertion transformer that can generate sequences in parallel given some input tokens as constraint.
• Meta-Learning for Effective Multi-task and Multilingual Modelling
  • Ishan Tarunesh, Sushil Khyalia, Vishwajeet Kumar, Ganesh Ramakrishnan, Preethi Jyothi
  • TLDR: Meta-learning of natural language processing tasks and languages using knowledge of other languages.
• “Killing Me” Is Not a Spoiler: Spoiler Detection Model using Graph Neural Networks with Dependency Relation-Aware Attention Mechanism
  • Buru Chang, Inggeol Lee, Hyunjae Kim, Jaewoo Kang
  • TLDR: We propose a new spoiler detection model based on syntax-aware graph neural networks.
• BERTese: Learning to Speak to BERT
  • Adi Haviv, Jonathan Berant, Amir Globerson
  • TLDR: We propose a method for automatically rewriting queries into paraphrase queries that are directly optimized towards better knowledge extraction.
• Lifelong Knowledge-Enriched Social Event Representation Learning
  • Prashanth Vijayaraghavan, Deb Roy
  • TLDR: We propose a new method for learning social event embeddings by leveraging social commonsense knowledge and continual learning strategies.
• GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition
  • Xinyan Zhao, Haibo Ding, Zhe Feng
  • TLDR: We propose GLARA, a graph-based labeling rule augmentation framework, to learn new labeling rules from unlabeled data.
• An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning
  • Markus Eberts, Adrian Ulges
  • TLDR: We present a multi-task approach for entity-level relation extraction from documents.
• WER-BERT: Automatic WER Estimation with BERT in a Balanced Ordinal Classification Paradigm
  • Akshay Krishna Sheshadri, Anvesh Rao Vijjini, Sukhdeep Kharbanda
  • TLDR: We propose a new balanced paradigm for automatic WER evaluation and a distance loss function for automatic classification of automatic speech recognition systems.
• Two Training Strategies for Improving Relation Extraction over Universal Graph
  • Qin Dai, Naoya Inoue, Ryo Takahashi, Kentaro Inui
  • TLDR: We propose two novel methods for learning a Universal Graph for DS-RE and show that they can improve the performance of the model.
• Adaptation of Back-translation to Automatic Post-Editing for Synthetic Data Generation
  • WonKee Lee, Baikjin Jung, Jaehun Shin, Jong-Hyeok Lee
  • TLDR: We propose two new methods of synthesizing additional MT outputs by adapting back-translation to the APE task, obtaining robust enlargements of the existing synthetic APE training dataset.
• Removing Word-Level Spurious Alignment between Images and Pseudo-Captions in Unsupervised Image Captioning
  • Ukyo Honda, Yoshitaka Ushiku, Atsushi Hashimoto, Taro Watanabe, Yuji Matsumoto
  • TLDR: We propose a simple gating mechanism that is trained to align image features with only the most reliable words in pseudo-captions: the detected object labels.
• Towards More Fine-grained and Reliable NLP Performance Prediction
  • Zihuiwen Ye, Pengfei Liu, Jinlan Fu, Graham Neubig
  • TLDR: We propose new methods for improving performance prediction for NLP tasks.
• Metrical Tagging in the Wild: Building and Annotating Poetry Corpora with Rhythmic Features
  • Thomas Haider
  • TLDR: We provide large poetry corpora for English and German, and annotate prosodic features in smaller corpora to train corpus driven neural models that enable robust large scale analysis.
• Enhancing Aspect-level Sentiment Analysis with Word Dependencies
  • Yuanhe Tian, Guimin Chen, Yan Song
  • TLDR: We propose a novel approach to enhance aspect-level sentiment analysis with word dependencies, where the type information is modeled by key-value memory networks and different dependency results are selectively leveraged.