EMNLP 2021

• German
• Italian
• Chinese
• Japanese

TLDRs

• Knowledge Router: Learning Disentangled Representations for Knowledge Graphs
  • Shuai Zhang, Xi Rao, Yi Tay, Ce Zhang
  • TLDR: We propose a new method for learning disentangled representations of KG entities that are more complex than we think.
• Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors
  • Fenia Christopoulou, Makoto Miwa, Sophia Ananiadou
  • TLDR: We propose a multi-task, probabilistic approach to facilitate distantly supervised relation extraction by bringing closer the representations of sentences that contain the same Knowledge Base pairs.
• Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks
  • Minh Van Nguyen, Viet Dac Lai, Thien Huu Nguyen
  • TLDR: We propose a novel deep learning model to solve the four tasks of information extraction in a single model (called FourIE) that captures the connections between the types expressed in an input sentence.
• Abstract Meaning Representation Guided Graph Encoding and Decoding for Joint Information Extraction
  • Zixuan Zhang, Heng Ji
  • TLDR: We propose a novel AMR-guided framework for joint information extraction and show that it can significantly improve the performance of Rich Semantic Parsing and Information Extraction.
• A Frustratingly Easy Approach for Entity and Relation Extraction
  • Zexuan Zhong, Danqi Chen
  • TLDR: We present a simple pipelined approach for entity and relation extraction which achieves state-of-the-art results on ACE04, ACE05 and SciERC.
• Event Time Extraction and Propagation via Graph Attention Networks
  • Haoyang Wen, Yanru Qu, Heng Ji, Qiang Ning, Jiawei Han, Avi Sil, Hanghang Tong, Dan Roth
  • TLDR: Graph attention network-based approach to embed temporal information over document-level event graphs.
• Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers
  • Hongfei Xu, Josef van Genabith, Qiuhui Liu, Deyi Xiong
  • TLDR: We show that word translation already happens progressively in Transformer layers and even in the input embeddings.
• Mediators in Determining what Processing BERT Performs First
  • Aviv Slobodkin, Leshem Choshen, Omri Abend
  • TLDR: We show that not controlling for context length in neural model predictions can lead to contradictory conclusions as to the localization patterns of the network, depending on the distribution of the probing dataset.
• Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
  • Yonatan Bitton, Gabriel Stanovsky, Roy Schwartz, Michael Elhadad
  • TLDR: We present a novel method which leverages rich semantic input representation to automatically generate contrast sets for the visual question answering task GQA and show that, despite GQAs compositionality and carefully balanced label distribution, two high-performing models drop 13-17% in accuracy compared to the original test set.
• Multilingual Language Models Predict Human Reading Behavior
  • Nora Hollenstein, Federico Pirovano, Ce Zhang, Lena Jäger, Lisa Beinborn
  • TLDR: We analyze if large language models are able to predict patterns of human reading behavior.
• Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing
  • Rowan Hall Maudslay, Ryan Cotterell
  • TLDR: We show that syntactic probes do not properly isolate syntax.
• A Non-Linear Structural Probe
  • Jennifer C. White, Tiago Pimentel, Naomi Saphra, Ryan Cotterell
  • TLDR: We propose a novel non-linear probe model that learns metric representations and show that it can encode syntactic structure non-linearly.
• Concealed Data Poisoning Attacks on NLP Models
  • Eric Wallace, Tony Zhao, Shi Feng, Sameer Singh
  • TLDR: We develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input.
• Backtranslation Feedback Improves User Confidence in MT, Not Quality
  • Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya
  • TLDR: We show that backward translation feedback has a mixed effect on the whole process: it increases user confidence in the produced translation, but not the objective quality.
• Data Filtering using Cross-Lingual Word Embeddings
  • Christian Herold, Jan Rosendahl, Joris Vanvinckenroye, Hermann Ney
  • TLDR: We present several novel methods for data filtering for machine translation, based on cross-lingual word embeddings, which outperform the strongest data filtering methods from the WMT 2018 shared task.
• Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation
  • Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser
  • TLDR: We propose a new method for cross-lingual pretraining in unsupervised neural machine translation using lexical-level subword embeddings.
• Neural Machine Translation without Embeddings
  • Uri Shaham, Omer Levy
  • TLDR: Byte-to-byte machine translation with token dropout.
• Counterfactual Data Augmentation for Neural Machine Translation
  • Qi Liu, Matt Kusner, Phil Blunsom
  • TLDR: We propose a data augmentation method for neural machine translation that improves translation, backtranslation and translation robustness.
• Cultural and Geographical Influences on Image Translatability of Words across Languages
  • Nikzad Khani, Isidora Tourni, Mohammad Sadegh Rasooli, Chris Callison-Burch, Derry Tanti Wijaya
  • TLDR: We study image translatability of words, which we define as the translatable of words via images, by measuring intra- and inter-cluster similarities of image representations of words that are translations of each other.
• Multilingual BERT Post-Pretraining Alignment
  • Lin Pan, Chung-Wei Hang, Haode Qi, Abhishek Shah, Saloni Potdar, Mo Yu
  • TLDR: We propose a simple method to align multilingual contextual embeddings as a post-pretraining step for improved cross-lingual transferability of the pretrained language models.
• A Million Tweets Are Worth a Few Points: Tuning Transformers for Customer Service Tasks
  • Amir Hadifar, Sofie Labat, Veronique Hoste, Chris Develder, Thomas Demeester
  • TLDR: Pretraining a generic multilingual transformer model on a multilingual social media corpus, before finetuning on specific end tasks, consistently boosts performance, especially in non-English settings.
• Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases
  • Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis
  • TLDR: We introduce paragraph-level rationale extraction and propose a new constraint for paragraph-based rationale extraction.
• Answering Product-Questions by Utilizing Questions from Other Contextually Similar Products
  • Ohad Rozen, David Carmel, Avihai Mejer, Vitaly Mirkis, Yftah Ziser
  • TLDR: We propose a novel and complementary approach for answering subjective and opinion-based questions based on the answers for similar questions asked on similar products.
• EnSidNet: Enhanced Hybrid Siamese-Deep Network for grouping clinical trials into drug-development pathways
  • Lucia Pagani
  • TLDR: We present a novel approach for drug-development pathway reconstruction based on a hybrid Siamese-Deep Neural Network and a novel algorithm for semi-supervised learning.
• DATE: Detecting Anomalies in Text via Self-Supervision of Transformers
  • Andrei Manolache, Florin Brad, Elena Burceanu
  • TLDR: We learn a novel pretext task for anomaly detection in text using deep neural networks and show strong performance on both semi-supervised and unsupervised datasets.
• A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code
  • Nadezhda Chirkova, Sergey Troshin
  • TLDR: We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers.
• Fast and Scalable Dialogue State Tracking with Explicit Modular Decomposition
  • Dingmin Wang, Chenghua Lin, Qi Liu, Kam-Fai Wong
  • TLDR: We present a fast and scalable architecture called Explicit Modular Decomposition (EMD), in which we incorporate both classification-based and extraction-based methods and design four modules (for clas- sification and sequence labelling) to jointly extract dialogue states.
• Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks
  • Nandan Thakur, Nils Reimers, Johannes Daxenberger, Iryna Gurevych
  • TLDR: We present a simple yet efficient data augmentation strategy for pairwise sentence scoring that improves the performance of both cross- and bi-encoders.
• SmBoP: Semi-autoregressive Bottom-up Semantic Parsing
  • Ohad Rubin, Jonathan Berant
  • TLDR: We propose a new semi-autoregressive bottom-up parser for semantic parsing that constructs at decoding step t the top-K sub-trees of height ≤ t.
• SGL: Speaking the Graph Languages of Semantic Parsing via Multilingual Translation
  • Luigi Procopio, Rocco Tripodi, Roberto Navigli
  • TLDR: We propose a new architecture for graph-based semantic parsing that is capable of parsing multiple formalisms and outperforms all existing models on cross-lingual cross-language parsing.
• Unifying Cross-Lingual Semantic Role Labeling with Heterogeneous Linguistic Resources
  • Simone Conia, Andrea Bacciu, Roberto Navigli
  • TLDR: We present a unified model for cross-lingual SRL over heterogeneous linguistic resources.
• Fool Me Twice: Entailment from Wikipedia Gamification
  • Julian Eisenschlos, Bhuwan Dhingra, Jannis Bulian, Benjamin Börschinger, Jordan Boyd-Graber
  • TLDR: We present FoolMeTwice, a large dataset of challenging entailment pairs collected through a fun multi-player game.
• Meta-Learning for Domain Generalization in Semantic Parsing
  • Bailin Wang, Mirella Lapata, Ivan Titov
  • TLDR: Meta-learning for semantic parsing.
• Aspect-Controlled Neural Argument Generation
  • Benjamin Schiller, Johannes Daxenberger, Iryna Gurevych
  • TLDR: We present the Arg-CTRL - a language model for argument generation that can be controlled to generate sentence-level arguments for a given topic, stance, and aspect.
• Text Generation from Discourse Representation Structures
  • Jiangming Liu, Shay B. Cohen, Mirella Lapata
  • TLDR: We propose neural models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs).
• APo-VAE: Text Generation in Hyperbolic Space
  • Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, Jingjing Liu
  • TLDR: We propose a novel adversarial adversarial model for learning hierarchical language representations in hyperbolic latent space.
• DART: Open-Domain Structured Data Record to Text Generation
  • Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani
  • TLDR: We present DART, an open domain structured DAta Record to Text generation dataset with over 82k instances (DARTs).
• When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
  • Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, Djamé Seddah
  • TLDR: We show that large-scale multilingual language models behave in multiple ways on unseen languages and show that transliterating those languages significantly improves the potential of large-Scale multilingual models on downstream tasks.
• Multi-Adversarial Learning for Cross-Lingual Word Embeddings
  • Haozhou Wang, James Henderson, Paola Merlo
  • TLDR: We propose a multi-adversarial learning method for cross-lingual word embeddings, which improves performance over previous single-mapping methods, especially for distant languages.
• Multi-view Subword Regularization
  • Xinyi Wang, Sebastian Ruder, Graham Neubig
  • TLDR: We propose Multi-view Subword Regularization, a method for fine-tuning pre-trained multilingual representations that improves the effectiveness of cross-lingual transfer.
• mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
  • Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel
  • TLDR: We present a multilingual variant of T5 that achieves state-of-the-art results on a wide variety of English-language NLP tasks.
• MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
  • Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
  • TLDR: Meta-learning based transfer learning for extremely low-resource languages.
• Open Domain Question Answering over Tables via Dense Retrieval
  • Jonathan Herzig, Thomas Müller, Syrine Krichene, Julian Eisenschlos
  • TLDR: We present a new retriever for open-domain QA over tables that improves retrieval results from 72.0 to 81.1 recall@10 and end-to-end QA results from 33.8 to 37.7 exact match, over a BERT based retriever.
• Open-Domain Question Answering Goes Conversational via Question Rewriting
  • Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, Srinivas Chappidi
  • TLDR: We present a new dataset for question rewriting in conversational context and provide a strong baseline for question answering.
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
  • Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec
  • TLDR: We propose a new model, QA-GNN, which addresses the problem of answering questions using knowledge from pre-trained language models and knowledge graphs, and show its improvement over existing LM and LM+KG models, as well as its capability to perform interpretable and structured reasoning, e.g., correctly handling negation in questions.
• XOR QA: Cross-lingual Open-Retrieval Question Answering
  • Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, Hannaneh Hajishirzi
  • TLDR: We present a new cross-lingual open-retrieval question answering task framework for multilingual question answering.
• SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
  • Tiancheng Zhao, Xiaopeng Lu, Kyusong Lee
  • TLDR: We introduce SPARTA, a novel neural retrieval method that shows great promise in performance, generalization, and interpretability for open-domain question answering.
• Implicitly Abusive Language – What does it actually look like and why are we not getting there?
  • Michael Wiegand, Josef Ruppenhofer, Elisabeth Eder
  • TLDR: We present a list of subtypes of implicitly abusive language and formulate research tasks and questions for future research.
• The Importance of Modeling Social Factors of Language: Theory and Practice
  • Dirk Hovy, Diyi Yang
  • TLDR: We show that current NLP systems systematically break down when faced with interpreting the social factors of language.
• On learning and representing social meaning in NLP: a sociolinguistic perspective
  • Dong Nguyen, Laura Rosseel, Jack Grieve
  • TLDR: We introduce the concept of social meaning to NLP and discuss how insights from sociolinguistics can inform work on representation learning in NLP.
• Preregistering NLP research
  • Emiel van Miltenburg, Chris van der Lee, Emiel Krahmer
  • TLDR: We present several questions for NLP researchers to preregister their work, and propose a new form for NPs that could provide firmer grounds for slow science in NLP research.
• Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
  • Tal Schuster, Adam Fisch, Regina Barzilay
  • TLDR: We present VitaminC, a benchmark infused with challenging cases that require fact verification models to discern and adjust to slight factual changes.
• Representing Numbers in NLP: a Survey and a Vision
  • Avijit Thawani, Jay Pujara, Filip Ilievski, Pedro Szekely
  • TLDR: We present a comprehensive taxonomy of tasks and methods for numeracy in NLP and provide a unified evaluation of the number representation in text.
• Extending Multi-Document Summarization Evaluation to the Interactive Setting
  • Ori Shapira, Ramakanth Pasunuru, Hadar Ronen, Mohit Bansal, Yael Amsterdamer, Ido Dagan
  • TLDR: We develop an end-to-end evaluation framework for interactive summarization, focusing on expansion-based interaction, which considers the accumulating information along a user session.
• Identifying Helpful Sentences in Product Reviews
  • Iftah Gamzu, Hila Gonen, Gilad Kutiel, Ran Levy, Eugene Agichtein
  • TLDR: We propose a novel task for extracting helpful sentences from a set of reviews for a given product and demonstrate its reliability despite the inherent subjectivity involved.
• Noisy Self-Knowledge Distillation for Text Summarization
  • Yang Liu, Sheng Shen, Mirella Lapata
  • TLDR: We propose a novel method for summarizing text that uses self-knowledge distillation to improve performance on single reference and noisy datasets.
• Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation
  • Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
  • TLDR: We introduce WikiTransfer, a novel and generalizable method for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner.
• Enhancing Factual Consistency of Abstractive Summarization
  • Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, Meng Jiang
  • TLDR: We propose a fact-aware summarization model FASum to extract and integrate factual relations into the summary generation process via graph attention.
• Few-shot Intent Classification and Slot Filling with Retrieved Examples
  • Dian Yu, Luheng He, Yuan Zhang, Xinya Du, Panupong Pasupat, Qi Li
  • TLDR: We propose a span-level retrieval method for few-shot learning that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective.
• “Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses
  • Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng
  • TLDR: We propose categories of ad hominems and a classifier to analyze human and dialogue system responses to English Twitter posts.
• Human-like informative conversations: Better acknowledgements using conditional mutual information
  • Ashwin Paranjape, Christopher Manning
  • TLDR: We show that conversational history is better captured by pointwise conditional mutual information than by the established use of pointwise mutual information (pmi).
• A Comparative Study on Schema-Guided Dialogue State Tracking
  • Jie Cao, Yi Zhang
  • TLDR: We present a new set of bench-marking descriptions for describing the domain ontology of dialog state tracking models and show the robustness of the model on both homogeneous and heterogeneous description styles.
• Spoken Language Understanding for Task-oriented Dialogue Systems with Augmented Memory Networks
  • Jie Wu, Ian Harris, Hongzhi Zhao
  • TLDR: We propose a novel approach to model long-term slot context and to fully utilize the semantic correlation between slots and intents.
• How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
  • Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston
  • TLDR: We propose a novel reinforcement learning system that learns to act and talk naturally with respect to their motivations in a large-scale crowd-sourced fantasy text-game.
• Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas
  • Yogarshi Vyas, Miguel Ballesteros
  • TLDR: We present a new approach to entity linking that uses flat strings to convert entities from arbitrary KBs into flat strings and improve generalization of existing models.
• Self-Training with Weak Supervision
  • Giannis Karamanolakis, Subhabrata Mukherjee, Guoqing Zheng, Ahmed Hassan Awadallah
  • TLDR: We develop a weak supervision framework for task-specific unlabeled data and a semi-supervised learning objective for end-to-end training with unlabeling data, domain-specific rules, and a small amount of labeled data.
• Neural Language Modeling for Contextualized Temporal Graph Generation
  • Aman Madaan, Yiming Yang
  • TLDR: We present a novel algorithm for generating graph-level temporal graphs for document generation using large-scale pre-trained language models.
• Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning
  • Xuelu Chen, Michael Boratko, Muhao Chen, Shib Sankar Dasgupta, Xiang Lorraine Li, Andrew McCallum
  • TLDR: We propose a novel uncertain knowledge graph embedding method with probabilistic semantics and a novel knowledge graph model that can capture high-order dependencies among facts.
• Document-Level Event Argument Extraction by Conditional Generation
  • Sha Li, Heng Ji, Jiawei Han
  • TLDR: We propose a document-level neural event argument extraction model by formulating the task as conditional generation following event templates.
• Template Filling with Generative Transformers
  • Xinya Du, Alexander Rush, Claire Cardie
  • TLDR: We present a novel framework for template filling that models the dependence between entities in a document and across multiple events.
• Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models
  • Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu
  • TLDR: We propose a shortcut mitigation framework for NLU models which can suppress the model from making overconfident predictions for samples with large shortcut degree.
• On Attention Redundancy: A Comprehensive Study
  • Yuchen Bian, Jiaji Huang, Xingyu Cai, Jiahong Yuan, Kenneth Church
  • TLDR: We provide a comprehensive study on attention redundancy in multi-layer multi-head self-attention mechanism and propose a simple zero-shot pruning method for GLUE tasks.
• Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?
  • Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, Byron Wallace
  • TLDR: We propose a battery of approaches intended to recover Personal Health Information (PHI) from a trained BERT.
• Low-Complexity Probing via Finding Subnetworks
  • Steven Cao, Victor Sanh, Alexander Rush
  • TLDR: We propose a new subnetwork probe for linguistic properties that is both better at finding properties of interest and worse at learning on its own.
• An Empirical Comparison of Instance Attribution Methods for NLP
  • Pouya Pezeshkpour, Sarthak Jain, Byron Wallace, Sameer Singh
  • TLDR: We evaluate the degree to which different potential instance attribution methods agree with respect to the importance of training samples.
• Generalization in Instruction Following Systems
  • Soham Dan, Michael Zhou, Dan Roth
  • TLDR: We propose a new set of expectations for instruction following models and show that state-of-the-art models fall short of these expectations and are extremely brittle.
• LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  • Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, Jingjing Liu
  • TLDR: We present a novel approach for Image-text retrieval that significantly speeds up inference time of pre-trained models without sacrificing accuracy.
• Measuring Social Biases in Grounded Vision and Language Embeddings
  • Candace Ross, Boris Katz, Andrei Barbu
  • TLDR: We generalize the notion of measuring social biases in word embeddings to visually grounded word embedding and show that these biases are present in both language and vision.
• MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences
  • Jianing Yang, Yongxin Wang, Ruitao Yi, Yuying Zhu, Azaan Rehman, Amir Zadeh, Soujanya Poria, Louis-Philippe Morency
  • TLDR: We propose a graph-based neural model for multimodal sequential data that captures rich interactions across modalities and through time.
• Grounding Open-Domain Instructions to Automate Web Support Tasks
  • Nancy Xu, Sam Masling, Michael Du, Giovanni Campagna, Larry Heck, James Landay, Monica Lam
  • TLDR: We propose a novel method for grounding natural language instructions on the web to perform previously unseen tasks.
• Modular Networks for Compositional Instruction Following
  • Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell
  • TLDR: We propose a modular architecture for following natural language instructions that describe sequences of diverse subgoals.
• Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information
  • Jialu Li, Hao Tan, Mohit Bansal
  • TLDR: Synthesis information derived from dependency trees can help to align the language instructions with the current visual information.
• Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning
  • Hui Liu, Danqing Zhang, Bing Yin, Xiaodan Zhu
  • TLDR: We propose a Reinforced label Hierarchy Reasoning (RLHR) approach to improve pretrained models with label hierarchies on the ZS-MTC task.
• Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach
  • Yue Yu, Simiao Zuo, Haoming Jiang, Wendi Ren, Tuo Zhao, Chao Zhang
  • TLDR: We develop a contrastive self-training framework for fine-tuning pre-trained language models without any labeled data.
• Posterior Differential Regularization with f-divergence for Improving Model Robustness
  • Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao
  • TLDR: We address the problem of enhancing model robustness through regularization.
• Understanding Hard Negatives in Noise Contrastive Estimation
  • Wenzheng Zhang, Karl Stratos
  • TLDR: We show that the choice of negative examples is important in noise contrastive estimation, and show that setting the negative distribution to be the model distribution results in bias reduction.
• Certified Robustness to Word Substitution Attack with Differential Privacy
  • Wenjie Wang, Pengfei Tang, Jian Lou, Li Xiong
  • TLDR: We propose WordDP to achieve certified robustness against word substitution at- tacks in text classification via differential privacy (DP).
• DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference
  • Shikhar Murty, Tatsunori B. Hashimoto, Christopher Manning
  • TLDR: We propose DReCA (Decomposing datasets into Reasoning Categories), a simple method for discovering and using latent reasoning categories in a dataset, to form additional high quality tasks.
• Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
  • Xavier Garcia, Aditya Siddhant, Orhan Firat, Ankur Parikh
  • TLDR: We present a novel multilinguality-based unsupervised translation algorithm for 5 low-resource languages to and from English directions, which outperforms all current state-of-the-art unsupervisional baselines for these languages.
• Macro-Average: Rare Types Are Important Too
  • Thamme Gowda, Weiqiu You, Constantine Lignos, Jonathan May
  • TLDR: We present a type-based classifier metric for machine translation evaluation that can be used to assess the adequacy of machine translation models.
• Assessing Reference-Free Peer Evaluation for Machine Translation
  • Sweta Agrawal, George Foster, Markus Freitag, Colin Cherry
  • TLDR: We present a robust and robust model for translation evaluation that matches BLEU.
• The Curious Case of Hallucinations in Neural Machine Translation
  • Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt
  • TLDR: We present a new hypothesis that explains hallucinations in neural machine translation under source perturbation and show that they can be generated and explained through specific corpus-level noise patterns.
• Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
  • Xavier Garcia, Noah Constant, Ankur Parikh, Orhan Firat
  • TLDR: We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual translation.
• Towards Modeling the Style of Translators in Neural Machine Translation
  • Yue Wang, Cuong Hoang, Marcello Federico
  • TLDR: We show that style-augmented translation models are able to capture the style variations of translators and to generate translations with different styles on new data.
• Self-Supervised Test-Time Learning for Reading Comprehension
  • Pratyay Banerjee, Tejas Gokhale, Chitta Baral
  • TLDR: We present a novel method for reading comprehension that performs test-time learning on text passage without requiring training on large-scale human-authored datasets.
• Capturing Row and Column Semantics in Transformer Based Question Answering over Tables
  • Michael Glass, Mustafa Canim, Alfio Gliozzo, Saneem Chemmengath, Vishwajeet Kumar, Rishav Chakravarti, Avi Sil, Feifei Pan, Samarth Bharadwaj, Nicolas Rodolfo Fauceglia
  • TLDR: We propose two novel approaches for online QA systems over tables by materializing embeddings for existing tables.
• Explainable Multi-hop Verbal Reasoning Through Internal Monologue
  • Zhengzhong Liang, Steven Bethard, Mihai Surdeanu
  • TLDR: We propose the Explainable multi-hop Verbal Reasoner, a new SOTA language model that can generate all reasoning steps in natural language and generalizes better than other strong methods when trained on simpler tasks or less training data.
• Robust Question Answering Through Sub-part Alignment
  • Jifan Chen, Greg Durrett
  • TLDR: We model question answering as an alignment problem and use it to explore constraints to prevent certain types of bad model behavior arising in cross-domain settings.
• Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models
  • Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, Ashish Sabharwal
  • TLDR: We propose a general framework called Text Modular Networks(TMNs) for building interpretable systems that learn to solve complex tasks by decomposing them into simpler ones solvable by existing models.
• RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
  • Srinivasan Iyer, Sewon Min, Yashar Mehdad, Wen-tau Yih
  • TLDR: We develop a successful re-ranking approach for span-extraction tasks that improves upon the performance of MRC models, even beyond large-scale pre-training.
• On the Transferability of Minimal Prediction Preserving Inputs in Question Answering
  • Shayne Longpre, Yi Lu, Chris DuBois
  • TLDR: We investigate the existence of short, uninterpretable input fragments that yield high confidence and accuracy in neural models.
• Understanding by Understanding Not: Modeling Negation in Language Models
  • Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, Aaron Courville
  • TLDR: We propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus.
• DuoRAT: Towards Simpler Text-to-SQL Models
  • Torsten Scholak, Raymond Li, Dzmitry Bahdanau, Harm de Vries, Chris Pal
  • TLDR: We propose a new model for text-to-SQL translation that uses only relation-aware transformers as the building blocks.
• Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization
  • Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Lorraine Li, Pavan Kapanipathi, Kartik Talamadupula
  • TLDR: We show that the relatively shorter length of premises in traditional NLI datasets is the primary challenge prohibiting usage in downstream applications (which do better with longer contexts); (2) this challenge can be addressed by automatically converting resource-rich reading comprehension datasets into longer-premise NLI dataset; and (3) models trained on the converted, longer-term datasets outperform those trained using short-premises traditional NLIs on downstream tasks primarily due to the difference in premise lengths.
• Structure-Grounded Pretraining for Text-to-SQL
  • Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, Matthew Richardson
  • TLDR: Weakly supervised Structure-Grounded pretraining framework for text-to-SQL.
• Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System
  • Congying Xia, Wenpeng Yin, Yihao Feng, Philip Yu
  • TLDR: We propose a novel text classification task in the NLP domain where the system incrementally learns new classes by incrementally learning a batch of new classes with few labeled examples per class.
• Temporal Reasoning on Implicit Events from Distant Supervision
  • Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, Dan Roth
  • TLDR: We propose a neuro-symbolic temporal reasoning model that can infer implicit events from natural language text.
• Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models
  • James Y. Huang, Kuan-Hao Huang, Kai-Wei Chang
  • TLDR: We present a syntax-guided sentence embedding model that learns to disentangle semantics and syntax in sentence embeddings obtained by pre-trained language models.
• Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs
  • Jiaao Chen, Diyi Yang
  • TLDR: We propose to explicitly model the rich structures in conversations for more precise and accurate conversation summarization, by first incorporating discourse relations between utterances and action triples (“who-doing-what”) in utterances through structured graphs to better encode conversations, and then designing a multi-granularity decoder to generate summaries by combining all levels of information.
• A New Approach to Overgenerating and Scoring Abstractive Summaries
  • Kaiqiang Song, Bingqing Wang, Zhe Feng, Fei Liu
  • TLDR: We propose a new approach to generate multiple variants of the target summary with diverse content and varying lengths, then score and select admissible ones according to users’ needs.
• D2S: Document-to-Slide Generation Via Query-Based Text Summarization
  • Edward Sun, Yufang Hou, Dakuo Wang, Yunfeng Zhang, Nancy X. R. Wang
  • TLDR: We present D2S, a novel system that tackles the document-to-slides generation task with a two-step approach: 1) Use slide titles to retrieve relevant and engaging text, figures, and tables; 2) Summarize the retrieved context into bullet points with long-form question answering.
• Efficient Attentions for Long Document Summarization
  • Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, Lu Wang
  • TLDR: We propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source.
• RefSum: Refactoring Neural Summarization
  • Yixin Liu, Zi-Yi Dou, Pengfei Liu
  • TLDR: We present a unified view of text summarization and summaries combination and propose a novel framework for text summarisation and summary combination.
• Annotating and Modeling Fine-grained Factuality in Summarization
  • Tanya Goyal, Greg Durrett
  • TLDR: We explore both synthetic and human-labeled data sources for training models to identify factual errors in summarization, and study factuality at the word-, dependency-, and sentence-level.
• Larger-Context Tagging: When and Why Does It Work?
  • Jinlan Fu, Liangjing Feng, Qi Zhang, Xuanjing Huang, Pengfei Liu
  • TLDR: We investigate when and why larger-context training can work in tagging tasks and present an attribute-aided evaluation method to interpret the improvement brought by larger-Context training.
• Neural Sequence Segmentation as Determining the Leftmost Segments
  • Yangming Li, Lemao Liu, Kaisheng Yao
  • TLDR: We propose a novel framework that incrementally segments natural language sentences at segment level.
• PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols
  • Songlin Yang, Yanpeng Zhao, Kewei Tu
  • TLDR: We present a new parameterization form of Probabilistic context-free grammars based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols.
• GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input
  • Tao Meng, Anjie Fang, Oleg Rokhlenko, Shervin Malmasi
  • TLDR: We propose GEMNET, a novel approach for gazetteer knowledge integration, including (1) a flexible Contextual Gazetteer Representation encoder that can be fused with any word-level model; and (2) a Mixture-of- Experts gating network that overcomes the feature overuse issue by learning to conditionally combine the context and gazeter features, instead of assigning them fixed weights.
• Video-aided Unsupervised Grammar Induction
  • Songyang Zhang, Linfeng Song, Lifeng Jin, Kun Xu, Dong Yu, Jiebo Luo
  • TLDR: We investigate video-aided grammar induction, which learns a constituency parser from both unlabeled text and its corresponding video.
• Generating Negative Samples by Manipulating Golden Responses for Unsupervised Learning of a Response Evaluation Model
  • ChaeHun Park, Eugene Jang, Wonsuk Yang, Jong Park
  • TLDR: We propose a method of manipulating a golden response to create a new negative response that is designed to be inappropriate within the context while maintaining high similarity with the original golden response.
• How Robust are Fact Checking Systems on Colloquial Claims?
  • Byeongchang Kim, Hyunwoo Kim, Seokhee Hong, Gunhee Kim
  • TLDR: We show that existing fact checking systems that perform well on claims in formal style significantly degenerate on colloquial claims with the same semantics.
• Fine-grained Post-training for Improving Retrieval-based Dialogue Systems
  • Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko, Jungyun Seo
  • TLDR: We propose a new fine-grained post-training method for dialogue retrieval that reflects the characteristics of the multi-turn dialogue.
• Put Chatbot into Its Interlocutor’s Shoes: New Framework to Learn Chatbot Responding with Intention
  • Hsuan Su, Jiun-Hao Jhan, Fan-yun Sun, Saurav Sahay, Hung-yi Lee
  • TLDR: We propose a novel framework to train chatbots to possess human-like intentions.
• Adding Chit-Chat to Enhance Task-Oriented Dialogues
  • Kai Sun, Seungwhan Moon, Paul Crook, Stephen Roller, Becka Silvert, Bing Liu, Zhiguang Wang, Honglei Liu, Eunjoon Cho, Claire Cardie
  • TLDR: Add Chit-Chat to ENhance Task-ORiented dialogues (ACCENTOR) to make virtual assistant conversations more engaging and interactive.
• Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network
  • Fan Jiang, Trevor Cohn
  • TLDR: We present a graph-based model to incorporate syntactic and semantic structures of sentences.
• Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition
  • Yingxue Zhang, Fandong Meng, Peng Li, Ping Jian, Jie Zhou
  • TLDR: Graph-based Context Tracking Network for Implicit Discourse relation recognition.
• Improving Neural RST Parsing Model with Silver Agreement Subtrees
  • Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, Masaaki Nagata
  • TLDR: We propose a method for improving neural RST parsing models by exploiting silver data, i.e., automatically annotated data.
• RST Parsing from Scratch
  • Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, Xiaoli Li
  • TLDR: We present a novel top-down end-to-end formulation of document level discourse parsing in the Rhetorical Structure Theory framework that uses a seq2seq network to model the splitting decisions.
• Did they answer? Subjective acts and intents in conversational discourse
  • Elisa Ferracane, Greg Durrett, Junyi Jessy Li, Katrin Erk
  • TLDR: We present the first discourse dataset with multiple and subjective interpretations of English conversation in the form of perceived conversation acts and intents.
• Evaluating the Impact of a Hierarchical Discourse Representation on Entity Coreference Resolution Performance
  • Sopan Khosla, James Fiacco, Carolyn Rosé
  • TLDR: We leverage automatically constructed discourse parse trees within a neural approach and demonstrate a significant improvement on two benchmark entity coreference-resolution datasets.
• Bridging Resolution: Making Sense of the State of the Art
  • Hideo Kobayashi, Vincent Ng
  • TLDR: We propose a hybrid rule-based and MTL approach that would enable a better understanding of their comparative strengths and weaknesses; and perform a manual analysis of the errors made by the MTL model.
• Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
  • Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
  • TLDR: Syntax-aware language model with incremental parser and dynamic oracle.
• Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation
  • Samuel Kiegeland, Julia Kreutzer
  • TLDR: Policy gradient algorithms have been criticized for their poor performance in NLP, but we provide empirical counter-evidence to these claims.
• Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study
  • Chongyang Tao, Shen Gao, Juntao Li, Yansong Feng, Dongyan Zhao, Rui Yan
  • TLDR: We investigate how order information in natural language learning can be used to encode natural languages without orders.
• Mask Attention Networks: Rethinking and Strengthen Transformer
  • Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang
  • TLDR: We present a novel understanding of Transformer as Mask Attention Networks (MANs) and show that they are two special cases of MANs with static mask matrices.
• ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
  • Dongling Xiao, Yu-Kun Li, Han Zhang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
  • TLDR: We propose ERNIE-Gram, an explicitly n-gram masking method to enhance the integration of coarse-grained information into pre-training.
• Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models
  • Yuxuan Lai, Yijia Liu, Yansong Feng, Songfang Huang, Dongyan Zhao
  • TLDR: We propose a novel pre-training paradigm for Chinese language models that explicitly incorporates word representations along with characters, thus can model a sentence in a multi-granularity manner.
• Modeling Event Plausibility with Consistent Conceptual Abstraction
  • Ian Porada, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
  • TLDR: We show that Transformer-based plausibility models are inconsistent across the conceptual classes of a lexical hierarchy, inferring that “a person breathing” is plausible while “A dentist breathing’ is not, for example.
• UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus
  • George Michalopoulos, Yuanxin Wang, Hussam Kaka, Helen Chen, Alexander Wong
  • TLDR: We present a novel knowledge augmentation strategy for contextual word embedding models that integrates clinical domain knowledge into word embeddings and outperforms existing domain-specific models on common named-entity recognition (NER) and clinical natural language inference tasks.
• Field Embedding: A Unified Grain-Based Framework for Word Representation
  • Junjie Luo, Xi Chen, Jichao Sun, Yuejia Xiang, Ningyu Zhang, Xiang Wan
  • TLDR: We propose a framework for learning word embeddings and grain embeddents that capture the semantic information of words and grain.
• MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theories
  • Minjin Choi, Sunkyung Lee, Eunseong Choi, Heesoo Park, Junhyuk Lee, Dongwon Lee, Jongwuk Lee
  • TLDR: We propose a novel metaphor detection model, namely
• Non-Parametric Few-Shot Learning for Word Sense Disambiguation
  • Howard Chen, Mengzhou Xia, Danqi Chen
  • TLDR: We propose MetricWSD, a non-parametric few-shot learning approach to mitigate the data imbalance issue in word sense disambiguation.
• Why Do Document-Level Polarity Classifiers Fail?
  • Karen Martins, Pedro O.S Vaz-de-Melo, Rodrygo Santos
  • TLDR: We propose a methodology to characterize, quantify and measure the impact of hard instances in the task of polarity classification of movie reviews.
• A Unified Span-Based Approach for Opinion Mining with Syntactic Constituents
  • Qingrong Xia, Bo Zhang, Rui Wang, Zhenghua Li, Yue Zhang, Fei Huang, Luo Si, Min Zhang
  • TLDR: We propose a unified span-based approach for fine-grained opinion mining and propose a novel method for syntactic constituents in the end-to-end approach.
• Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction
  • Yuhao Feng, Yanghui Rao, Yuyao Tang, Ninghua Wang, He Liu
  • TLDR: Target-Specified sequence labeling with multi-head self-attention for Aspect Based Sentiment Analysis.
• Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa
  • Junqi Dai, Hang Yan, Tianxiang Sun, Pengfei Liu, Xipeng Qiu
  • TLDR: We show that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree for the aspect-based sentiment analysis task.
• Domain Divergences: A Survey and Empirical Analysis
  • Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, Roger Zimmermann
  • TLDR: We propose a taxonomy of domain divergence measures and identify the common use-cases of these measures.
• Target-Aware Data Augmentation for Stance Detection
  • Yingjie Li, Cornelia Caragea
  • TLDR: We propose a new method for data augmentation of stance detection that generates target-aware sentences by predicting the masked word conditioned on both its context and the auxiliary sentence that contains target and label information.
• End-to-end ASR to jointly predict transcriptions and linguistic annotations
  • Motoi Omachi, Yuya Fujita, Shinji Watanabe, Matthew Wiesner
  • TLDR: We propose a Transformer-based sequence-to-sequence model for automatic speech recognition (ASR) capable of simultaneously transcribing and annotating audio with linguistic information such as phonemic transcripts or part-of-speech (POS) tags.
• Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation
  • Hirofumi Inaguma, Tatsuya Kawahara, Shinji Watanabe
  • TLDR: We propose sequence-level knowledge distillation from external text-based NMT models for end-to-end speech translation.
• Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
  • Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe
  • TLDR: We present an end-to-end framework that exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks.
• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
  • Yu-An Chung, Chenguang Zhu, Michael Zeng
  • TLDR: We propose a novel semi-supervised language understanding framework, SPLAT, to jointly pre-train the speech and language modules.
• Worldly Wise (WoW) - Cross-Lingual Knowledge Fusion for Fact-based Visual Spoken-Question Answering
  • Kiran Ramnath, Leda Sari, Mark Hasegawa-Johnson, Chang Yoo
  • TLDR: We present a new task and a synthetically-generated dataset to do Fact-based Visual Spoken-Question Answering (FVSQA) and a new reference implementation to perform end-to-end cross-lingual FVSQAs.
• Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment
  • Ethan A. Chi, Julian Salazar, Katrin Kirchhoff
  • TLDR: Iterative realignment of latent alignments in neural networks.
• Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis
  • Xiao Liu, Da Yin, Yansong Feng, Yuting Wu, Dongyan Zhao
  • TLDR: Graph-based Causal Inference framework for legal practitioners.
• Counterfactual Supporting Facts Extraction for Explainable Medical Record Based Diagnosis with Graph Network
  • Haoran Wu, Wei Chen, Shuang Xu, Bo Xu
  • TLDR: We propose a counterfactual multi-granularity graph supporting facts extraction method to extract supporting facts from irregular EMRs without external knowledge bases.
• Personalized Response Generation via Generative Split Memory Network
  • Yuwei Wu, Xuezhe Ma, Diyi Yang
  • TLDR: We develop a text generation system that generates personalized responses for Reddit questions by utilizing personalized user profiles and posting histories.
• Towards Few-shot Fact-Checking via Perplexity
  • Nayeon Lee, Yejin Bang, Andrea Madotto, Pascale Fung
  • TLDR: We propose a new way of utilizing the powerful transfer learning ability of a language model via a perplexity score for fact-checking.
• Active^2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation
  • Rishi Hazra, Parag Dutta, Shubham Gupta, Mohammed Abdul Qaathir, Ambedkar Dukkipati
  • TLDR: We propose a novel approach to manually annotate data by iteratively selecting a small number of examples for manual annotation based on their estimated utility in training the given model.
• Generating An Optimal Interview Question Plan Using A Knowledge Graph And Integer Linear Programming
  • Soham Datta, Prabir Mallick, Sangameshwar Patil, Indrajit Bhattacharya, Girish Palshikar
  • TLDR: We propose an interview assistant system to automatically, and in an objective manner, select an optimal set of technical questions (from question banks) personalized for a candidate.
• Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!
  • Xuanli He, Lingjuan Lyu, Lichao Sun, Qiongkai Xu
  • TLDR: We present adversarial attacks against BERT-based API services, and show that adversarial transferability can effectively compromise the target models.
• A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models
  • Kaiyuan Liao, Yi Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He
  • TLDR: We propose a novel Past-Future method to make comprehensive predictions from a global perspective.
• Masked Conditional Random Fields for Sequence Labeling
  • Tianwen Wei, Jianwei Qi, Shenghuan He, Songtao Sun
  • TLDR: We propose Masked Conditional Random Field (MCRF), an easy-to-implement variant of CRF that imposes restrictions on candidate paths during both training and decoding phases.
• Heterogeneous Graph Neural Networks for Concept Prerequisite Relation Learning in Educational Data
  • Chenghao Jia, Yongliang Shen, Yechun Tang, Lu Sun, Weiming Lu
  • TLDR: We propose a novel concept prerequisite relation learning approach, named CPRL, which combines both concept representation learned from a heterogeneous graph and concept pairwise features.
• Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models
  • Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He
  • TLDR: We present a new method for data poisoning NLP models by modifying word embedding vectors in a data-free way.
• DA-Transformer: Distance-aware Transformer
  • Chuhan Wu, Fangzhao Wu, Yongfeng Huang
  • TLDR: We propose a distance-aware Transformer that can exploit the real distance information of tokens to improve the performance of Transformer and improve the accuracy of the model.
• ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction
  • Jiahao Bu, Lei Ren, Shuang Zheng, Yang Yang, Jingang Wang, Fuzheng Zhang, Wei Wu
  • TLDR: We present a large-scale Chinese restaurant review dataset for sentiment analysis and propose a novel joint model for both tasks.
• Are NLP Models really able to Solve Simple Math Word Problems?
  • Arkil Patel, Satwik Bhattamishra, Navin Goyal
  • TLDR: We show that the existing MWP solvers rely on shallow heuristics to achieve high performance on the benchmark datasets.
• WRIME: A New Dataset for Emotional Intensity Estimation with Subjective and Objective Annotations
  • Tomoyuki Kajiwara, Chenhui Chu, Noriko Takemura, Yuta Nakashima, Hajime Nagahara
  • TLDR: We annotate 17,000 SNS posts with both the writer’s subjective emotional intensity and the reader’m objective one to construct a Japanese emotion analysis dataset.
• KPQA: A Metric for Generative Question Answering Using Keyphrase Weights
  • Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Joongbo Shin, Kyomin Jung
  • TLDR: We propose KPQA metric, a new metric for evaluating the correctness of GenQA.
• StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer
  • Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, Louis-Philippe Morency
  • TLDR: We introduce a large-scale benchmark for controllable text style transfer and show that existing methods struggle to model fine-grained changes and have an even more difficult time composing multiple styles.
• Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
  • Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei
  • TLDR: We propose a large and diverse Chinese dataset for creating and understanding cant from a computational linguistics perspective.
• COVID-19 Named Entity Recognition for Vietnamese
  • Thinh Hung Truong, Mai Hoang Dao, Dat Quoc Nguyen
  • TLDR: We present the first manually-annotated COVID-19 domain-specific dataset for Vietnamese.
• Framing Unpacked: A Semi-Supervised Interpretable Multi-View Model of Media Frames
  • Shima Khanehzar, Trevor Cohn, Gosia Mikolajczak, Andrew Turpin, Lea Frermann
  • TLDR: We propose a novel semi-supervised model for document-level frame prediction and interpretable article frame representation.
• Automatic Classification of Neutralization Techniques in the Narrative of Climate Change Scepticism
  • Shraey Bhatia, Jey Han Lau, Timothy Baldwin
  • TLDR: Neutralisation techniques, e.g. denial of responsibility and denial of victim, are used in the narrative of climate change scepticism to justify lack of action or to promote an alternative view.
• Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning
  • Ramit Sawhney, Harshit Joshi, Rajiv Ratn Shah, Lucie Flek
  • TLDR: We propose a framework jointly leveraging a user’s emotional history and social information from a user’s neighborhood in a network to contextualize the interpretation of the latest tweet of a user on Twitter.
• WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia
  • Kokil Jaidka, Andrea Ceolin, Iknoor Singh, Niyati Chhaya, Lyle Ungar
  • TLDR: We present a dataset of Wikipedia Talk Talk and edit histories for online cooperation and conversation modeling.
• The structure of online social networks modulates the rate of lexical change
  • Jian Zhu, David Jurgens
  • TLDR: We show that the structure of online communities is a significant factor in lexical change and show that topic-based communities are more likely to experience lexical leveling than offline communities.
• Modeling Framing in Immigration Discourse on Social Media
  • Julia Mendelsohn, Ceren Budak, David Jurgens
  • TLDR: We show how users’ ideology and region impact framing choices, and how a message’s framing influences audience responses.
• Modeling the Severity of Complaints in Social Media
  • Mali Jin, Nikolaos Aletras
  • TLDR: We study the severity level of complaints for the first time in computational linguistics.
• What About the Precedent: An Information-Theoretic Analysis of Common Law
  • Josef Valvoda, Tiago Pimentel, Niklas Stoehr, Ryan Cotterell, Simone Teufel
  • TLDR: We show that the arguments of the precedent cases are the main determinant of the outcome of a new case, rather than the facts of the case.
• Introducing CAD: the Contextual Abuse Dataset
  • Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, Rebekah Tromble
  • TLDR: We present a new dataset of primarily English Reddit entries which addresses several limitations of prior work.
• Lifelong Learning of Hate Speech Classification on Social Media
  • Jing Qian, Hong Wang, Mai ElSherief, Xifeng Yan
  • TLDR: We propose lifelong learning of hate speech classification on social media using variational representation learning and LB-SOINN memory modules.
• Learning to Recognize Dialect Features
  • Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, Jacob Eisenstein
  • TLDR: We present a novel approach to learning to recognize dialect features in speech and text using pretrained transformers and show that it can be as effective for training as thousands of labeled examples.
• It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
  • Timo Schick, Hinrich Schütze
  • TLDR: We show that language models can be trained to perform better than GPT-3 on text input and output, and exploit unlabeled data to improve performance.
• Static Embeddings as Efficient Knowledge Bases?
  • Philipp Dufter, Nora Kassner, Hinrich Schütze
  • TLDR: We show that static embeddings are better than PLMs for nearest neighbor matching and that BERT is a better model for semantic analysis.
• Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis
  • Xutan Peng, Guanyi Chen, Chenghua Lin, Mark Stevenson
  • TLDR: We present a novel and efficient knowledge graph embedding algorithm which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance.
• Rethinking Network Pruning – under the Pre-train and Fine-tune Paradigm
  • Dongkuan Xu, Ian En-Hsu Yen, Jinxi Zhao, Zhibin Xiao
  • TLDR: We propose a knowledge-aware sparse pruning method for BERT that outperforms existing literature on the basis of its knowledge transfer and loss.
• Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers
  • Andrew Silva, Pradyumna Tambwekar, Matthew Gombolay
  • TLDR: We investigate gender and racial bias across ubiquitous pre-trained language models, including GPT-2, XLNet, BERT, RoBERTa, ALBERT and DistilBERT.
• Detoxifying Language Models Risks Marginalizing Minority Voices
  • Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, Dan Klein
  • TLDR: We show that detoxification techniques for language models can decrease the utility of LMs on language used by marginalized groups.
• HONEST: Measuring Hurtful Sentence Completion in Language Models
  • Debora Nozza, Federico Bianchi, Dirk Hovy
  • TLDR: Language models capture and proliferate hurtful stereotypes, especially in text generation.
• EaSe: A Diagnostic Tool for VQA based on Answer Diversity
  • Shailza Jolly, Sandro Pezzelle, Moin Nabi
  • TLDR: We propose EASE, a simple diagnostic tool for Visual Question Answering (VQA) which quantifies the difficulty of an image, question sample.
• DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy Minimization
  • Zineng Tang, Jie Lei, Mohit Bansal
  • TLDR: We propose a novel video-and-language pre-training method that incorporates dense captions and entropy minimization to improve the performance of video- and language-and video-to-video models.
• Improving Generation and Evaluation of Visual Stories via Semantic Consistency
  • Adyasha Maharana, Darryl Hannan, Mohit Bansal
  • TLDR: We present a number of improvements to recurrent generative models for story visualization, including a novel model for sequential sequential visualization and a novel copy-transform mechanism for sequential generation of images.
• Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
  • Po-Yao Huang, Mandela Patrick, Junjie Hu, Graham Neubig, Florian Metze, Alexander Hauptmann
  • TLDR: We propose a Transformer-based model that learns contextual multilingual multimodal embeddings for multilingual text-to-video search and show that it significantly improves video search in non-English languages without additional annotations.
• Video Question Answering with Phrases via Semantic Roles
  • Arka Sadhu, Kan Chen, Ram Nevatia
  • TLDR: We present a new VidQA evaluation metric that poses VidQAs as a fill-in-the-phrase task.
• From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding
  • Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanović, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, Barbara Plank
  • TLDR: We propose a new benchmark for cross-lingual (x) Slot and Intent Detection in 13 languages from 6 language families, including a very low-resource dialect.
• WEC: Deriving a Large-scale Cross-document Event Coreference dataset from Wikipedia
  • Alon Eirew, Arie Cattan, Ido Dagan
  • TLDR: We present Wikipedia Event Coreference (WEC), an efficient methodology for gathering a large-scale dataset for cross-document event coreference from Wikipedia, where coreference links are not restricted within predefined topics.
• Challenging distributional models with a conceptual network of philosophical terms
  • Yvette Oortwijn, Jelke Bloem, Pia Sommerauer, Francois Meyer, Wei Zhou, Antske Fokkens
  • TLDR: We investigate the possibilities and limitations of using distributional semantic models for analyzing philosophical data by means of a realistic use-case.
• KILT: a Benchmark for Knowledge Intensive Language Tasks
  • Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
  • TLDR: We present a benchmark for knowledge-intensive language tasks and show that a seq2seq model is a strong baseline for knowledge intensive language tasks.
• A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios
  • Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow
  • TLDR: We present a structured overview of methods that enable learning when training data is sparse.
• Temporal Knowledge Graph Completion using a Linear Temporal Regularizer and Multivector Embeddings
  • Chengjin Xu, Yung-Yu Chen, Mojtaba Nayyeri, Jens Lehmann
  • TLDR: We present a novel time-aware knowledge graph embebdding approach, TeLM, which performs 4th-order tensor factorization of a Temporal knowledge graph using a Linear temporal regularizer and Multivector embeddings.
• UDALM: Unsupervised Domain Adaptation through Language Modeling
  • Constantinos Karouzos, Georgios Paraskevopoulos, Alexandros Potamianos
  • TLDR: We propose UDALM, a fine-tuning procedure for domain adaptation of pretrained language models, which can adapt to the target domain distribution in a robust and sample efficient manner.
• Beyond Black & White: Leveraging Annotator Disagreement via Soft-Label Multi-Task Learning
  • Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, Massimo Poesio
  • TLDR: We propose a novel method to incorporate disagreement as information in the prediction of soft-labels in a multi-task neural network.
• Clustering-based Inference for Biomedical Entity Linking
  • Rico Angell, Nicholas Monath, Sunil Mohan, Nishant Yadav, Andrew McCallum
  • TLDR: We propose a new entity linking model which learns to link mentions of unseen entities using learned representations of entities.
• Variance-reduced First-order Meta-learning for Natural Language Processing Tasks
  • Lingxiao Wang, Kevin Huang, Tengyu Ma, Quanquan Gu, Jing Huang
  • TLDR: We propose a novel variance reduction term to the gradient estimation used in first-order meta-learning to address the overfitting issue when we have heterogeneous tasks.
• Diversity-Aware Batch Active Learning for Dependency Parsing
  • Tianze Shi, Adrian Benton, Igor Malioutov, Ozan İrsoy
  • TLDR: We propose a diversity-aware strategy for training a strong dependency parser using batch active learning.
• How many data points is a prompt worth?
  • Teven Le Scao, Alexander Rush
  • TLDR: We show that prompting provides a benefit to training pretrained models for classification, and that this benefit can be quantified per task.
• Can Latent Alignments Improve Autoregressive Machine Translation?
  • Adi Haviv, Lior Vassertail, Omer Levy
  • TLDR: We show that latent alignment objectives are incompatible with teacher forcing, and show that autoregressive machine translation models are degenerate.
• Smoothing and Shrinking the Sparse Seq2Seq Search Space
  • Ben Peters, André F. T. Martins
  • TLDR: We show that entmax-based models effectively solve the cat got your tongue problem, removing a major source of model error for neural machine translation.
• Unified Pre-training for Program Understanding and Generation
  • Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang
  • TLDR: We present PLBART, a sequence-to-sequence model capable of performing a broad spectrum of program and language understanding and generation tasks.
• Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding
  • Ting Hua, Yilin Shen, Changsheng Zhao, Yen-Chang Hsu, Hongxia Jin
  • TLDR: We propose a hyperparameter-free continual learning model for domain classification that can stably produce high performance under various environments.
• On the Embeddings of Variables in Recurrent Neural Networks for Source Code
  • Nadezhda Chirkova
  • TLDR: We propose dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of a variable when it obtains more information about the variable’s role in the program.
• Cross-Lingual Word Embedding Refinement by \ell_{1} Norm Optimisation
  • Xutan Peng, Chenghua Lin, Mark Stevenson
  • TLDR: We propose a simple post-processing step to improve cross-lingual word embedding algorithms.
• Semantic Frame Forecast
  • Chieh-Yang Huang, Ting-Hao Huang
  • TLDR: We propose a novel task that predicts the semantic frames that will occur in the next 10, 100, or even 1,000 sentences in a long story.
• MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task
  • Yiqun Yao, Michalis Papakostas, Mihai Burzo, Mohamed Abouelenien, Rada Mihalcea
  • TLDR: We propose a novel multi-task learning algorithm for multimodal stress detection and demonstrate its effectiveness in both internal and external auxiliary tasks.
• Learning to Decompose and Organize Complex Tasks
  • Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen White, Dan Roth
  • TLDR: We propose a novel end-to-end pipeline that consumes a complex task and induces a dependency graph from unstructured text to represent sub-tasks and their relationships.
• Continual Learning for Text Classification with Information Disentanglement Based Regularization
  • Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, Diyi Yang
  • TLDR: We propose an information disentanglement based regularization method for continual learning on text classification.
• Learning from Executions for Semantic Parsing
  • Bailin Wang, Mirella Lapata, Ivan Titov
  • TLDR: We propose a new set of training objectives for semi-supervised semantic parsing that outperform conventional methods on Overnight and GeoQuery.
• Learning to Synthesize Data for Semantic Parsing
  • Bailin Wang, Wenpeng Yin, Xi Victoria Lin, Caiming Xiong
  • TLDR: We propose a generative model for semantic parsing that learns to synthesize data from existing data.
• Edge: Enriching Knowledge Graph Embeddings with External Text
  • Saed Rezayi, Handong Zhao, Sungchul Kim, Ryan Rossi, Nedim Lipka, Sheng Li
  • TLDR: We propose a novel knowledge graph enrichment and embedding framework based on external text and a novel graph alignment term.
• FLIN: A Flexible Natural Language Interface for Web Navigation
  • Sahisnu Mazumder, Oriana Riva
  • TLDR: We propose FLIN, a natural language interface for web navigation that maps user commands to concept-level actions (rather than low-level UI actions), thus being able to flexibly adapt to different websites and handle their transient nature.
• Game-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index
  • Roma Patel, Marta Garnelo, Ian Gemp, Chris Dyer, Yoram Bachrach
  • TLDR: We propose a vocabulary selection method that views words as members of a team trying to maximize the model’s performance.
• Incorporating External Knowledge to Enhance Tabular Reasoning
  • J. Neeraja, Vivek Gupta, Vivek Srikumar
  • TLDR: We propose easy and effective modifications to how information is presented to a model for tabular inference that substantially improve tabular reasoning.
• Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention
  • Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas
  • TLDR: We propose a span-level supervised attention loss that improves compositional generalization in semantic parsers.
• Domain Adaptation for Arabic Cross-Domain and Cross-Dialect Sentiment Analysis from Contextualized Word Embedding
  • Abdellah El Mekki, Abdelkader El Mahdaouy, Ismail Berrada, Ahmed Khoumsi
  • TLDR: Unsupervised domain adaptation for Arabic cross-domain and cross-dialect sentiment analysis.
• Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification
  • Andrew Moore, Jeremy Barnes
  • TLDR: We propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks, including negation and speculation scope detection, to create English-language models that are more robust to these phenomena.
• A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews
  • Gabriele Pergola, Lin Gui, Yulan He
  • TLDR: We propose a neural topic model combined with adversarial training to disentangle opinion topics from plot/neutral ones in movie and book reviews.
• Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification
  • Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, Bowen Zhou
  • TLDR: We propose a simple yet effective graph ensemble technique, GraphMerge, to make use of the predictions from different parsers to make efficient graph neural networks.
• Emotion-Infused Models for Explainable Psychological Stress Detection
  • Elsbeth Turcan, Smaranda Muresan, Kathleen McKeown
  • TLDR: We present work exploring the use of a semantically related task, emotion detection, for equally competent but more explainable and human-like psychological stress detection as compared to a black-box model.
• Aspect-based Sentiment Analysis with Type-aware Graph Convolutional Networks and Layer Ensemble
  • Yuanhe Tian, Guimin Chen, Yan Song
  • TLDR: We propose a novel approach to explicitly utilize dependency types for aspect-based sentiment analysis using type-aware graph convolutional networks.
• Supertagging-based Parsing with Linear Context-free Rewriting Systems
  • Thomas Ruprecht, Richard Mörbitz
  • TLDR: We present the first supertagging-based parser for linear context-free rewriting systems (LCFRS).
• Outside Computation with Superior Functions
  • Parker Riley, Daniel Gildea
  • TLDR: We show that a general algorithm for efficient computation of outside values under the minimum of superior functions framework proposed by Knuth (1977) would yield a sub-exponential time algorithm for SAT, violating the Strong Exponential Time Hypothesis (SETH).
• Learning Syntax from Naturally-Occurring Bracketings
  • Tianze Shi, Ozan İrsoy, Igor Malioutov, Lillian Lee
  • TLDR: We develop a partial-brackets-aware structured ramp loss in learning for constituency parsing that is more accurate than competing unsupervised systems.
• Bot-Adversarial Dialogue for Safe Conversational Agents
  • Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan
  • TLDR: We present a new framework for evaluating the toxicity of generative models and propose two novel methods for safe conversational agents.
• Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
  • Arun Babu, Akshat Shrivastava, Armen Aghajanyan, Ahmed Aly, Angela Fan, Marjan Ghazvininejad
  • TLDR: We propose a novel sequence-to-sequence model architecture for semantic parsing using convolutional neural networks and non-autoregressive prediction.
• Example-Driven Intent Prediction with Observers
  • Shikib Mehri, Mihail Eric
  • TLDR: We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training.
• Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management
  • Zhengxu Hou, Bang Liu, Ruihui Zhao, Zijing Ou, Yafei Liu, Xi Chen, Yefeng Zheng
  • TLDR: We propose a multi-level reward modeling approach that factorizes a reward into a three-level hierarchy: domain, act, and slot.
• Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems
  • Derek Chen, Howard Chen, Yi Yang, Alexander Lin, Zhou Yu
  • TLDR: We present a goal-oriented dialogue dataset with over 10K human-to-human dialogues containing 55 distinct user intents requiring unique sequences of actions constrained by policies to achieve task success.
• Controlling Dialogue Generation with Semantic Exemplars
  • Prakhar Gupta, Jeffrey Bigham, Yulia Tsvetkov, Amy Pavel
  • TLDR: We present an Exemplar-based Dialogue Generation model that uses the semantic frames present in exemplar responses to guide response generation.
• COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
  • Luyu Gao, Zhuyun Dai, Jamie Callan
  • TLDR: We present a contextualized exact match retrieval architecture based on overlapping query document tokens’ contextualized representations.
• X-Class: Text Classification with Extremely Weak Supervision
  • Zihan Wang, Dheeraj Mekala, Jingbo Shang
  • TLDR: We propose a novel framework X-Class to learn text classification with extremely weak supervision, i.e., only relying on the surface text of class names.
• Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling
  • Aaron Mueller, Mark Dredze
  • TLDR: We propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual neural topic modeling.
• Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification
  • Wilson Fearn, Orion Weller, Kevin Seppi
  • TLDR: We provide a comprehensive study that examines how preprocessing techniques affect the vocabulary size, model performance, and model run-time, evaluating ten techniques over four models and two datasets.
• Faithfully Explainable Recommendation via Neural Logic Reasoning
  • Yaxin Zhu, Yikun Xian, Zuohui Fu, Gerard de Melo, Yongfeng Zhang
  • TLDR: We propose neural logic reasoning for explainable recommendation (LOGER) by drawing on interpretable logical rules to guide the path-reasoning process for explanation generation.
• You Sound Like Someone Who Watches Drama Movies: Towards Predicting Movie Preferences from Conversational Interactions
  • Sergey Volokhin, Joyce Ho, Oleg Rokhlenko, Eugene Agichtein
  • TLDR: We propose a novel method for conversational recommendation based on the input of a user’s sentiment and external reviewers, and show that it can improve the accuracy of predicting users’ ratings for new movies by exploiting conversation content and external data.
• Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents
  • Shunyu Yao, Karthik Narasimhan, Matthew Hausknecht
  • TLDR: We show that game text-based text-driven agents can learn to understand and leverage game text without language semantics.
• SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
  • Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Parikh, Ramprasaath R. Selvaraju
  • TLDR: We present a new approach to improve visual question answering by improving model consistency and improving visual grounding.
• Semi-Supervised Policy Initialization for Playing Games with Language Hints
  • Tsu-Jui Fu, William Yang Wang
  • TLDR: We propose semi-supervised initialization that allows the agent to learn from various possible hints before training under different tasks.
• Revisiting Document Representations for Large-Scale Zero-Shot Learning
  • Jihyung Kil, Wei-Lun Chao
  • TLDR: We propose a novel way to extract visual sentence representations from Wikipedia pages that can be used for zero-shot learning.
• Negative language transfer in learner English: A new dataset
  • Leticia Farias Wanderley, Nicole Zhao, Carrie Demmans Epp
  • TLDR: Automatic personalized corrective feedback can help language learners from different backgrounds better acquire a new language.
• SentSim: Crosslingual Semantic Evaluation of Machine Translation
  • Yurun Song, Junchen Zhao, Lucia Specia
  • TLDR: We propose a new metric for evaluating machine translation that uses strong pretrained multilingual word and sentence representations to compare the source with the machine translated sentence, thus avoiding the need for both reference translations and labelled training data.
• Quality Estimation for Image Captions Based on Large-scale Human Evaluations
  • Tomer Levinboim, Ashish V. Thapliyal, Piyush Sharma, Radu Soricut
  • TLDR: We propose a new caption quality estimation task that uses crowdsourced caption quality ratings to detect and filter out low-quality image captions.
• CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems
  • Kushal Chawla, Jaysa Ramirez, Rene Clever, Gale Lucas, Jonathan May, Jonathan Gratch
  • TLDR: We present a novel corpus of over a thousand negotiation dialogues in English that can be used to train a system to negotiate with humans.
• News Headline Grouping as a Challenging NLU Task
  • Philippe Laban, Lucas Bandarkar, Marti A. Hearst
  • TLDR: We propose a novel unsupervised Headline Generator Swap model for the task of HeadLine Grouping and a corresponding dataset (HLGD) consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group.
• Olá, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer
  • Eleftheria Briakou, Di Lu, Ke Zhang, Joel Tetreault
  • TLDR: We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian.
• Grouping Words with Semantic Diversity
  • Karine Chubarian, Abdul Rafae Khan, Anastasios Sidiropoulos, Jia Xu
  • TLDR: We propose a new approach to group input words based on their semantic diversity to simplify input language representation with low ambiguity.
• Noise Stability Regularization for Improving BERT Fine-tuning
  • Hang Hua, Xingjian Li, Dejing Dou, Chengzhong Xu, Jiebo Luo
  • TLDR: We propose a novel and effective regulariza-tion method to improve fine-tuning on NLP tasks, referred to as Layer-wiseNoiseStabilityRegularization (LNSR).
• FlowPrior: Learning Expressive Priors for Latent Variable Sentence Models
  • Xiaoan Ding, Kevin Gimpel
  • TLDR: We propose a new VAE model that learns expressive priors over the latent variable and show substantial improvement in language modeling tasks compared to strong baselines.
• HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization
  • Zhongfen Deng, Hao Peng, Dongxiao He, Jianxin Li, Philip Yu
  • TLDR: We propose HTCInfoMax, a hierarchical text classification model which incorporates information maximization and label prior matching to address label imbalance in hierarchical text classifier.
• Knowledge Guided Metric Learning for Few-Shot Text Classification
  • Dianbo Sui, Yubo Chen, Binjie Mao, Delai Qiu, Kang Liu, Jun Zhao
  • TLDR: We propose to introduce external knowledge into few-shot learning to imitate human knowledge.
• Ensemble of MRR and NDCG models for Visual Dialog
  • Idan Schwartz
  • TLDR: We propose a novel non-parametric ranking approach that can combine strong MRR and NDCG metrics and achieve state-of-the-art performance on both metrics.
• Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents
  • Iryna Haponchyk, Alessandro Moschitti
  • TLDR: We propose a new method for supervised clustering based on latent structured prediction loss and Transformer models.
• ConVEx: Data-Efficient and Few-Shot Slot Labeling
  • Matthew Henderson, Ivan Vulić
  • TLDR: We propose ConVEx, a novel pretraining and fine-tuning neural approach for slot-labeling dialog tasks.
• CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues
  • Bo-Hsiang Tseng, Shruti Bhargava, Jiarui Lu, Joel Ruben Antony Moniz, Dhivya Piraviperumal, Lin Li, Hong Yu
  • TLDR: We propose a novel joint learning framework for modeling coreference resolution and query rewriting for multi-turn dialogue understanding.
• Knowledge-Driven Slot Constraints for Goal-Oriented Dialogue Systems
  • Piyawat Lertvittayakumjorn, Daniele Bonadiman, Saab Mansour
  • TLDR: We propose a new task of constraint violation detection in goal-oriented dialogue systems and show that some combinations of slot values can be invalid according to external knowledge.
• Clipping Loops for Sample-Efficient Dialogue Policy Optimisation
  • Yen-Chen Wu, Carl Edward Rasmussen
  • TLDR: We propose loop-clipping policy optimisation to eliminate useless responses from dialogue history.
• Integrating Lexical Information into Entity Neighbourhood Representations for Relation Prediction
  • Ian Wood, Mark Johnson, Stephen Wan
  • TLDR: We present a novel approach to relation prediction based on text-based representations of entities and predicates extracted from a document corpus.
• Noisy-Labeled NER with Confidence Estimation
  • Kun Liu, Yao Fu, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang, Sheng Gao
  • TLDR: We propose a method for estimating confidence scores for named entity recognition under noisy labeled settings with calibrated confidence estimation.
• TABBIE: Pretrained Representations of Tabular Data
  • Hiroshi Iida, Dung Thai, Varun Manjunatha, Mohit Iyyer
  • TLDR: We present a novel method for tabular representation-learning that learns to model tables and associated text jointly using only tabular data.
• Better Feature Integration for Named Entity Recognition
  • Lu Xu, Zhanming Jie, Wei Lu, Lidong Bing
  • TLDR: Synergized-LSTM for named entity recognition with long-distance structured information captured by dependency trees.
• ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning
  • Chih-Yao Chen, Cheng-Te Li
  • TLDR: We propose a novel multi-task learning model, Zero-Shot BERT (ZS-BERT), to directly predict unseen relations without hand-crafted attribute labeling and multiple pairwise classifications.
• Graph Convolutional Networks for Event Causality Identification with Rich Document-level Structures
  • Minh Tran Phu, Thien Huu Nguyen
  • TLDR: We propose a graph-based model for document-level event causality identification that learns document context-augmented representations for causality prediction between event mention pairs in text.
• A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution
  • Tuan Lai, Heng Ji, Trung Bui, Quan Hung Tran, Franck Dernoncourt, Walter Chang
  • TLDR: We propose a novel context-dependent gated module to adaptively control the information flows from the input symbolic features for event coreference resolution.
• Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus
  • Navita Goyal, Balaji Vasan Srinivasan, Anandhavelu N, Abhilasha Sancheti
  • TLDR: We present a new style transfer model that can control styles across multiple style dimensions without any additional annotations.
• FUDGE: Controlled Text Generation With Future Discriminators
  • Kevin Yang, Dan Klein
  • TLDR: We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation.
• Controllable Text Simplification with Explicit Paraphrasing
  • Mounica Maddela, Fernando Alva-Manchego, Wei Xu
  • TLDR: We propose a novel hybrid approach to text simplification that combines linguistically-motivated rules for splitting and deletion with a neural paraphrasing model to produce varied rewriting styles.
• Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training
  • Oshin Agarwal, Heming Ge, Siamak Shakeri, Rami Al-Rfou
  • TLDR: We verbalize Wikidata KG triples into natural text and show that this approach can be used to integrate structured KGs and natural language corpora.
• Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models
  • Elizabeth Clark, Noah A. Smith
  • TLDR: We present Choose Your Own Adventure, a collaborative writing setup for pairwise model evaluation.
• InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
  • Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou
  • TLDR: We propose a unified view of cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
• Context-Interactive Pre-Training for Document Machine Translation
  • Pengcheng Yang, Pei Zhang, Boxing Chen, Jun Xie, Weihua Luo
  • TLDR: We propose a simple yet effective context-interactive pre-training approach for document machine translation, which can significantly improve the performance of existing document machine translations.
• Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
  • Samson Tan, Shafiq Joty
  • TLDR: We present two strong black-box adversarial attacks for multilingual models that push their ability to handle code-mixed sentences to the limit.
• X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering
  • Meryem M’hamdi, Doo Soon Kim, Franck Dernoncourt, Trung Bui, Xiang Ren, Jonathan May
  • TLDR: We propose X-METRA-ADA, a cross-lingual MEta-TRAnsfer learning ADAptation approach for NLU.
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders
  • Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, Graham Neubig
  • TLDR: We present a new method for learning multilingual encoders that can be used for transfer-learning of NLP systems from high-resource languages to low-resource language.
• Cross-lingual Cross-modal Pretraining for Multimodal Retrieval
  • Hongliang Fei, Tan Yu, Ping Li
  • TLDR: We propose a novel approach to learn cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages.
• Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks
  • Iacer Calixto, Alessandro Raganato, Tommaso Pasini
  • TLDR: We propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap across different languages by means of a shared vocabulary of entities.
• multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning
  • Swarnadeep Saha, Prateek Yadav, Mohit Bansal
  • TLDR: We propose a new proof-set generation model for compositional reasoning over natural language rule-bases that can learn from multiple proofs for better interpretability of such reasoning systems.
• Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge
  • Pat Verga, Haitian Sun, Livio Baldini Soares, William Cohen
  • TLDR: We develop a neural language model that includes an interpretable neuro-symbolic KB in the form of a “fact memory”.
• CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images
  • Shailaja Keyur Sampat, Akshay Kumar, Yezhou Yang, Chitta Baral
  • TLDR: We present a new visual question answering task that involves mentally simulating the hypothetical consequences of performing specific actions in a given scenario.
• Refining Targeted Syntactic Evaluation of Language Models
  • Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt
  • TLDR: We present new metrics for targeted syntactic evaluation of subject-verb number agreement in English that capture both systematicity and likely behavior of language models.
• Universal Adversarial Attacks with Natural Triggers for Text Classification
  • Liwei Song, Xinwei Yu, Hsuan-Tung Peng, Karthik Narasimhan
  • TLDR: We present novel adversarial attacks that appear closer to natural English phrases and yet confuse classification systems when added to benign inputs.
• QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval
  • Peiyang Liu, Sen Wang, Xi Wang, Wei Ye, Shikun Zhang
  • TLDR: We present a new BERT-style retrieval model for embedding-based large-scale query-document retrieval tasks.
• Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack
  • Liwen Wang, Yuanmeng Yan, Keqing He, Yanan Wu, Weiran Xu
  • TLDR: We propose a novel adversarial disentangled debiasing method for text representation learning that dynamically decouples social bias attributes from intermediate representations trained on the downstream task.
• An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls
  • Ramit Sawhney, Arshiya Aggarwal, Rajiv Ratn Shah
  • TLDR: We present the first study to discover the gender bias in multimodal neural financial models due to gender-sensitive audio features and fewer female executives in earnings calls of one of the world’s biggest stock indexes, the S&P 500 index.
• Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing
  • Boaz Shmueli, Jan Fell, Soumya Ray, Lun-Wei Ku
  • TLDR: We present ethical considerations for the use of crowdworkers in NLP research and provide a framework for evaluating the ethical use of crowdsourcing.
• On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
  • Xisen Jin, Francesco Barbieri, Brendan Kennedy, Aida Mostafazadeh Davani, Leonardo Neves, Xiang Ren
  • TLDR: We show that the effects of downstream bias mitigation on language model fine-tuning are transferable to new downstream tasks and domains.
• Case Study: Deontological Ethics in NLP
  • Shrimai Prabhumoye, Brendon Boldt, Ruslan Salakhutdinov, Alan W Black
  • TLDR: We study ethical foundations of natural language processing and show how these principles can be used to improve the ethical foundations and avoid ethical issues in NLP systems.
• Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels
  • Fatemehsadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
  • TLDR: We present two privacy-preserving regularization methods for neural language models that enable joint optimization of utility and privacy through (1) the use of a discriminator and (2) the inclusion of a novel triplet-loss term.
• On the Impact of Random Seeds on the Fairness of Clinical Classifiers
  • Silvio Amir, Jan-Willem van de Meent, Byron Wallace
  • TLDR: We show that the observed subgroup performance in MIMIC-III is surprisingly sensitive to changes in random seed(s) and show that we can estimate disparities in fairness by looking at intersections of minority groups and somewhat rare conditions.
• Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures
  • Caitlin Doogan, Wray Buntine
  • TLDR: We propose four understandings of interpretability of topic models and evaluate their generalizability in an applied setting.
• Discourse Probing of Pretrained Language Models
  • Fajri Koto, Jey Han Lau, Timothy Baldwin
  • TLDR: We present a new document-level discourse probing task for pretrained language models and show that the best models capture discourse information in their encoder and encode it in the encoder.
• UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
  • Zhen Wu, Lijun Wu, Qi Meng, Yingce Xia, Shufang Xie, Tao Qin, Xinyu Dai, Tie-Yan Liu
  • TLDR: We propose a new dropout technique for Transformer architecture that improves performance on neural machine translation and text classification tasks.
• tWT–WT: A Dataset to Assert the Role of Target Entities for Detecting Stance of Tweets
  • Ayush Kaushal, Avirup Saha, Niloy Ganguly
  • TLDR: We show that it is possible to detect the stance of a tweet or a text for a target without looking at the target sentence.
• Learning to Learn to be Right for the Right Reasons
  • Pride Kavumba, Benjamin Heinzerling, Ana Brassard, Kentaro Inui
  • TLDR: Meta-learning to improve generalization on held-out data.
• Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
  • Chong Zhang, Jieyu Zhao, Huan Zhang, Kai-Wei Chang, Cho-Jui Hsieh
  • TLDR: We propose a novel method to uncover model weaknesses beyond the test dataset.
• Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
  • Hanjie Chen, Song Feng, Jatin Ganhotra, Hui Wan, Chulaka Gunasekara, Sachindra Joshi, Yangfeng Ji
  • TLDR: We propose a novel method for generating post-hoc explanations for neural network models by grouping correlated words from the input text pair together and measure their contribution to the corresponding NLP tasks as a whole.
• Almost Free Semantic Draft for Neural Machine Translation
  • Xi Ai, Bin Fang
  • TLDR: We present an efficient method to sample and consider a semantic draft as global information from semantic space for decoding with almost free of cost.
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation
  • Shuhao Gu, Yang Feng, Wanying Xie
  • TLDR: We propose a method of domain adaptation based on the importance of neurons or parameters for the translation model.
• Multi-Hop Transformer for Document-Level Machine Translation
  • Long Zhang, Tong Zhang, Haibo Zhang, Baosong Yang, Wei Ye, Shikun Zhang
  • TLDR: We propose a novel Multi-Hop Transformer for document-level neural machine translation which can tackle discourse phenomena, such as coreference error and the problem of polysemy.
• Continual Learning for Neural Machine Translation
  • Yue Cao, Hao-Ran Wei, Boxing Chen, Xiaojun Wan
  • TLDR: We propose a new continual learning framework for neural machine translation models that alleviates catastrophic forgetting and improve performance in all settings.
• Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios
  • Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao
  • TLDR: We propose self-training mechanisms for unsupervised neural machine translation that can improve the performance of existing UNMT systems when there is not adequate training corpus for one language.
• Smart-Start Decoding for Neural Machine Translation
  • Jian Yang, Shuming Ma, Dongdong Zhang, Juncheng Wan, Zhoujun Li, Ming Zhou
  • TLDR: We propose a novel method for neural machine translation that breaks up the limitation of these monotonic decoding orders, called Smart-Start decoding.
• Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation
  • Yongchang Hao, Shilin He, Wenxiang Jiao, Zhaopeng Tu, Michael Lyu, Xing Wang
  • TLDR: We propose to adopt multi-task learning to transfer the Autoregressive machine Translation (AT) knowledge to NAT models through encoder sharing.
• ER-AE: Differentially Private Text Generation for Authorship Anonymization
  • Haohan Bo, Steven H. H. Ding, Benjamin C. M. Fung, Farkhund Iqbal
  • TLDR: We propose a novel text generation model with a two-set exponential mechanism for authorship anonymization.
• Distantly Supervised Transformers For E-Commerce Product QA
  • Happy Mittal, Aniket Chakrabarti, Belhassen Bayar, Animesh Anant Sharma, Nikhil Rasiwasia
  • TLDR: We propose a practical instant question answering system on product pages of e-commerce services, where for each user query, relevant community question answer (CQA) pairs are retrieved.
• Quantitative Day Trading from Natural Language using Reinforcement Learning
  • Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah
  • TLDR: We propose a novel NLP approach that makes time-aware decisions to trade stocks while optimizing profit using textual data.
• Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation
  • Kyeongpil Kang, Kyohoon Jin, Soyoung Yang, Soojin Jang, Jaegul Choo, Youngbin Kim
  • TLDR: We present a multi-task learning approach to restore and translate historical documents based on a self-attention mechanism, specifically utilizing two Korean historical records, ones of the most voluminous historical records in the world.
• Modeling Diagnostic Label Correlation for Automatic ICD Coding
  • Shang-Chi Tsai, Chao-Wei Huang, Yun-Nung Chen
  • TLDR: We propose a two-stage framework to improve automatic ICD coding by capturing the label correlation.
• Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents
  • Mohammad Kachuee, Hao Yuan, Young-Bum Kim, Sungjin Lee
  • TLDR: We propose a novel few-shot transfer learning approach that improves the transferability of pre-trained models in conversational agents and improves the generalization on unseen skills.
• A recipe for annotating grounded clarifications
  • Luciana Benotti, Patrick Blackburn
  • TLDR: Dialogue clarification mechanisms make explicit the process of interpreting the communicative intents of the speaker’s utterances by grounding them in the various modalities in which the dialogue is situated.
• Grey-box Adversarial Attack And Defence For Sentiment Classification
  • Ying Xu, Xu Zhong, Antonio Jimeno Yepes, Jey Han Lau
  • TLDR: We introduce a grey-box adversarial attack and defence framework for sentiment classification.
• How low is too low? A monolingual take on lemmatisation in Indian languages
  • Kumar Saunack, Kumar Saurav, Pushpak Bhattacharyya
  • TLDR: We show that monolingual approaches with data augmentation can give competitive accuracy even in the low resource setting, which augurs well for NLP in low resource.
• Causal Effects of Linguistic Properties
  • Reid Pryzant, Dallas Card, Dan Jurafsky, Victor Veitch, Dhanya Sridhar
  • TLDR: We propose a novel algorithm for estimating the causal effects of linguistic properties of interest from observational data.
• Dynabench: Rethinking Benchmarking in NLP
  • Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams
  • TLDR: We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking.
• Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research
  • Denis Newman-Griffis, Jill Fain Lehman, Carolyn Rosé, Harry Hochheiser
  • TLDR: We present a new paradigm for Translational NLP research, which aims to structure and facilitate the processes by which basic and applied NLP knowledge exchange is facilitated.
• Predicting Discourse Trees from Transformer-based Neural Summarizers
  • Wen Xiao, Patrick Huber, Giuseppe Carenini
  • TLDR: We show that discourse information learned from pre-trained neural summarizers can be used to infer document-level discourse trees from pregenerated self-attention matrices.
• Probing for Bridging Inference in Transformer Language Models
  • Onkar Pandit, Yufang Hou
  • TLDR: We probe pre-trained transformer language models for bridging inference and show that bridging anaphora resolution is substantially captured by pre-training language models.
• Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models
  • Anne Beyer, Sharid Loáiciga, David Schlangen
  • TLDR: Extendable set of test suites addressing different aspects of discourse and dialogue coherence and their role in neural language models.
• Stay Together: A System for Single and Split-antecedent Anaphora Resolution
  • Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Massimo Poesio
  • TLDR: We propose a new system for resolving both single and split-antecedent anaphora, and evaluate it in a realistic setting that uses predicted mentions.
• Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness
  • Florian Boudin, Ygor Gallina
  • TLDR: We show that the commonly drawn distinction between present and absent keyphrases is not made explicit enough, and show that this small fraction of words is behind much of the gains observed in retrieval effectiveness.
• CoRT: Complementary Rankings from Transformers
  • Marco Wrzalik, Dirk Krechel
  • TLDR: We propose a novel first-stage neural first-level ranking model that leverages contextual representations from pretrained language models such as BERT to complement term-based ranking functions while causing no significant delay at query time.
• Multi-source Neural Topic Modeling in Multi-view Embedding Spaces
  • Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze
  • TLDR: We present a novel neural topic modeling framework using multi-view embed ding spaces: (1) pretrained topic-embeddings, and (2) pret trained word-embedding (context-insensitive from Glove and context-sensitive from BERT models) jointly from one or many sources to improve topic quality and better deal with polysemy.
• Inductive Topic Variational Graph Auto-Encoder for Text Classification
  • Qianqian Xie, Jimin Huang, Pan Du, Min Peng, Jian-Yun Nie
  • TLDR: We propose a novel model named inductive Topic Variational Graph Auto-Encoder (T-VGAE), which incorporates a topic model into variational graph-auto-encoder (VGAEs) to capture the hidden semantic information between documents and words.
• Self-Alignment Pretraining for Biomedical Entity Representations
  • Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier
  • TLDR: We propose a novel biomedical entity-level pretraining scheme that self-aligns biomedical entities and achieve state-of-the-art results on six MEL benchmarking datasets.
• TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names
  • Jiaming Shen, Wenda Qiu, Yu Meng, Jingbo Shang, Xiang Ren, Jiawei Han
  • TLDR: We propose a novel HMTC framework that uses only class surface names as supervision signals to perform hierarchical multi-label text classification.
• MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding
  • Tuhin Chakrabarty, Xurui Zhang, Smaranda Muresan, Nanyun Peng
  • TLDR: We propose a method to generate a metaphoric sentence given a literal expression by replacing relevant verbs with a sequence of metaphors.
• On Learning Text Style Transfer with Direct Rewards
  • Yixin Liu, Graham Neubig, John Wieting
  • TLDR: We propose a novel approach to train text style transfer models that explicitly assess the preservation of content between system outputs and input texts.
• Focused Attention Improves Document-Grounded Generation
  • Shrimai Prabhumoye, Kazuma Hashimoto, Yingbo Zhou, Alan W Black, Ruslan Salakhutdinov
  • TLDR: We propose novel methods for document grounded generation based on context driven representation and human evaluation.
• NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
  • Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
  • TLDR: We propose NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models – supervised or not – to generate fluent text while satisfying complex lexical constraints.
• Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge
  • Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, Julian McAuley
  • TLDR: We propose a model for clarification question generation where we first identify what is missing by taking a difference between the global and the local view and then train a model to identify what useful and generate a question about it.
• Progressive Generation of Long Text with Pretrained Language Models
  • Bowen Tan, Zichao Yang, Maruan Al-Shedivat, Eric Xing, Zhiting Hu
  • TLDR: We propose a progressive method of generating long passages of text in a progressive manner, inspired by generating images from low to high resolution.
• SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain
  • Ruochen Zhang, Carsten Eickhoff
  • TLDR: We present a new task for tracking state changes in sports commentary accompanied by discrete events and show that even sophisticated existing methods struggle on the state tracking task when the definition of state broadens or non-event chatter becomes prevalent.
• Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation
  • Sarik Ghazarian, Zixi Liu, Akash S M, Ralph Weischedel, Aram Galstyan, Nanyun Peng
  • TLDR: We present a new approach to learnable evaluation metrics for open-domain story generation.
• MultiOpEd: A Corpus of Multi-Perspective News Editorials
  • Siyi Liu, Sihao Chen, Xander Uyttendaele, Dan Roth
  • TLDR: We propose MultiOpEd, an open-domain news editorial corpus that supports various tasks pertaining to the argumentation structure in news editorials, focusing on automatic perspective discovery.
• Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality
  • Mina Lee, Chris Donahue, Robin Jia, Alexander Iyabor, Percy Liang
  • TLDR: We present a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context.
• “I’m Not Mad”: Commonsense Implications of Negation and Contradiction
  • Liwei Jiang, Antoine Bosselut, Chandra Bhagavatula, Yejin Choi
  • TLDR: We present the first comprehensive study focusing on commonsense implications of negated statements and contradictions.
• Identifying Medical Self-Disclosure in Online Communities
  • Mina Valizadeh, Pardis Ranjbar-Noiey, Cornelia Caragea, Natalie Parde
  • TLDR: We present a new dataset of health-related posts collected from online social platforms, categorized into three groups (No Self-Disclosure, Possible Self-Defclosure, and Clear Self-Conflicting Posts) with high inter-annotator agreement.
• Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction
  • Federico Bianchi, Ciro Greco, Jacopo Tagliabue
  • TLDR: We investigate grounded language learning through real-world data, by modelling a teacher-learner dynamics through the natural interactions occurring between users and search engines; in particular, we explore the emergence of semantic generalization from unsupervised dense representations outside of synthetic environments.
• Finding Concept-specific Biases in Form–Meaning Associations
  • Tiago Pimentel, Brian Roark, Søren Wichmann, Ryan Cotterell, Damián Blasi
  • TLDR: We present an information-theoretic operationalisation of cross-linguistic non-arbitrariness, and show that it is not a significant effect of language non-arrrforcement.
• How (Non-)Optimal is the Lexicon?
  • Tiago Pimentel, Irene Nikkarinen, Kyle Mahowald, Ryan Cotterell, Damián Blasi
  • TLDR: We show that morphology and graphotactics can sufficiently account for most of the complexity of natural codes, but that usage pressures and local constraints on sequences of symbols can significantly reduce the capacity of natural languages.
• Word Complexity is in the Eye of the Beholder
  • Sian Gooding, Ekaterina Kochmar, Seid Muhie Yimam, Chris Biemann
  • TLDR: We investigate which aspects contribute to the notion of lexical complexity in various groups of readers, focusing on native and non-native speakers of English, and how the notion changes depending on the proficiency level of a non-Native reader.
• Linguistic Complexity Loss in Text-Based Therapy
  • Jason Wei, Kelly Finn, Emma Templeton, Thalia Wheatley, Soroush Vosoughi
  • TLDR: We analyze the complexity loss paradox in online text-based therapy and show that it can be leveraged as a marker for anxiety.
• Ab Antiquo: Neural Proto-language Reconstruction
  • Carlo Meloni, Shauli Ravfogel, Yoav Goldberg
  • TLDR: We present a novel comparative method for proto-word reconstruction based on observed forms in daughter languages and show that neural sequence models outperform conventional methods applied to this task so far.
• On Biasing Transformer Attention Towards Monotonicity
  • Annette Rios, Chantal Amrhein, Noëmi Aepli, Rico Sennrich
  • TLDR: We introduce a monotonicity loss function that is compatible with standard attention mechanisms and test it on several sequence-to-sequence tasks: grapheme-tophoneme conversion, morphological inflection, transliteration, and dialect normalization.
• Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
  • Tom Hope, Aida Amini, David Wadden, Madeleine van Zuylen, Sravanthi Parasa, Eric Horvitz, Daniel Weld, Roy Schwartz, Hannaneh Hajishirzi
  • TLDR: We propose a new knowledge base of mechanisms, a unified schema for scientific literature that can be used to facilitate interdisciplinary scientific search over COVID-19 literature.
• Constrained Multi-Task Learning for Event Coreference Resolution
  • Jing Lu, Vincent Ng
  • TLDR: We propose a neural event coreference model in which event coreferences are jointly trained with five tasks: trigger detection, entity coreference, anaphoricity determination, realis detection, real is detection, and argument extraction.
• Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality
  • Adithya V Ganesan, Matthew Matero, Aravind Reddy Ravula, Huy Vu, H. Andrew Schwartz
  • TLDR: We provide a systematic study on the role of dimension reduction methods (principal components analysis, factorization techniques, or multi-layer auto-encoders) as well as the dimensionality of embedding vectors and sample sizes as a function of predictive performance.
• Leveraging Deep Representations of Radiology Reports in Survival Analysis for Predicting Heart Failure Patient Mortality
  • Hyun Gi Lee, Evan Sholle, Ashley Beecy, Subhi Al’Aref, Yifan Peng
  • TLDR: We present a novel method of using BERT-based hidden layer representations of clinical texts as covariates for proportional hazards models to predict patient survival outcomes.
• On the Use of Context for Predicting Citation Worthiness of Sentences in Scholarly Articles
  • Rakesh Gosangi, Ravneet Arora, Mohsen Gheisarieha, Debanjan Mahata, Haimin Zhang
  • TLDR: We study the importance of context in predicting the citation worthiness of sentences in scholarly articles.
• Data and Model Distillation as a Solution for Domain-transferable Fact Verification
  • Mitch Paul Mithun, Sandeep Suntwal, Mihai Surdeanu
  • TLDR: We present a combination of two strategies to mitigate this dependence on lexicalized information in fact verification tasks.
• Adapting Coreference Resolution for Processing Violent Death Narratives
  • Ankith Uppunda, Susan Cochran, Jacob Foster, Alina Arseniev-Koehler, Vickie Mays, Kai-Wei Chang
  • TLDR: We propose a probabilistic data augmentation algorithm for training coreference models that can better handle text data about LGBT individuals.
• Time-Stamped Language Model: Teaching Language Models to Understand The Flow of Events
  • Hossein Rajaby Faghihi, Parisa Kordjamshidi
  • TLDR: We propose a Time-Stamped Language Model for procedural text understanding.
• If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering
  • Vikas Yadav, Steven Bethard, Mihai Surdeanu
  • TLDR: We propose a simple approach that retrieves and reranks set of evidence facts jointly and show that it improves multi-hop reasoning performance on two multi-task question answering datasets.
• SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning
  • Roshanak Mirzaee, Hossein Rajaby Faghihi, Qiang Ning, Parisa Kordjamshidi
  • TLDR: We propose a question-answering benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models.
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
  • Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
  • TLDR: We present a dataset of 5049 questions over 1585 Natural Language Processing papers that are designed to answer document-based, information-seeking question answering questions.
• Differentiable Open-Ended Commonsense Reasoning
  • Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang Ren, William Cohen
  • TLDR: We propose a new commonsense reasoning method that uses commonsense facts to answer commonsense questions without any pre-defined choices.
• Does Structure Matter? Encoding Documents for Machine Reading Comprehension
  • Hui Wan, Song Feng, Chulaka Gunasekara, Siva Sankalp Patel, Sachindra Joshi, Luis Lastras
  • TLDR: We propose a new Transformer-based method that reads a document as tree slices and outperforms several competitive baseline approaches on two datasets from varied domains.
• Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval
  • Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, Hal Daumé III
  • TLDR: We propose a new multi-step retrieval approach that iteratively forms an evidence chain through beam search in dense representations.
• Scalable and Interpretable Semantic Change Detection
  • Syrielle Montariol, Matej Martinc, Lidia Pivovarova
  • TLDR: We propose a scalable method for semantic change detection with contextual embeddings that offers large gains in processing time and significant memory savings while offering the same interpretability and better performance than unscalable methods.
• Scalar Adjective Identification and Multilingual Ranking
  • Aina Garí Soler, Marianna Apidianaki
  • TLDR: We present a new multilingual dataset for evaluating the intensity relationship between scalar adjectives and their contextual representations.
• ESC: Redesigning WSD with Extractive Sense Comprehension
  • Edoardo Barba, Tommaso Pasini, Roberto Navigli
  • TLDR: We propose a transformer-based neural architecture for word sense disambiguation and show that it can outperform all existing models on the English WSD task.
• Recent advances in neural metaphor processing: A linguistic, cognitive and social perspective
  • Xiaoyu Tong, Ekaterina Shutova, Martha Lewis
  • TLDR: We provide a comprehensive review and discussion of recent developments in automated metaphor processing, in light of the findings about metaphor in the mind, language, and communication, and from the perspective of downstream NLP tasks.
• Constructing Taxonomies from Pretrained Language Models
  • Catherine Chen, Kevin Lin, Dan Klein
  • TLDR: We present a method for constructing taxonomic trees (e.g., WordNet) using pretrained language models.
• Event Representation with Sequential, Semi-Supervised Discrete Variables
  • Mehdi Rezaee, Francis Ferraro
  • TLDR: We propose a new method for neural sequence modeling that takes partially-observed sequences of discrete, external knowledge into account.
• Seq2Emo: A Sequence to Multi-Label Emotion Classification Model
  • Chenyang Huang, Amine Trabelsi, Xuebin Qin, Nawshad Farruque, Lili Mou, Osmar Zaïane
  • TLDR: We propose a sequence-to-emotion approach for multi-label emotion classification, which implicitly models emotion correlations in a bi-directional decoder.
• Knowledge Enhanced Masked Language Model for Stance Detection
  • Kornraphop Kawintiranon, Lisa Singh
  • TLDR: We propose a novel BERT-based fine-tuning method that enhances the masked language model for stance detection on Twitter.
• Learning Paralinguistic Features from Audiobooks through Style Voice Conversion
  • Zakaria Aldeneh, Matthew Perez, Emily Mower Provost
  • TLDR: We present a new framework for learning paralinguistic embeddings from speech using data that are not annotated for emotion.
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks
  • Zixuan Ke, Hu Xu, Bing Liu
  • TLDR: We propose a novel capsule network based continuous learning algorithm for aspect sentiment classification tasks.
• Adversarial Learning for Zero-Shot Stance Detection on Social Media
  • Emily Allaway, Malavika Srikanth, Kathleen McKeown
  • TLDR: We propose a new model for zero-shot stance detection on Twitter that uses adversarial learning to generalize across topics.
• Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters
  • Ramakanth Pasunuru, Mengwen Liu, Mohit Bansal, Sujith Ravi, Markus Dreyer
  • TLDR: We present a graph-enhanced approach to multi-document summarization that improves on previous work on multi-news and transfer-only datasets.
• Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization
  • Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao
  • TLDR: We present a novel architecture for abstractive summarization that uses structured intermediate representations for the syntactic structure and structure for the semantic content.
• What’s in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization
  • Griffin Adams, Emily Alsentzer, Mert Ketenci, Jason Zucker, Noémie Elhadad
  • TLDR: We present a novel, multi-document summarization task for hospital-course summarization.
• Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics
  • Artidoro Pagnoni, Vidhisha Balachandran, Yulia Tsvetkov
  • TLDR: We propose a typology of factual errors and benchmark factuality metrics for summarization models and show their correlation with human judgement and their strengths and weaknesses.
• GSum: A General Framework for Guided Neural Abstractive Summarization
  • Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, Graham Neubig
  • TLDR: We propose a general and extensible guided summarization framework for neural abstractive summarization models that can improve fidelity and improve consistency.
• What Will it Take to Fix Benchmarking in Natural Language Understanding?
  • Samuel R. Bowman, George Dahl
  • TLDR: We argue that adversarial evaluation for NLU tasks is broken, and that adversarially-constructed, out-of-distribution test sets are not the answer.
• TuringAdvice: A Generative and Dynamic Evaluation of Language Use
  • Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, Yejin Choi
  • TLDR: We propose TuringAdvice, a new challenge task and dataset for language understanding models.
• Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures
  • Ramit Sawhney, Puneet Mathur, Taru Jain, Akash Kumar Gautam, Rajiv Ratn Shah
  • TLDR: We propose a novel task for automated identification of narratives related to the sexual abuse disclosures in online posts that leverages their emotional attributes through multitask learning.
• Self Promotion in US Congressional Tweets
  • Jun Wang, Kelly Cui, Bei Yu
  • TLDR: We found that women in Congress actually perform more self-promotion on Twitter, indicating a reversal of traditional gender norms where women self-advertise less than men.
• Profiling of Intertextuality in Latin Literature Using Word Embeddings
  • Patrick J. Burns, James A. Brofos, Kyle Li, Pramit Chaudhuri, Joseph P. Dexter
  • TLDR: We present a new empirical analysis of intertextuality in classical Latin literature using word embedding models and show that word embeddings capture salient aspects of literary style.
• Identifying inherent disagreement in natural language inference
  • Xinliang Frederick Zhang, Marie-Catherine de Marneffe
  • TLDR: We propose Artificial Annotators to simulate the uncertainty in the annotation process by capturing the modes in annotations.
• Modeling Human Mental States with an Entity-based Narrative Graph
  • I-Ta Lee, Maria Leonor Pacheco, Dan Goldwasser
  • TLDR: We propose an Entity-based Narrative Graph to model the internal-state of characters in a story.
• A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation
  • Yan Zeng, Jian-Yun Nie
  • TLDR: We propose a multi-task learning approach to leverage both labeled dialogue and text data for conditioned dialogue generation.
• Hurdles to Progress in Long-form Question Answering
  • Kalpesh Krishna, Aurko Roy, Mohit Iyyer
  • TLDR: We show that the task formulation of long-form question answering is fundamentally flawed and that the dataset is not well-structured, and we propose a new system that solves this problem.
• ENTRUST: Argument Reframing with Language Models and Entailment
  • Tuhin Chakrabarty, Christopher Hidey, Smaranda Muresan
  • TLDR: We propose a method for reframing arguments for positive effects.
• Paragraph-level Simplification of Medical Texts
  • Ashwin Devaraj, Iain Marshall, Byron Wallace, Junyi Jessy Li
  • TLDR: We propose a new metric for medical text simplification based on likelihood scores from a masked language model pretrained on scientific texts.
• An Empirical Study on Neural Keyphrase Generation
  • Rui Meng, Xingdi Yuan, Tong Wang, Sanqiang Zhao, Adam Trischler, Daqing He
  • TLDR: We provide extensive experimental results and analyze the most crucial factors impacting the generalizability of neural keyphrase generation models.
• Attention Head Masking for Inference Time Content Selection in Abstractive Summarization
  • Shuyang Cao, Lu Wang
  • TLDR: We present a simple-yet-effective attention head masking technique, which is applied on encoder-decoder attentions to pinpoint salient content at inference time.
• • Zexuan Zhong, Dan Friedman, Danqi Chen
  • TLDR: We propose OptiPrompt, a novel and efficient method which directly optimizes in continuous embedding space for factual probing and show that it can predict 6.4% of facts in the LAMA benchmark.
• Evaluating Saliency Methods for Neural Language Models
  • Shuoyang Ding, Philipp Koehn
  • TLDR: We evaluate the quality of prediction interpretations from two perspectives that each represents a desirable property of these interpretations: plausibility and faithfulness.
• Contextualized Perturbation for Textual Adversarial Attack
  • Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, Bill Dolan
  • TLDR: We present CLARE, a ContextuaLized AdversaRial Example generation model that produces fluent and grammatical outputs through a mask-then-infill procedure.
• DirectProbe: Studying Representations without Classifiers
  • Yichu Zhou, Vivek Srikumar
  • TLDR: We propose a heuristic for probing the structure of contextualized embeddings that can predict the classifier performance of the representation.
• Evaluating the Values of Sources in Transfer Learning
  • Md Rizwan Parvez, Kai-Wei Chang
  • TLDR: We develop an efficient source valuation framework for quantifying the usefulness of the sources (e.g., ) in transfer learning based on the Shapley value method.
• Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications
  • Daniel Biś, Maksim Podkorytov, Xiuwen Liu
  • TLDR: We show that language models learned from Transformer architecture do not occupy a narrow cone, but rather drift in common directions.
• On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
  • Tianyi Zhang, Tatsunori B. Hashimoto
  • TLDR: We show that masking and predicting tokens in an unsupervised fashion can give rise to linguistic structures and downstream performance gains.
• Limitations of Autoregressive Models and Their Alternatives
  • Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, Jason Eisner
  • TLDR: We propose a new autoregressive language model that can model distributions whose next-symbol probability is polynomial-time.
• On the Transformer Growth for Progressive BERT Training
  • Xiaotao Gu, Liyuan Liu, Hongkun Yu, Jing Li, Chen Chen, Jiawei Han
  • TLDR: We show that Transformer growth also favors compound scaling and propose CompoundGrow, a progressive training method for BERT.
• Revisiting Simple Neural Probabilistic Language Models
  • Simeng Sun, Mohit Iyyer
  • TLDR: We present a new language model that achieves perplexity decreases across three word-level language modeling datasets.
• ReadTwice: Reading Very Large Documents with Memories
  • Yury Zemlyanskiy, Joshua Ainslie, Michiel de Jong, Philip Pham, Ilya Eckstein, Fei Sha
  • TLDR: We propose ReadTwice, a simple and effective technique that combines several strengths of prior approaches to model long-range dependencies with Transformers.
• SCRIPT: Self-Critic PreTraining of Transformers
  • Erik Nijkamp, Bo Pang, Ying Nian Wu, Caiming Xiong
  • TLDR: We introduce Self-CRItic Pretraining Transformers (SCRIPT) for representation learning of text.
• Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
  • Guanghui Qin, Jason Eisner
  • TLDR: We learn natural-language prompts by gradient descent, and show that language models retain factual knowledge from their training corpora that can be extracted by asking them to “fill in the blank” in a sentential prompt.
• Nutri-bullets Hybrid: Consensual Multi-document Summarization
  • Darsh Shah, Lili Yu, Tao Lei, Regina Barzilay
  • TLDR: We present a method for generating comparative summaries that highlight similarities and contradictions in input documents.
• AVA: an Automatic eValuation Approach for Question Answering Systems
  • Thuy Vu, Alessandro Moschitti
  • TLDR: We introduce AVA, an automatic evaluation approach for Question Answering, which given a set of questions associated with Gold Standard answers (references), can estimate system Accuracy.
• SpanPredict: Extraction of Predictive Document Spans with Neural Attention
  • Vivek Subramanian, Matthew Engelhard, Sam Berchuck, Liqun Chen, Ricardo Henao, Lawrence Carin
  • TLDR: We propose a novel method for extracting text spans from clinical notes that are predictive of future medical diagnoses.
• Text Editing by Command
  • Felix Faltings, Michel Galley, Gerold Hintz, Chris Brockett, Chris Quirk, Jianfeng Gao, Bill Dolan
  • TLDR: We propose a novel text editing task and a transformer-based model for text generation that outperforms existing models.
• A Deep Metric Learning Approach to Account Linking
  • Aleem Khan, Elizabeth Fleming, Noah Schofield, Marcus Bishop, Nicholas Andrews
  • TLDR: We propose a novel algorithm for linking social media accounts that belong to the same author in an automated fashion on the basis of the content and meta-data of the corresponding document streams.
• Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
  • Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, Dan Jurafsky
  • TLDR: We propose a new report generation system that optimizes reinforcement learning to generate more accurate and consistent radiology report generation.
• Multimodal End-to-End Sparse Model for Emotion Recognition
  • Wenliang Dai, Samuel Cahyawijaya, Zihan Liu, Pascale Fung
  • TLDR: We develop a fully end-to-end model that connects the two phases and optimizes them jointly.
• MIMOQA: Multimodal Input Multimodal Output Question Answering
  • Hrituraj Singh, Anshul Nasery, Denil Mehta, Aishwarya Agarwal, Jatin Lamba, Balaji Vasan Srinivasan
  • TLDR: We propose a novel multimodal question-answering task in which the output is also multimodally multimodaled.
• OCID-Ref: A 3D Robotic Dataset With Embodied Language For Clutter Scene Grounding
  • Ke-Jyun Wang, Yun-Hsuan Liu, Hung-Ting Su, Jen-Wei Wang, Yu-Siang Wang, Winston Hsu, Wen-Chin Chen
  • TLDR: We propose a novel OCID-Ref dataset featuring a referring expression segmentation task with referring expressions of occluded objects.
• Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
  • Liunian Harold Li, Haoxuan You, Zhecan Wang, Alireza Zareian, Shih-Fu Chang, Kai-Wei Chang
  • TLDR: We propose to learn a strong V&L representation model without image-caption corpora.
• Multitasking Inhibits Semantic Drift
  • Athul Paul Jacob, Mike Lewis, Jacob Andreas
  • TLDR: We show that multitask training of neural latent language policies in a complex strategy game reduces drift and while improving sample efficiency.
• Probing Contextual Language Models for Common Ground with Visual Representations
  • Gabriel Ilharco, Rowan Zellers, Ali Farhadi, Hannaneh Hajishirzi
  • TLDR: We propose a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations.
• BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification
  • Ishani Mondal
  • TLDR: We propose BBAEG (Biomedical BERT-based Adversarial Example Generation), a black-box attack algorithm for biomedical text classification, leveraging the strengths of both domain-specific synonym replacement for biomedical named entities and BERT predictions, spelling variation and number replacement.
• Targeted Adversarial Training for Natural Language Understanding
  • Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
  • TLDR: Targeted Adversarial Training for Natural Language Understanding.
• Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection
  • Xu Guo, Boyang Li, Han Yu, Chunyan Miao
  • TLDR: We propose a generalized latent optimization strategy that allows different losses to accommodate each other and improves training dynamics.
• Self-training Improves Pre-training for Natural Language Understanding
  • Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
  • TLDR: We present a novel data augmentation method for self-training that improves on standard text classification benchmarks and knowledge-distillation.
• Supporting Clustering with Contrastive Learning
  • Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
  • TLDR: We propose Supporting Clustering with Contrastive Learning (SCCL) – a novel framework to leverage contrastive learning to promote better separation between different categories in unsupervised clustering.
• TITA: A Two-stage Interaction and Topic-Aware Text Matching Model
  • Xingwu Sun, Yanling Cui, Hongyin Tang, Qiuyu Zhu, Fuzheng Zhang, Beihong Jin
  • TLDR: We propose a novel method for keyword and document matching by considering different relevance levels.
• Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction
  • Zhenghao Liu, Xiaoyuan Yi, Maosong Sun, Liner Yang, Tat-Seng Chua
  • TLDR: We present a neural verification network for grammatical error correction with multiple hypotheses.
• Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects
  • Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He
  • TLDR: We show that neural network tuning can be controlled by the number of changed parameters and thus, we propose to conduct a new method to measure side effects.
• Discrete Argument Representation Learning for Interactive Argument Pair Identification
  • Lu Ji, Zhongyu Wei, Jing Li, Qi Zhang, Xuanjing Huang
  • TLDR: We propose a novel method for identifying interactive argument pairs from two posts with opposite stances to a certain topic.
• On Unifying Misinformation Detection
  • Nayeon Lee, Belinda Z. Li, Sinong Wang, Pascale Fung, Hao Ma, Wen-tau Yih, Madian Khabsa
  • TLDR: We present a general-purpose misinformation model that learns a richer representation of misinformation and demonstrate its generalizability to unseen events.
• Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model
  • Honai Ueoka, Yugo Murawaki, Sadao Kurohashi
  • TLDR: We present a new approach to linguistic steganography that uses a masked language model to generate text that is more secure and payload-efficient than generation-based models.
• Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning
  • Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, Shiqi Xu
  • TLDR: We present a simple training strategy for few-shot text classification that improves performance and robustness to high amounts of noising.
• Do RNN States Encode Abstract Phonological Alternations?
  • Miikka Silfverberg, Francis Tyers, Garrett Nicolai, Mans Hulden
  • TLDR: We show that sequence-to-sequence models can encode 17 different consonant gradation processes in a handful of dimensions in the RNN.
• Pre-training with Meta Learning for Chinese Word Segmentation
  • Zhen Ke, Liang Shi, Songtao Sun, Erli Meng, Bin Wang, Xipeng Qiu
  • TLDR: MetaSeg is a pre-trained pre-training model for Chinese word segmentation that incorporates meta learning algorithm into a multi-criteria pre- training task.
• Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation
  • Hua Zheng, Damai Dai, Lei Li, Tianyu Liu, Zhifang Sui, Baobao Chang, Yang Liu
  • TLDR: We propose a new method for word formation generation in Chinese, which generates definitions for words by combining formation components.
• User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalization
  • Shohei Higashiyama, Masao Utiyama, Taro Watanabe, Eiichiro Sumita
  • TLDR: We present a publicly available Japanese UGT corpus for morphological analysis and lexical normalization, and show that the corpus is a challenging benchmark for further research on UGT.
• GPT Perdetry Test: Generating new meanings for new words
  • Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, Nebojsa Jojic
  • TLDR: We show that GPT-3 is capable of generating plausible definitions for new words and showing that they are preferable to those invented by humans.
• Universal Semantic Tagging for English and Mandarin Chinese
  • Wenxi Li, Yiyang Hou, Yajie Ye, Li Liang, Weiwei Sun
  • TLDR: We propose a novel way to unify meaning representations for multiple languages at the word level and empirically evaluate its plausibility.
• ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser
  • Zhi Chen, Lu Chen, Yanbin Zhao, Ruisheng Cao, Zihan Xu, Su Zhu, Kai Yu
  • TLDR: We propose a new architecture for unseen database schemas, which can generalize to unseen and rare schemas.
• Contextualized and Generalized Sentence Representations by Contrastive Self-Supervised Learning: A Case Study on Discourse Relation Analysis
  • Hirokazu Kiyomaru, Sadao Kurohashi
  • TLDR: We propose a method to learn contextualized and generalized sentence representations using contrastive self-supervised learning.
• AMR Parsing with Action-Pointer Transformer
  • Jiawei Zhou, Tahira Naseem, Ramón Fernandez Astudillo, Radu Florian
  • TLDR: We propose a new way to parse meaning representation parsers that combines hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments.
• NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction
  • Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Hassan Awadallah
  • TLDR: We present a model for interpreting natural language feedback in the interaction context to generate a sequence of edits that can be applied to the initial parse to correct its errors.
• Unsupervised Concept Representation Learning for Length-Varying Text Similarity
  • Xuchao Zhang, Bo Zong, Wei Cheng, Jingchao Ni, Yanchi Liu, Haifeng Chen
  • TLDR: We propose an unsupervised concept representation learning approach to address the information gap caused by context and vocabulary mismatches when comparing varying-length texts.
• Augmenting Knowledge-grounded Conversations with Sequential Knowledge Transition
  • Haolan Zhan, Hainan Zhang, Hongshen Chen, Zhuoye Ding, Yongjun Bao, Yanyan Lan
  • TLDR: We propose a novel knowledge transition model for sequential multi-turn multi-talkative dialogues that fully utilizes the limited knowledge data and generates knowledge-irrelevant response.
• Adversarial Self-Supervised Learning for Out-of-Domain Detection
  • Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Hong Xu, Weiran Xu
  • TLDR: We propose a self-supervised contrastive learning framework to detect out-of-domain intents from unlabeled data.
• Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking
  • Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba
  • TLDR: We propose a slot descriptions enhanced generative approach for zero-shot cross-domain dialogue state tracking.
• Hierarchical Transformer for Task Oriented Dialog Systems
  • Bishal Santra, Potnuru Anusha, Pawan Goyal
  • TLDR: We propose a generalized framework for Hierarchical Transformer Encoders and show how a standard transformer can be morphed into any hierarchical encoder, including HRED and HIBERT like models, by using specially designed attention masks and positional encodings.
• Measuring the ‘I don’t know’ Problem through the Lens of Gricean Quantity
  • Huda Khayrallah, João Sedoc
  • TLDR: We propose Relative Utterance Quantity (RUQ) to diagnose the ‘I don’t know’ problem, in which a dialog system produces generic responses.
• RTFE: A Recursive Temporal Fact Embedding Framework for Temporal Knowledge Graph Completion
  • Youri Xu, Haihong E, Meina Song, Wenyu Song, Xiaodong Lv, Wang Haotian, Yang Jinrui
  • TLDR: We propose a Recursive Temporal Fact Embedding framework to transplant static knowledge graph embedding models to temporal knowledge graph (TKG) embedding.
• Open Hierarchical Relation Extraction
  • Kai Zhang, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun
  • TLDR: We propose a novel hierarchical triplet objective and hierarchical curriculum training paradigm for OpenRE and present a novel OHRE framework for the task.
• Jointly Extracting Explicit and Implicit Relational Triples with Reasoning Pattern Enhanced Binary Pointer Network
  • Yubo Chen, Yunqi Zhang, Changran Hu, Yongfeng Huang
  • TLDR: We propose a unified framework to jointly extract explicit and implicit relational triples that lack explicit expressions.
• Multi-Grained Knowledge Distillation for Named Entity Recognition
  • Xuan Zhou, Xiao Zhang, Chenyang Tao, Junya Chen, Bing Xu, Wei Wang, Jing Xiao
  • TLDR: We present a novel distillation scheme to efficiently transfer knowledge learned from big models to their more affordable counterpart.
• SGG: Learning to Select, Guide, and Generate for Keyphrase Generation
  • Jing Zhao, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, Bowen Zhou
  • TLDR: Select-Guide-Generate is a novel keyphrase generation algorithm which combines present and absent keyphrase generating in a hierarchical fashion.
• Towards Sentiment and Emotion aided Multi-modal Speech Act Classification in Twitter
  • Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, Pushpak Bhattacharyya
  • TLDR: We present a new multi-modal, emotion-TA dataset for Twitter.
• Generative Imagination Elevates Machine Translation
  • Quanyu Long, Mingxuan Wang, Lei Li
  • TLDR: We propose ImagiT, a novel multimodal neural machine translation method via visual imagination.
• Non-Autoregressive Translation by Learning Target Categorical Codes
  • Yu Bao, Shujian Huang, Tong Xiao, Dongqi Wang, Xinyu Dai, Jiajun Chen
  • TLDR: We propose CNAT, a novel non-autoregressive translation model that learns implicitly categorical codes as latent variables into the non-Autoregressive decoding.
• Training Data Augmentation for Code-Mixed Translation
  • Abhirut Gupta, Aditya Vavre, Sunita Sarawagi
  • TLDR: We present an efficient and effective translation model based on ternary sequence labeling and data augmentation strategy.
• Rethinking Perturbations in Encoder-Decoders for Fast Training
  • Sho Takase, Shun Kiyono
  • TLDR: We compare several perturbations in sequence-to-sequence problems with respect to computational time.
• Context-aware Decoder for Neural Machine Translation using a Target-side Document-Level Language Model
  • Amane Sugiyama, Naoki Yoshinaga
  • TLDR: We present a novel method to perform context-aware translation with pre-trained sentence-level translation models by using document-level language models.
• Machine Translated Text Detection Through Text Similarity with Round-Trip Translation
  • Hoang-Quoc Nguyen-Son, Tran Thao, Seira Hidano, Ishita Gupta, Shinsaku Kiyomoto
  • TLDR: We propose a new detector for detecting a translated text from a strange translator.
• TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
  • Deming Ye, Yankai Lin, Yufei Huang, Maosong Sun
  • TLDR: We propose a dynamic token reduction approach to accelerate PLMs’ inference by 2-5 times to satisfy various performance demands.
• Breadth First Reasoning Graph for Multi-hop Question Answering
  • Yongjie Huang, Meng Yang
  • TLDR: We propose a novel model of Breadth First Reasoning Graph (BFR-Graph) which is proposed to improve answer span prediction and interpretable score aggregation.
• Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph
  • Yucheng Zhou, Xiubo Geng, Tao Shen, Wenqiang Zhang, Daxin Jiang
  • TLDR: We propose a novel adversarial learning strategy to solve multilingual question answering over knowledge graph with augmented training data and adversarial adversarial training strategy to alleviate syntax-disorder of the augmented data.
• RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
  • Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
  • TLDR: We propose an optimized training approach for dense passage retrieval based on cross-batch negatives, denoised hard negatives and data augmentation.
• DAGN: Discourse-Aware Graph Network for Logical Reasoning
  • Yinya Huang, Meng Fang, Yu Cao, Liwei Wang, Xiaodan Liang
  • TLDR: We propose a graph-based graph network for logical reasoning QA that learns discourse-aware features and uses them to solve logical reasoning questions.
• Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering
  • Sohee Yang, Minjoon Seo
  • TLDR: We present several orthogonal strategies to drastically reduce the footprint of a retrieve-and-read open-domain QA system by up to 160x.
• Unsupervised Multi-hop Question Answering by Question Generation
  • Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, William Yang Wang
  • TLDR: We propose MQA-QG, an unsupervised framework that can generate human-like multi-hop question answering data from both homogeneous and heterogeneous data sources.
• Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents
  • Peng Cui, Le Hu
  • TLDR: We propose a novel sliding selector network with dynamic memory for extractive summarization of long-form documents, which employs a sliding window to extract summary sentences segment by segment.
• AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
  • Tiezheng Yu, Zihan Liu, Pascale Fung
  • TLDR: We present a study of domain adaptation for the abstractive summarization task across six diverse target domains in a low-resource setting.
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
  • Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev
  • TLDR: We present a new query-based multi-domain meeting summarization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task.
• MM-AVS: A Full-Scale Dataset for Multi-modal Summarization
  • Xiyan Fu, Jun Wang, Zhenglu Yang
  • TLDR: We present a new multimodal dataset for multimodality summarization and propose Jump-Attention mechanism for multimodial summarization.
• MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
  • Chenguang Zhu, Yang Liu, Jie Mei, Michael Zeng
  • TLDR: We present a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries.
• Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
  • Sihao Chen, Fan Zhang, Kazoo Sone, Dan Roth
  • TLDR: We propose a novel method for generating and selecting alternative candidate summaries for neural abstractive summarization that are not unfaithful to the original text.
• Inference Time Style Control for Summarization
  • Shuyang Cao, Lu Wang
  • TLDR: We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model to generate summaries of different styles without requiring corpora in the target styles, or training separate models.
• ReinforceBug: A Framework to Generate Adversarial Textual Examples
  • Bushra Sabir, Muhammad Ali Babar, Raj Gaire
  • TLDR: We present ReinforceBug, a reinforcement learning framework, that learns a policy that is transferable on unseen datasets and generates utility-preserving and transferable (on other models) adversarial examples.