ACL 2022

• German
• Italian
• Chinese
• Japanese

TLDRs

• Investigating label suggestions for opinion mining in German Covid-19 social media
  • Tilman Beck, Ji-Ung Lee, Christina Viehmann, Marcus Maurer, Oliver Quiring, Iryna Gurevych
  • TLDR: We investigate the use of interactively updated label suggestions to improve the efficiency of gathering annotations on the task of opinion mining in German Covid-19 social media data.
• How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements
  • Chen Shani, Nadav Borenstein, Dafna Shahaf
  • TLDR: We use humor mining to learn classifiers for detecting funny and unusual scientific papers.
• Engage the Public: Poll Question Generation for Social Media Posts
  • Zexin Lu, Keyang Ding, Yuji Zhang, Jing Li, Baolin Peng, Lemao Liu
  • TLDR: We propose a novel task to generate poll questions for social media posts by exploiting latent topics from user comments and discover latent topics therein as contexts.
• HateCheck: Functional Tests for Hate Speech Detection Models
  • Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, Janet Pierrehumbert
  • TLDR: We present a suite of functional tests for hate speech detection models that provide critical diagnostic insights into the weaknesses of popular models.
• Unified Dual-view Cognitive Model for Interpretable Claim Verification
  • Lianwei Wu, Yuan Rao, Yuqian Lan, Ling Sun, Zhaoyin Qi
  • TLDR: We propose a Dual-view model based on the views of Collective and Individual Cognition for interpretable claim verification.
• DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling
  • Lanqing Xue, Kaitao Song, Duocai Wu, Xu Tan, Nevin L. Zhang, Tao Qin, Wei-Qiang Zhang, Tie-Yan Liu
  • TLDR: We develop DeepRapper, a Transformer-based rap generation system that can model both rhymes and rhythms.
• PENS: A Dataset and Generic Framework for Personalized News Headline Generation
  • Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, Xing Xie
  • TLDR: We propose a novel personalized news headline generation problem that uses user preference to generate personalized headlines based on both a user’s reading interests and a candidate news body to be exposed to her.
• Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization
  • Dongkyu Lee, Zhiliang Tian, Lanqing Xue, Nevin L. Zhang
  • TLDR: We propose a novel method for text style transfer that preserves the content of a sentence while preserving its style.
• Mention Flags (MF): Constraining Transformer-based Text Generators
  • Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, Mark Johnson
  • TLDR: We propose Mention Flags, a novel approach to constrained text generation which can guarantee constraint satisfaction in an S2S decoder.
• Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation
  • Giulio Zhou, Gerasimos Lampouras
  • TLDR: Language Agnostic Delexicalisation for concept-to-text generation using multilingual pretrained embeddings and post-editing.
• Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances
  • Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, Jie Zhou
  • TLDR: We propose a new approach to generate acceptable responses according to the historical context based on the large-scale pre-trained language models.
• Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking
  • Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang
  • TLDR: We present a new two-stage DSS-DSTT which achieves state-of-the-art performance on MultiWOZ 2.0, MultiWOAZ 2, MultiOZ 3.1, and MultiWOWZ 2.
• Transferable Dialogue Systems and User Simulators
  • Bo-Hsiang Tseng, Yinpei Dai, Florian Kreyssig, Bill Byrne
  • TLDR: We propose a new dialogue system that can incorporate new dialogue scenarios through self-play between the two agents.
• BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data
  • Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, Ting Liu
  • TLDR: We propose a novel BERT-over-BERT model for persona-based dialogue generation and consistency understanding.
• GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling
  • Libo Qin, Fuxuan Wei, Tianbao Xie, Xiao Xu, Wanxiang Che, Ting Liu
  • TLDR: We propose a non-autoregressive model for multi-intent SLU that achieves state-of-the-art performance while being 11.5 times faster.
• Accelerating BERT Inference for Sequence Labeling via Early-Exit
  • Xiaonan Li, Yunfan Shao, Tianxiang Sun, Hang Yan, Xipeng Qiu, Xuanjing Huang
  • TLDR: We extend sentence-level early-exit mechanism to sequence labeling tasks and propose a token-level mechanism to accelerate the inference of PTMs for sequence labeling.
• Modularized Interaction Network for Named Entity Recognition
  • Fei Li, Zheng Wang, Siu Cheung Hui, Lejian Liao, Dandan Song, Jing Xu, Guoxiu He, Meihuizi Jia
  • TLDR: We propose a novel Modularized Interaction Network (MIN) model which utilizes both segment-level information and word-level dependencies, and incorporates an interaction mechanism to support information sharing between boundary detection and type prediction to enhance the performance for the NER task.
• Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder
  • Xi Xiangyu, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, Wei Wu
  • TLDR: We propose a novel method for generating argument roles for event argument extraction by incorporating contextual entities’ argument role predictions.
• UniRE: A Unified Label Space for Entity Relation Extraction
  • Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei Li, Junchi Yan
  • TLDR: We propose a novel entity relation extraction model that learns the labels of entities and relations from a table.
• Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction
  • Li Cui, Deqing Yang, Jiaxin Yu, Chengwei Hu, Jiayang Cheng, Jingjie Yi, Yanghua Xiao
  • TLDR: We propose a novel method for continual relation extraction that uses memory information to improve the stability of the model and avoid catastrophic forgetting.
• Contrastive Learning for Many-to-many Multilingual Neural Machine Translation
  • Xiao Pan, Mingxuan Wang, Liwei Wu, Lei Li
  • TLDR: We propose mRASP2, a training method to obtain a single unified multilingual translation model.
• Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation
  • Mathias Müller, Rico Sennrich
  • TLDR: We empirically investigate the properties of minimum Bayes Risk decoding on neural machine translation and show that it improves robustness to copy noise.
• Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation
  • Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Meng Zhang
  • TLDR: We present a new Transformer translation model that uses self-attention networks for context modelling.
• A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment
  • Jingyi Zhang, Josef van Genabith
  • TLDR: We present a bidirectional Transformer based word alignment model for neural machine translation and show that it outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++.
• Learning Language Specific Sub-network for Multilingual Machine Translation
  • Zehui Lin, Liwei Wu, Mingxuan Wang, Lei Li
  • TLDR: Language Specific Sub-network for multilingual neural machine translation.
• Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis
  • Linyi Yang, Jiazheng Li, Padraig Cunningham, Yue Zhang, Barry Smyth, Ruihai Dong
  • TLDR: We propose a novel approach to automatically generate counterfactually augmented datasets for the purpose of data augmentation and explanation in NLP models.
• Bridge-Based Active Domain Adaptation for Aspect Term Extraction
  • Zhuang Chen, Tieyun Qian
  • TLDR: We propose a novel active domain adaptation method for aspect term extraction that uses transferable knowledge to augment the knowledge of aspect terms.
• Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks
  • Xiaocui Yang, Shi Feng, Yifei Zhang, Daling Wang
  • TLDR: We propose a multimodal sentiment expression detection approach based on the global characteristics of the dataset.
• Aspect-Category-Opinion-Sentiment Quadruple Extraction with Implicit Aspects and Opinions
  • Hongjie Cai, Rui Xia, Jianfei Yu
  • TLDR: We introduce a new task for aspect-based sentiment analysis with implicit aspects and implicit opinions.
• PASS: Perturb-and-Select Summarizer for Product Reviews
  • Nadav Oved, Ran Levy
  • TLDR: We propose a novel system for summarizing reviews of a given product by generating multiple different summaries per product, which are more informative, diverse and coherent.
• Deep Differential Amplifier for Extractive Summarization
  • Ruipeng Jia, Yanan Cao, Fang Fang, Yuchen Zhou, Zheng Fang, Yanbing Liu, Shi Wang
  • TLDR: We present a novel approach for sentence-level extractive summarization that uses a deep differential amplifier to rebalance the sentence-specific features.
• Multi-TimeLine Summarization (MTLS): Improving Timeline Summarization by Generating Multiple Summaries
  • Yi Yu, Adam Jatowt, Antoine Doucet, Kazunari Sugiyama, Masatoshi Yoshikawa
  • TLDR: We propose a novel unsupervised summarization framework based on two-stage affinity propagation and a quantitative evaluation measure for it.
• Self-Supervised Multimodal Opinion Summarization
  • Jinbae Im, Moonki Kim, Hoyeop Lee, Hyunsouk Cho, Sehee Chung
  • TLDR: We propose a self-supervised multimodal opinion summarization framework based on multimodality data and propose a multimodally-fused multimodaling algorithm to resolve the inherent heterogeneity of multimodals.
• A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancy
  • Wang Chen, Piji Li, Irwin King
  • TLDR: We propose a new meta-meta-meta summarization evaluation metric based on pseudo-referenced pseudo-reference and redundancy.
• DESCGEN: A Distantly Supervised Datasetfor Generating Entity Descriptions
  • Weijia Shi, Mandar Joshi, Luke Zettlemoyer
  • TLDR: We present a multi-document summarization task for entity descriptions that uses Wikipedia and Fandom to generate a summary of relevant information.
• Introducing Orthogonal Constraint in Structural Probes
  • Tomasz Limisiewicz, David Mareček
  • TLDR: We propose a new type of structural probing that uses orthogonal constraints to approximate syntactic dependency in pre-trained models.
• Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
  • Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun
  • TLDR: We propose a novel textual backdoor attack method that uses the syntactic structure as the trigger in textual backdoor attacks.
• Examining the Inductive Bias of Neural Language Models with Artificial Languages
  • Jennifer C. White, Ryan Cotterell
  • TLDR: We propose a novel method for investigating the inductive biases of language models using artificial languages.
• Explaining Contextualization in Language Models using Visual Analytics
  • Rita Sevastjanova, Aikaterini-Lida Kalouli, Christin Beck, Hanna Schäfer, Mennatallah El-Assady
  • TLDR: We explore the continuum between function and content words with respect to contextualization in BERT, based on linguistically-informed insights.
• Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification
  • George Chrysostomou, Nikolaos Aletras
  • TLDR: We propose a new family of Task-Scaling mechanisms that learn task-specific non-contextualised information to scale the original attention weights.
• Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem
  • Raphael Schumann, Stefan Riezler
  • TLDR: We present a neural model that learns to generate navigation instructions that contain visible and salient landmarks from human natural language instructions.
• E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning
  • Haiyang Xu, Ming Yan, Chenliang Li, Bin Bi, Songfang Huang, Wenming Xiao, Fei Huang
  • TLDR: We propose the first end-to-end vision-language pre-trained model for both V+L understanding and generation, namely E2E-VLP, where we build a unified Transformer framework to jointly learn visual representation, and semantic alignments between image and text.
• Learning Relation Alignment for Calibrated Cross-modal Retrieval
  • Shuhuai Ren, Junyang Lin, Guangxiang Zhao, Rui Men, An Yang, Jingren Zhou, Xu Sun, Hongxia Yang
  • TLDR: We propose a new metric, Intra-modal Self-attention Distance, to quantify the relation consistency between linguistic and visual relations in image-text retrieval.
• KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation
  • Yiran Xing, Zai Shi, Zhao Meng, Gerhard Lakemeyer, Yunpu Ma, Roger Wattenhofer
  • TLDR: We present Knowledge Enhanced Multimodal BART, a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts.
• Cascaded Head-colliding Attention
  • Lin Zheng, Zhiyong Wu, Lingpeng Kong
  • TLDR: We present a probabilistic model for multi-head attention in Transformer architecture.
• Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor
  • Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
  • TLDR: We derive a factorized form of the knowledge distillation objective for structured prediction, which is tractable for many typical choices of the teacher and student models.
• Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
  • Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, James Henderson
  • TLDR: We propose a new multi-task learning framework that learns adapter parameters for all layers and tasks by generating shared hypernetworks, which condition on task, adapter position, and layer id in a transformer model.
• COSY: COunterfactual SYntax for Cross-Lingual Understanding
  • Sicheng Yu, Hao Zhang, Yulei Niu, Qianru Sun, Jing Jiang
  • TLDR: We propose a novel approach to train language models with universal syntax, based on the observation that universal syntax is transferable across different languages.
• OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification
  • Seonghyeon Lee, Dongha Lee, Hwanjo Yu
  • TLDR: We propose a new approach to finding and regularizing the remainder of the space, referred to as out-of-manifold, which cannot be accessed through the words.
• Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model
  • Kathleen C. Fraser, Isar Nejadgholi, Svetlana Kiritchenko
  • TLDR: We propose a computational approach to interpret stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology.
• Structurizing Misinformation Stories via Rationalizing Fact-Checks
  • Shan Jiang, Christo Wilson
  • TLDR: We propose a new way to classify misinformation stories by leveraging fact-check articles.
• Modeling Language Usage and Listener Engagement in Podcasts
  • Sravana Reddy, Mariya Lazarova, Yongze Yu, Rosie Jones
  • TLDR: We show that various factors in podcast language correlate with engagement, based on analysis of the creators’ written descriptions and transcripts of the audio.
• Breaking Down the Invisible Wall of Informal Fallacies in Online Discussions
  • Saumya Sahai, Oana Balalau, Roxana Horincar
  • TLDR: We present the most frequent fallacies on Reddit, and present a new dataset of them.
• SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues
  • Liang Qiu, Yuan Liang, Yizhou Zhao, Pan Lu, Baolin Peng, Zhou Yu, Ying Nian Wu, Song-Chun Zhu
  • TLDR: We propose a new approach for inferring social relations from dialogues by modeling the social network as an And-or Graph.
• TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems
  • Bill Byrne, Karthik Krishnamoorthi, Saravanan Ganesh, Mihir Kale
  • TLDR: We present a data-driven, end-to-end approach to transaction-based dialog systems that performs at near-human levels in terms of verbal response quality and factual grounding accuracy.
• Improving Dialog Systems for Negotiation with Personality Modeling
  • Runzhe Yang, Jingxiao Chen, Karthik Narasimhan
  • TLDR: We propose a probabilistic formulation to model and infer personality types of opponents, predict their responses, and use this information to adapt a dialog agent’s high-level strategy in negotiation tasks.
• Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
  • Wangchunshu Zhou, Qifei Li, Chenle Li
  • TLDR: We propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better.
• Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
  • Hannah Rashkin, David Reitter, Gaurav Singh Tomar, Dipanjan Das
  • TLDR: We present a method for training a generative neural dialogue model that is controlled to stay faithful to the evidence.
• CitationIE: Leveraging the Citation Graph for Scientific Information Extraction
  • Vijay Viswanathan, Graham Neubig, Pengfei Liu
  • TLDR: We propose a citation-aware SciIE algorithm for scientific text extraction that can improve the quality of scientific information extraction and help identify methods and materials for a given problem.
• From Discourse to Narrative: Knowledge Projection for Event Relation Extraction
  • Jialong Tang, Hongyu Lin, Meng Liao, Yaojie Lu, Xianpei Han, Le Sun, Weijian Xie, Jin Xu
  • TLDR: We propose a knowledge projection paradigm for event relation extraction that can leverage multi-tier discourse knowledge effectively for event relations extraction.
• AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER
  • Weile Chen, Huiqiang Jiang, Qianhui Wu, Börje Karlsson, Yi Guan
  • TLDR: We propose a novel adversarial approach to learn entity domain knowledge from unlabeled data in a target language and use it to improve cross-lingual model performance.
• Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge
  • Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, Ming Zhou
  • TLDR: We propose a novel end-to-end graph neural model for fake news detection.
• Discontinuous Named Entity Recognition as Maximal Clique Discovery
  • Yucheng Wang, Bowen Yu, Hongsong Zhu, Tingwen Liu, Nan Yu, Limin Sun
  • TLDR: We propose a novel algorithm for named entity recognition that learns to recognize discontinuous entities by finding maximal cliques in a graph and concatenating the spans in each clique.
• LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking
  • Hang Jiang, Sairam Gurajada, Qiuhao Lu, Sumit Neelam, Lucian Popa, Prithviraj Sen, Yunyao Li, Alexander Gray
  • TLDR: We present a novel approach to entity linking that combines the advantages of using interpretable rules based on first-order logic with the performance of neural learning.
• Do Context-Aware Translation Models Pay the Right Attention?
  • Kayo Yin, Patrick Fernandes, Danish Pruthi, Aditi Chaudhary, André F. T. Martins, Graham Neubig
  • TLDR: We present a new dataset for contextualizing ambiguous translations and show that it can help to resolve ambiguous pronouns and polysemous words.
• Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data
  • Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, Mona Diab
  • TLDR: We propose a novel method for translating to and from a low-resource language with only monolingual data, in addition to any parallel data in the related high-resource languages.
• Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment
  • Haoyue Shi, Luke Zettlemoyer, Sida I. Wang
  • TLDR: We show that combining unsupervised bitext mining and unsupervision for lexicon generation can improve the quality of bilingual lexicons and provide a new approach to interpretable lexical meaning in context.
• Multilingual Speech Translation from Efficient Finetuning of Pretrained Models
  • Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli
  • TLDR: We present a simple yet effective approach to build multilingual speech-to-text (ST) translation through efficient transfer learning from a pretrained speech encoder and text decoder.
• Learning Faithful Representations of Causal Graphs
  • Ananth Balashankar, Lakshminarayanan Subramanian
  • TLDR: We propose a new method for learning causal graph embeddings that are more faithful to human validated graphs.
• What Context Features Can Transformer Language Models Use?
  • Joe O’Connor, Jacob Andreas
  • TLDR: We show that long contexts, but not their detailed syntactic and propositional content, are important for the low perplexity of current transformer language models.
• Integrated Directional Gradients: Feature Interaction Attribution for Neural NLP Models
  • Sandipan Sikdar, Parantapa Bhattacharya, Kieran Heese
  • TLDR: We propose a method for attribute-based feature group attribution in neural networks that captures semantic interactions in neural network models via negations and conjunctions.
• DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
  • John Giorgi, Osvald Nitski, Bo Wang, Gary Bader
  • TLDR: We present DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations, a novel approach for learning universal sentence embeddings that does not require labelled training data.
• XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation
  • Dongqin Xu, Junhui Li, Muhua Zhu, Min Zhang, Guodong Zhou
  • TLDR: We propose a novel cross-lingual pre-training approach via multi-task learning (MTL) for both zeroshot AMR parsing and AMR-to-text generation.
• Span-based Semantic Parsing for Compositional Generalization
  • Jonathan Herzig, Jonathan Berant
  • TLDR: We propose SpanBasedSP, a span-based parser that predicts a span tree over an input utterance, explicitly encoding how partial programs compose over spans in the input.
• Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?
  • Peter Shaw, Ming-Wei Chang, Panupong Pasupat, Kristina Toutanova
  • TLDR: We propose new train and test splits of non-synthetic datasets for compositional generalization and natural language variation in semantic parsing.
• A Targeted Assessment of Incremental Processing in Neural Language Models and Humans
  • Ethan Wilcox, Pranali Vani, Roger Levy
  • TLDR: We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic test suites across a range of structural phenomena.
• The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing
  • Valentina Pyatkin, Shoval Sadde, Aynat Rubinstein, Paul Portner, Reut Tsarfaty
  • TLDR: We present a novel event-based modality detection task for NLP and show that it improves the detection of modal events in their own right.
• To POS Tag or Not to POS Tag: The Impact of POS Tags on Morphological Learning in Low-Resource Settings
  • Sarah Moeller, Ling Liu, Mans Hulden
  • TLDR: We show that the presence or absence of POS tags does not have a significant bearing on performance on two computational morphological tasks with the Transformer architecture.
• Prosodic segmentation for parsing spoken dialogue
  • Elizabeth Nielsen, Mark Steedman, Sharon Goldwater
  • TLDR: We show that prosody can effectively replace gold standard sentence-like unit boundaries in a speech parser that receives an entire dialogue turn as input, and show that this improves parse performance.
• VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
  • Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux
  • TLDR: We introduce VoxPopuli, a large-scale multilingual corpus providing 400K hours of unlabeled speech data in 23 languages.
• Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
  • Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
  • TLDR: We identify a range of pitfalls in the measurement models for NLP tasks that threaten the validity of these benchmarks as measurement models.
• Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network
  • Justin Lovelace, Denis Newman-Griffis, Shikhar Vashishth, Jill Fain Lehman, Carolyn Rosé
  • TLDR: We develop a deep convolutional network for knowledge graph completion that outperforms recent KG completion methods in the realistic setting.
• A DQN-based Approach to Finding Precise Evidences for Fact Verification
  • Hai Wan, Haicheng Chen, Jianfeng Du, Weilin Luo, Rongzhen Ye
  • TLDR: We propose a DQN-based approach to compute precise evidences for claim verification and show improvements in accuracy.
• The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing
  • Ji Xin, Raphael Tang, Yaoliang Yu, Jimmy Lin
  • TLDR: We show that recent pre-trained transformer models simultaneously improve both model accuracy and confidence estimation effectiveness.
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers
  • Keyang Xu, Tongzheng Ren, Shikun Zhang, Yihao Feng, Caiming Xiong
  • TLDR: We propose a simple yet effective method to detect out-of-domain samples with only unsupervised in-domain data.
• MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
  • Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh
  • TLDR: We present MATE-KD, a novel text-based adversarial training algorithm which improves the performance of knowledge distillation.
• Selecting Informative Contexts Improves Language Model Fine-tuning
  • Richard Antonello, Nicole Beckage, Javier Turek, Alexander Huth
  • TLDR: We propose a general fine-tuning method that improves the overall training efficiency and final performance of language model fine-tuning.
• Explainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification
  • Cristina Garbacea, Mengtian Guo, Samuel Carton, Qiaozhu Mei
  • TLDR: We propose a new way to improve the out-of-sample text simplification performance of black-box models by improving the transparency and explainability of the process.
• Multi-Task Retrieval for Knowledge-Intensive Tasks
  • Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oguz, Veselin Stoyanov, Gargi Ghosh
  • TLDR: We propose a multi-task trained neural retriever for out-of-domain retrieval that outperforms existing models on multiple benchmarks.
• When Do You Need Billions of Words of Pretraining Data?
  • Yian Zhang, Alex Warstadt, Xiaocheng Li, Samuel R. Bowman
  • TLDR: We explore the extent to which Transformer LMs learn from large-scale pretraining and how they learn to encode syntactic and semantic features.
• Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation
  • Elena Voita, Rico Sennrich, Ivan Titov
  • TLDR: We propose a new method for evaluating the relative contribution of source and target tokens in neural machine translation models.
• Comparing Test Sets with Item Response Theory
  • Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, Samuel R. Bowman
  • TLDR: We show that the datasets used for NLP training are effective at distinguishing between strong and weak models, but are not able to detect future improvements.
• Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning
  • Forrest Davis, Marten van Schijndel
  • TLDR: We show that competing processes in a language act as constraints on model behavior and demonstrate that targeted fine-tuning can re-weight the learned constraints, uncovering otherwise dormant linguistic knowledge in models.
• More Identifiable yet Equally Performant Transformers for Text Classification
  • Rishabh Bhardwaj, Navonil Majumder, Soujanya Poria, Eduard Hovy
  • TLDR: We provide theoretical analysis and empirical observations on the identifiability of attention weights.
• AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation
  • Xinnuo Xu, Guoyin Wang, Young-Bum Kim, Sungjin Lee
  • TLDR: We propose AugNLG, a novel data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model, to automatically create MR-to-Text data from open-domain texts.
• Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children’s mindreading ability
  • Venelin Kovatchev, Phillip Smith, Mark Lee, Rory Devine
  • TLDR: We present a new state-of-the-art algorithm for automatic scoring of children’s ability to understand others’ thoughts, feelings, and desires (or “mindreading”).
• A Dataset and Baselines for Multilingual Reply Suggestion
  • Mozhi Zhang, Wei Wang, Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, Ahmed Hassan Awadallah
  • TLDR: We present MRS, a multilingual reply suggestion dataset with ten languages.
• What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?
  • Nikita Nangia, Saku Sugawara, Harsh Trivedi, Alex Warstadt, Clara Vania, Samuel R. Bowman
  • TLDR: We show that using crowdsourcing to improve NLU example difficulty is not as effective as asking workers to explain their examples.
• Align Voting Behavior with Public Statements for Legislator Representation Learning
  • Xinyi Mou, Zhongyu Wei, Lei Chen, Shangyi Ning, Yancheng He, Changjian Jiang, Xuanjing Huang
  • TLDR: We propose a novel graph-based model for modeling the ideology of legislators on Twitter.
• Measure and Evaluation of Semantic Divergence across Two Languages
  • Syrielle Montariol, Alexandre Allauzen
  • TLDR: We propose to track the evolution of a word and its translation across two languages and provide qualitative insight for the task.
• Improving Zero-Shot Translation by Disentangling Positional Information
  • Danni Liu, Jan Niehues, James Cross, Francisco Guzmán, Xian Li
  • TLDR: We show that removing residual connections in the encoder layer leads to language-independent representations in neural machine translation.
• Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning
  • Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, Xiang Ren
  • TLDR: We propose Mickey Probe, a language-general probing task for fairly evaluating the common sense of popular ML-LMs across different languages.
• Attention Calibration for Transformer in Neural Machine Translation
  • Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, Mu Li
  • TLDR: We propose to calibrate the attention weights by introducing a mask perturbation model that automatically evaluates each input’s contribution to the model outputs.
• Diverse Pretrained Context Encodings Improve Document Translation
  • Domenic Donato, Lei Yu, Chris Dyer
  • TLDR: We propose a new architecture for adapting a sentence-level sequence-to-sequence transformer by incorporating multiple pre-trained document context signals and assess the impact on translation performance of (1) different pretraining approaches for generating these signals, (2) the quantity of parallel data for which document context is available, and (3) conditioning on source, target, or source and target contexts.
• Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study
  • Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi
  • TLDR: We propose RelateLM, a novel approach to adapt multilingual language models to low web-resource languages by exploiting relatedness among languages in a language family.
• On Finding the K-best Non-projective Dependency Trees
  • Ran Zmigrod, Tim Vieira, Ryan Cotterell
  • TLDR: We provide a new algorithm for decoding the K-best spanning tree of a graph which is not subject to the root constraint of dependency trees.
• Towards Argument Mining for Social Good: A Survey
  • Eva Maria Vecchi, Neele Falk, Iman Jundi, Gabriella Lapesa
  • TLDR: We propose a novel definition of argument quality which is integrated with that of deliberative quality from the Social Science literature.
• Automated Generation of Storytelling Vocabulary from Photographs for use in AAC
  • Mauricio Fontana de Vargas, Karyn Moffatt
  • TLDR: We present a novel method for generating context-related vocabulary from photographs of personally relevant events aimed at supporting people with language impairments in retelling their past experiences.
• CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes
  • James Mullenbach, Yada Pruksachatkun, Sean Adler, Jennifer Seale, Jordan Swartz, Greg McKelvey, Hui Dai, Yi Yang, David Sontag
  • TLDR: We present a dataset of clinical action items annotated over MIMIC-III, the largest publicly available dataset of real clinical notes.
• Assessing Emoji Use in Modern Text Processing Tools
  • Abu Awal Md Shoeb, Gerard de Melo
  • TLDR: We investigate the ability of prominent NLP and text processing tools to handle text containing emojis.
• Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention
  • Wasi Ahmad, Xiao Bai, Soomin Lee, Kai-Wei Chang
  • TLDR: We present a novel approach to keyphrase generation.
• Factorising Meaning and Form for Intent-Preserving Paraphrasing
  • Tom Hosking, Mirella Lapata
  • TLDR: We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form.
• AggGen: Ordering and Aggregating while Generating
  • Xinnuo Xu, Ondřej Dušek, Verena Rieser, Ioannis Konstas
  • TLDR: We present AggGen (pronounced ‘again’) a data-to-text model which re-introduces two explicit sentence planning stages into neural data- to-text systems: input ordering and input aggregation.
• Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models
  • Peter West, Ximing Lu, Ari Holtzman, Chandra Bhagavatula, Jena D. Hwang, Yejin Choi
  • TLDR: We present Reflective Decoding, a novel unsupervised algorithm that allows for direct application of unidirectional LMs to non-sequential tasks.
• Towards Table-to-Text Generation with Numerical Reasoning
  • Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura, Hiroya Takamura
  • TLDR: We propose a novel table-to-text generation framework that uses pairs of table and paragraph of a table description to generate fluent text that is enriched with numerical reasoning.
• BACO: A Background Knowledge- and Content-Based Framework for Citing Sentence Generation
  • Yubin Ge, Ly Dinh, Xiaofeng Liu, Jinsong Su, Ziyao Lu, Ante Wang, Jana Diesner
  • TLDR: We present BACO, a BAckground knowledge- and COntent-based framework for citing sentence generation, which considers two types of information: (1) background knowledge by leveraging structural information from a citation network; and (2) content, which represents in-depth information about what to cite and why to cite.
• Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization
  • Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, Ting Liu
  • TLDR: We present a novel dialogue summarization system that uses pre-trained dialog background knowledge to label and annotate text.
• Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval
  • Akari Asai, Eunsol Choi
  • TLDR: We analyze why answering information-seeking queries is more challenging and where their prevalent unanswerabilities arise, on Natural Questions and TyDi QA.
• A Gradually Soft Multi-Task and Data-Augmented Approach to Medical Question Understanding
  • Khalil Mrini, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole
  • TLDR: We propose a novel Multi-Task Learning method with data augmentation for medical question understanding.
• Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification
  • Rami Aly, Andreas Vlachos, Ryan McDonald
  • TLDR: We present novel architectures for zero-shot NERC, which leverage the fact that textual descriptions for many entity classes occur naturally.
• MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition
  • Shuang Wu, Xiaoning Song, Zhenhua Feng
  • TLDR: We propose a novel Multi-metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition by fusing the structural information of Chinese characters.
• Factuality Assessment as Modal Dependency Parsing
  • Jiarui Yao, Haoling Qiu, Jin Zhao, Bonan Min, Nianwen Xue
  • TLDR: We develop a novel modal dependency parsing task that identifies the events and their sources, formally known as conceivers, and then determine the level of certainty that the sources are asserting with respect to the events.
• Directed Acyclic Graph Network for Conversational Emotion Recognition
  • Weizhou Shen, Siyue Wu, Yunyi Yang, Xiaojun Quan
  • TLDR: We propose a novel idea of encoding the utterances with a directed acyclic graph (DAG) to better model the intrinsic structure within a conversation, and design a directed a directed neural network, namely DAG-ERC, to implement this idea.
• Improving Formality Style Transfer with Context-Aware Rule Injection
  • Zonghai Yao, Hong Yu
  • TLDR: We propose a new method, CARI, to integrate rules for pre-trained language models.
• Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection
  • Lixing Zhu, Gabriele Pergola, Lin Gui, Deyu Zhou, Yulan He
  • TLDR: We propose a Topic-Driven Knowledge-Aware Transformer for dialogue emotion detection.
• Syntopical Graphs for Computational Argumentation Tasks
  • Joe Barrow, Rajiv Jain, Nedim Lipka, Franck Dernoncourt, Vlad Morariu, Varun Manjunatha, Douglas Oard, Philip Resnik, Henning Wachsmuth
  • TLDR: We introduce syntopical graphs for linking claims within a collection and demonstrate state-of-the-art performance on stance detection and aspect detection tasks.
• Stance Detection in COVID-19 Tweets
  • Kyle Glandt, Sarthak Khanal, Yingjie Li, Doina Caragea, Cornelia Caragea
  • TLDR: We present a new stance detection dataset for the COVID-19 pandemic, which is used to train several established stance detection models.
• Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification
  • Jiasheng Si, Deyu Zhou, Tongzhe Li, Xingyu Shi, Yulan He
  • TLDR: We propose a novel topic-aware evidence reasoning and stance-aware aggregation model for more accurate fact verification.
• Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset
  • Alexandra Ils, Dan Liu, Daniela Grunow, Steffen Eger
  • TLDR: We use the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic.
• Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions
  • Dorottya Demszky, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, Tatsunori Hashimoto
  • TLDR: We propose a new measure for measuring and improving teachers’ uptake of student contributions in conversation, which is capable of identifying a wider range of uptake phenomena like question answering and reformulation.
• A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies
  • A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock, Almeida Jacqueline Toribio
  • TLDR: We provide a survey of code-switching (C-S) covering the literature in linguistics with a reflection on the key issues in language technologies.
• Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection
  • Bertie Vidgen, Tristan Thrush, Zeerak Waseem, Douwe Kiela
  • TLDR: We present a human-and-model-in-in the-loop process for dynamically generating datasets and training better performing and more robust hate detection models.
• InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection
  • Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, Avi Sil
  • TLDR: We present a novel benchmark for fake news detection at the knowledge element level, as well as a solution for this task which incorporates cross-media consistency checking to detect the fine-grained knowledge elements making news articles misinformative.
• I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling
  • Yixin Nie, Mary Williamson, Mohit Bansal, Douwe Kiela, Jason Weston
  • TLDR: We present a new dialogue contradiction detection task and a new conversational dataset that provides supervision for dialogue contradiction discovery and show that it correlates well with human judgments and improves the consistency of state-of-the-art generative chatbots.
• A Sequence-to-Sequence Approach to Dialogue State Tracking
  • Yue Feng, Yang Wang, Hang Li
  • TLDR: We propose a new approach to dialogue state tracking, referred to as Seq2Seq-DU, which formalizes DST as a sequence-to-sequence problem.
• Discovering Dialog Structure Graph for Coherent Dialog Generation
  • Jun Xu, Zeyang Lei, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che
  • TLDR: We present an unsupervised model for learning discrete dialog structure graph from chitchat corpora and use it to facilitate coherent dialog generation in downstream systems.
• Dialogue Response Selection with Hierarchical Curriculum Learning
  • Yixuan Su, Deng Cai, Qingyu Zhou, Zibo Lin, Simon Baker, Yunbo Cao, Shuming Shi, Nigel Collier, Yan Wang
  • TLDR: We propose a hierarchical curriculum learning framework for dialogue response selection that improves the model performance across various evaluation metrics.
• A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech
  • Jingxuan Yang, Kerui Xu, Jun Xu, Si Li, Sheng Gao, Jun Guo, Nianwen Xue, Ji-Rong Wen
  • TLDR: We present a neural model for joint dropped pronoun recovery and conversational discourse parsing in Chinese conversational speech.
• A Systematic Investigation of KB-Text Embedding Alignment at Scale
  • Vardaan Pahuja, Yu Gu, Wenhu Chen, Mehdi Bahrami, Lei Liu, Wei-Peng Chen, Yu Su
  • TLDR: We investigate the role of KB-text embedding in joint reasoning and show how to leverage complementary knowledge to improve link prediction and analogical reasoning.
• Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
  • Haoming Jiang, Danqing Zhang, Tianyu Cao, Bing Yin, Tuo Zhao
  • TLDR: We propose a new multi-stage computational framework for deep NER training that effectively suppress the noise of the weak labels and outperforms existing methods.
• Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model
  • Hongliang Dai, Yangqiu Song, Haixun Wang
  • TLDR: We propose to obtain training data for ultra-fine entity typing by using a BERT Masked Language Model (MLM) and use it to generate hypernyms of the mention in a sentence.
• Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning
  • Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
  • TLDR: We propose a novel approach to find external contexts of a sentence by retrieving and selecting a set of semantically relevant texts through a search engine, with the original sentence as the query.
• Implicit Representations of Meaning in Neural Language Models
  • Belinda Z. Li, Maxwell Nye, Jacob Andreas
  • TLDR: We identify contextual word representations that function as models of entities and situations as they evolve throughout a discourse.
• Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models
  • Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, Yonatan Belinkov
  • TLDR: We investigate the magnitude of language models’ preferences for grammatical inflections, as well as whether neurons process subject-verb agreement similarly across sentences with different syntactic structures.
• Bird’s Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approach
  • Yifan Hou, Mrinmaya Sachan
  • TLDR: We propose a new information-theoretic probe method for detecting if and how contextualized text representations encode the information in linguistic graphs.
• Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
  • Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, Jin Xu
  • TLDR: We investigate the underlying predicting mechanisms of MLMs over different extraction paradigms and show that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts.
• Poisoning Knowledge Graph Embeddings via Relation Inference Patterns
  • Peru Bhardwaj, John Kelleher, Luca Costabello, Declan O’Sullivan
  • TLDR: We propose to exploit their inductive abilities which are captured through the relationship patterns like symmetry, inversion and composition in the knowledge graph.
• Bad Seeds: Evaluating Lexical Methods for Bias Measurement
  • Maria Antoniak, David Mimno
  • TLDR: We describe the different types of social biases and linguistic features encoded in the seeds of corpora and show how they affect subsequent bias measurements.
• A Survey of Race, Racism, and Anti-Racism in NLP
  • Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
  • TLDR: We survey 79 papers from the ACL anthology that mention race in NLP and show various types of race-related bias in all stages of NLP model development, highlighting the need for proactive consideration of how NLP systems can uphold racial hierarchies.
• Intrinsic Bias Metrics Do Not Correlate with Application Bias
  • Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya, Adam Lopez
  • TLDR: We show that intrinsic and extrinsic metrics for NLP are not correlated, and suggest that debiasing systems learn harmful societal biases that amplify inequality as they are deployed in more and more situations.
• RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models
  • Soumya Barikeri, Anne Lauscher, Ivan Vulić, Goran Glavaš
  • TLDR: We present REDDITBIAS, a conversational data set grounded in the actual human conversations from Reddit, allowing for bias measurement and mitigation across four important bias dimensions: gender,race,religion, and queerness.
• Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks
  • Weicheng Ma, Kai Zhang, Renze Lou, Lili Wang, Soroush Vosoughi
  • TLDR: We show that pruning a number of attention heads in a multi-lingual Transformer-based model has, in general, positive effects on its performance in cross-lingUAL and multi-language Transformer tasks and identify the attention heads to be pruned.
• Crafting Adversarial Examples for Neural Machine Translation
  • Xinze Zhang, Junzhe Zhang, Zhenhua Chen, Kun He
  • TLDR: We propose a novel method to craft NMT adversarial examples that can effectively attack the state-of-the-art NMT models with small perturbations.
• UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP
  • M Saiful Bari, Tasnim Mohiuddin, Shafiq Joty
  • TLDR: We propose a novel unsupervised data augmentation framework for zero-resource transfer learning scenarios.
• Glancing Transformer for Non-Autoregressive Neural Machine Translation
  • Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei Li
  • TLDR: We propose the Glancing Language Model (GLM) for single-pass parallel generation models for machine translation.
• Hierarchical Context-aware Network for Dense Video Event Captioning
  • Lei Ji, Xianglin Guo, Haoyang Huang, Xilin Chen
  • TLDR: We present a novel Hierarchical Context-aware Network for dense video event captioning (HCN) to capture context from various aspects.
• Control Image Captioning Spatially and Temporally
  • Kun Yan, Lei Ji, Huaishao Luo, Ming Zhou, Nan Duan, Shuai Ma
  • TLDR: We propose a novel model for image captioning that learns to attend to the correct visual objects under heuristic attention guidance.
• Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation
  • Jeff Da, Maxwell Forbes, Rowan Zellers, Anthony Zheng, Jena D. Hwang, Antoine Bosselut, Yejin Choi
  • TLDR: We present a new formalism to understand visual media manipulation as structured annotations with respect to the intents, emotional reactions, attacks on individuals, and the overall implications of disinformation.
• PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World
  • Rowan Zellers, Ari Holtzman, Matthew Peters, Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi, Yejin Choi
  • TLDR: We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language.
• Modeling Fine-Grained Entity Types with Box Embeddings
  • Yasumasa Onoe, Michael Boratko, Andrew McCallum, Greg Durrett
  • TLDR: We propose a novel type-based entity typing model that captures the latent type hierarchies of types even when these relationships are not defined explicitly in the ontology.
• ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
  • Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, Jiwei Li
  • TLDR: We propose ChineseBERT, which incorporates both the glyph and pinyin for language understanding.
• Weight Distillation: Transferring the Knowledge in Neural Network Parameters
  • Ye Lin, Yanyang Li, Ziyang Wang, Bei Li, Quan Du, Tong Xiao, Jingbo Zhu
  • TLDR: We propose Weight Distillation to transfer the knowledge in parameters of a large neural network to a small neural network through a parameter generator.
• Optimizing Deeper Transformers on Small Datasets
  • Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Simon J.D. Prince, Yanshuai Cao
  • TLDR: We show that training deep transformers from scratch with proper initialization and optimization can improve generalization on small datasets, including Text-to-SQL parsing and logical reading comprehension.
• BERTAC: Enhancing Transformer-based Language Models with Adversarially Pretrained Convolutional Neural Networks
  • Jong-Hoon Oh, Ryu Iida, Julien Kloetzer, Kentaro Torisawa
  • TLDR: We pretrain a CNN using Wikipedia data and Wikipedia data, and then integrate it with standard TLMs.
• COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic
  • Arkadiy Saakyan, Tuhin Chakrabarty, Smaranda Muresan
  • TLDR: We present a dataset of 4,086 claims concerning the COVID-19 pandemic and present a method for detecting false claims and generating counter-claims.
• Explaining Relationships Between Scientific Documents
  • Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, Noah A. Smith
  • TLDR: We address the task of explaining relationships between two scientific documents using natural language text.
• IrEne: Interpretable Energy Prediction for Transformers
  • Qingqing Cao, Yash Kumar Lal, Harsh Trivedi, Aruna Balasubramanian, Niranjan Balasubramanian
  • TLDR: We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models.
• Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach
  • Lu Cheng, Ahmadreza Mosallanezhad, Yasin Silva, Deborah Hall, Huan Liu
  • TLDR: We show that social biases in existing cyberbullying datasets can be exploited to bias the classification of cyberbullies.
• PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context
  • Xinyun Chen, Linyuan Gong, Alvin Cheung, Dawn Song
  • TLDR: We propose a new hierarchical encoder-decoder architecture for visualization programs that learns to predict the plot type and the input utterance of the visualization programs.
• Changing the World by Changing the Data
  • Anna Rogers
  • TLDR: We argue that fundamentally the point is moot: curation already is and will be happening, and it is changing the world.
• EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  • Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang, Jingjing Liu
  • TLDR: We propose a general computationally-efficient training algorithm applicable to both pre-training and fine-tuning of large-scale language models.
• On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation
  • Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, Luo Si
  • TLDR: We show that adapter-based tuning is more robust to overfitting and less sensitive to changes in learning rates than fine-tuning.
• Data Augmentation for Text Generation Without Any Augmented Data
  • Wei Bi, Huayang Li, Jiacheng Huang
  • TLDR: We propose a new objective to formulate the problem of data augmentation on text generation tasks without any use of augmented data constructed by specific mapping functions.
• Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval
  • Zijing Ou, Qinliang Su, Jianxing Yu, Bang Liu, Jingwen Wang, Ruihui Zhao, Changyou Chen, Yefeng Zheng
  • TLDR: We propose a graph-driven generative model for document hashing that preserves semantic and neighborhood information simultaneously.
• SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis
  • Joshua Feinglass, Yezhou Yang
  • TLDR: Typicality is a new formulation of evaluation for visual captioning that captures both style and grammar, and provides a new metric for evaluating captioner fluency.
• KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers
  • Chia-Hsuan Lee, Oleksandr Polozov, Matthew Richardson
  • TLDR: We present a new cross-domain evaluation dataset for text-to-SQL parsing and show that it improves the accuracy of existing zero-shot parsers by over 13.2%, doubling their performance.
• QASR: QCRI Aljazeera Speech Resource A Large Scale Annotated Arabic Speech Corpus
  • Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, Ahmed Ali
  • TLDR: We introduce the largest transcribed Arabic speech corpus, QASR, collected from the broadcast domain.
• An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models
  • Xueqing Liu, Chi Wang
  • TLDR: We investigate the performance of modern hyperparameter optimization methods (HPO) on fine-tuning pre-trained language models.
• Better than Average: Paired Evaluation of NLP systems
  • Maxime Peyrard, Wei Zhao, Steffen Eger, Robert West
  • TLDR: We show that the choice of aggregation mechanism matters and show that pairwise evaluation is better than averaging.
• Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL
  • Jiaqi Guo, Ziliang Si, Yu Wang, Qian Liu, Ming Fan, Jian-Guang Lou, Zijiang Yang, Ting Liu
  • TLDR: We present Chase, a large-scale and pragmatic Chinese dataset for XDTS, which highlights the challenging problems of XDTS.
• CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding
  • Dong Wang, Ning Ding, Piji Li, Haitao Zheng
  • TLDR: We propose Contrastive Learning with semantIc Negative Examples (CLINE), which constructs semantic negative examples unsupervised to improve the robustness under semantically adversarial attacking.
• Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference
  • Ziye Chen, Cheng Ding, Zusheng Zhang, Yanghui Rao, Haoran Xie
  • TLDR: We develop a tree-structured topic model by leveraging nonparametric neural variational inference.
• ExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning
  • Li Du, Xiao Ding, Kai Xiong, Ting Liu, Bing Qin
  • TLDR: We present an annotated causal event pair that provides evidence for the causality of events.
• Distributed Representations of Emotion Categories in Emotion Space
  • Xiangyu Wang, Chengqing Zong
  • TLDR: We propose a general framework to learn the distributed representations for emotion categories in emotion space from a given emotion classification dataset.
• Style is NOT a single variable: Case Studies for Cross-Stylistic Language Understanding
  • Dongyeop Kang, Eduard Hovy
  • TLDR: We propose a new cross-style language understanding benchmark and a novel cross-classifier for sentence-level cross-language understanding and evaluation.
• DynaSent: A Dynamic Benchmark for Sentiment Analysis
  • Christopher Potts, Zhengxuan Wu, Atticus Geiger, Douwe Kiela
  • TLDR: We introduce DynaSent, a new English-language benchmark task for ternary (positive/negative/neutral) sentiment analysis.
• A Hierarchical VAE for Calibrating Attributes while Generating Text using Normalizing Flow
  • Bidisha Samanta, Mohit Agrawal, NIloy Ganguly
  • TLDR: We propose a hierarchical architecture for text-style transfer, preserving content using attribute dis- entanglement.
• A Unified Generative Framework for Aspect-based Sentiment Analysis
  • Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, Zheng Zhang
  • TLDR: We propose a unified generative formulation for aspect-based sentiment analysis, which allows us to solve all ABSA subtasks in an end-to-end framework.
• Discovering Dialogue Slots with Weak Supervision
  • Vojtěch Hudeček, Ondřej Dušek, Zhou Yu
  • TLDR: We propose a method for automatic slot tagging without labeled training data.
• Enhancing the generalization for Intent Classification and Out-of-Domain Detection in SLU
  • Yilin Shen, Yen-Chang Hsu, Avik Ray, Hongxia Jin
  • TLDR: We propose a novel domain-regularized module for domain-based intent classification and OOD detection that achieves state-of-the-art performance against existing approaches and the strong baselines we created for the comparisons.
• PROTAUGMENT: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning
  • Thomas Dopierre, Christophe Gravier, Wilfried Logerais
  • TLDR: We propose ProtAugment, a meta-learning algorithm for few-shot intent detection, that limits overfitting on the bias introduced by the few-shots classification objective at each episode.
• Robustness Testing of Language Understanding in Task-Oriented Dialog
  • Jiexi Liu, Ryuichi Takanobu, Jiaxin Wen, Dazhen Wan, Hongguang Li, Weiran Nie, Cheng Li, Wei Peng, Minlie Huang
  • TLDR: We present a robustness evaluation and analysis of natural language understanding models in task-oriented dialog systems, and introduce three important aspects related to language understanding in real-world dialog systems.
• Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking?
  • Puhai Yang, Heyan Huang, Xian-Ling Mao
  • TLDR: We study and discuss how the context information of different granularities affect dialogue state tracking.
• OTTers: One-turn Topic Transitions for Open-Domain Dialogue
  • Karin Sevegnani, David M. Howcroft, Ioannis Konstas, Verena Rieser
  • TLDR: We present a new dataset of human one-turn topic transitions and show that bridging utterances are the most effective way to connect two topics in a cooperative and coherent manner.
• Towards Robustness of Text-to-SQL Models against Synonym Substitution
  • Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang
  • TLDR: We investigate the robustness of text-to-SQL models to synonym substitution and propose two new approaches to improve the model robustness.
• KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference
  • Qianglong Chen, Feng Ji, Xiangji Zeng, Feng-Lin Li, Ji Zhang, Haiqing Chen, Yin Zhang
  • TLDR: We propose a novel contrastive explanation approach in NLI based on contrastive counterfactual examples.
• Self-Guided Contrastive Learning for BERT Sentence Representations
  • Taeuk Kim, Kang Min Yoo, Sang-goo Lee
  • TLDR: We propose a contrastive learning objective for improving the quality of BERT sentence representations.
• LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations
  • Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, Kai Yu
  • TLDR: We propose a novel graph-based text-to-SQL model that uses the topology of the graph to encode the underlying relational features without constructing meta-paths.
• Multi-stage Pre-training over Simplified Multimodal Pre-training Models
  • Tongtong Liu, Fangxiang Feng, Xiaojie Wang
  • TLDR: We propose a new Multi-stage Pre-training method for multimodal pre-training models, which uses information at different granularities from word, phrase to sentence in both texts and images to pre-train a model in stages.
• Beyond Sentence-Level End-to-End Speech Translation: Context Helps
  • Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich
  • TLDR: We propose a novel context-aware speech translation model for document-based machine translation, which improves quality and robustness to segmentation errors.
• LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding
  • Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou
  • TLDR: We propose a new multi-modal Transformer architecture for pre-training text and layout in a single multi-mode framework for visually-rich document understanding tasks.
• UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning
  • Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, Haifeng Wang
  • TLDR: We propose a novel pre-training architecture for both single-modal and multi-modality understanding and generation tasks.
• Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities
  • Jinming Zhao, Ruichen Li, Qin Jin
  • TLDR: We propose a unified model, Missing Modality Imagination Network (MMIN), to deal with the uncertain missing modality problem.
• Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders
  • Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu
  • TLDR: We propose a new method for end-to-end Speech Translation that achieves comparable or even better BLEU performance than the cascaded ST counterpart when large-scale ASR and MT data is available.
• N-ary Constituent Tree Parsing with Recursive Semi-Markov Model
  • Xin Xin, Jinlong Li, Zeqi Tan
  • TLDR: We propose a novel graph-based framework for graph- based constituent parsing in the setting that binarization is not conducted as a pre-processing step, where a constituent tree may consist of nodes with more than two children.
• Automated Concatenation of Embeddings for Structured Prediction
  • Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
  • TLDR: Automated Concatenation of Embeddings for structured prediction tasks.
• Multi-View Cross-Lingual Structured Prediction with Minimum Supervision
  • Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
  • TLDR: We propose a multi-view framework for structured prediction tasks that learns from multiple source languages and transfer it to a target view based on a task-specific model.
• The Limitations of Limited Context for Constituency Parsing
  • Yuchen Li, Andrej Risteski
  • TLDR: We show that with limited context (either bounded, or unidirectional), there are PCFGs, for which these approaches cannot represent the max-likelihood parse; conversely, if the context is unlimited, they can represent the maximum-like likelihood parse of any PCFG.
• Neural Bi-Lexicalized PCFG Induction
  • Songlin Yang, Yanpeng Zhao, Kewei Tu
  • TLDR: We propose a new approach to parameterize neural lexicalized PCFGs without making implausible independence assumptions.
• Ruddit: Norms of Offensiveness for English Reddit Comments
  • Rishav Hada, Sohi Sudhir, Pushkar Mishra, Helen Yannakoudakis, Saif M. Mohammad, Ekaterina Shutova
  • TLDR: We present a dataset of English language Reddit comments that has fine-grained, real-valued scores between -1 (maximally supportive) and 1 (maximalally offensive) and show that the dataset produces highly reliable offensiveness scores.
• Towards Quantifiable Dialogue Coherence Evaluation
  • Zheng Ye, Liucun Lu, Lishan Huang, Liang Lin, Xiaodan Liang
  • TLDR: We propose Quantifiable Dialogue Coherence Evaluation, a novel framework aiming to train a quantifiable dialogue coherence metric that can reflect the actual human rating standards.
• Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels
  • Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, Marco Idiart, Aline Villavicencio
  • TLDR: We present a new dataset for annotating noun compounds in English and Portuguese at both type and token levels, and show that the tokens are better than type for capturing idiomaticity.
• Factoring Statutory Reasoning as Language Understanding Challenges
  • Nils Holzenberger, Benjamin Van Durme
  • TLDR: We decompose statutory reasoning into four types of language-understanding challenge problems, through the introduction of concepts and structure found in Prolog programs.
• Evaluating Evaluation Measures for Ordinal Classification and Ordinal Quantification
  • Tetsuya Sakai
  • TLDR: We provide a new set of evaluation measures for both Ordinal Classification and Ordinal Quantification tasks that take the ordinal nature of the classes into account.
• Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making
  • Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai
  • TLDR: We propose a novel EM framework that consists of Heterogeneous information fusion and Key Attribute Tree Induction to decouple feature representation from matching decision.
• Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition
  • Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, Weiming Lu
  • TLDR: We propose a two-stage entity identifier for named entity recognition that can handle nested NER.
• Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction
  • Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, Shaoyi Chen
  • TLDR: We propose Text2Event, a sequence-to-structure generation paradigm that can directly extract events from the text in an end-to end manner.
• A Large-Scale Chinese Multimodal NER Dataset with Speech Clues
  • Dianbo Sui, Zhengkun Tian, Yubo Chen, Kang Liu, Jun Zhao
  • TLDR: We propose a family of strong and representative multimodal named entity recognition models, which leverage textual features or multimodality features.
• A Neural Transition-based Joint Model for Disease Named Entity Recognition and Normalization
  • Zongcheng Ji, Tian Xia, Mei Han, Jing Xiao
  • TLDR: We propose a novel neural transition-based joint model for disease recognition and normalization task.
• OntoED: Low-resource Event Detection with Ontology Embedding
  • Shumin Deng, Ningyu Zhang, Luoqiu Li, Chen Hui, Tou Huaixiao, Mosha Chen, Fei Huang, Huajun Chen
  • TLDR: We propose a novel event detection framework that leverages and propagates correlation knowledge from event ontology to event types.
• Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation
  • Wenxiang Jiao, Xing Wang, Zhaopeng Tu, Shuming Shi, Michael Lyu, Irwin King
  • TLDR: We propose to improve the sampling procedure for self-training by selecting the most informative monolingual sentences to complement the parallel data.
• Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training
  • Linqing Chen, Junhui Li, Zhengxian Gong, Boxing Chen, Weihua Luo, Min Zhang, Guodong Zhou
  • TLDR: We propose a novel approach to improve context-aware neural machine translation by taking advantage of the availability of both large-scale sentence-level parallel dataset and source-side monolingual documents.
• Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
  • Yang Feng, Shuhao Gu, Dengji Guo, Zhengxin Yang, Chenze Shao
  • TLDR: We introduce another decoder, called seer decoder and use knowledge distillation to simulate the behaviors of the seer, and use it to train the conventional decoder to perform like the sener decoder.
• Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?
  • Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, Marco Turchi
  • TLDR: We present a systematic comparison of state-of-the-art systems representative of the two paradigms in speech translation and show that the gap between the two is now closed.
• Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning
  • Cheonbok Park, Yunwon Tae, TaeHee Kim, Soyoung Yang, Mohammad Azam Khan, Lucy Park, Jaegul Choo
  • TLDR: Meta-learning algorithm for unsupervised neural machine translation.
• Lightweight Cross-Lingual Sentence Representation Learning
  • Zhuoyuan Mao, Prakhar Gupta, Chenhui Chu, Martin Jaggi, Sadao Kurohashi
  • TLDR: We propose a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations.
• ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
  • SiYu Ding, Junyuan Shang, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
  • TLDR: We propose ERNIE-Doc, a document-level language pretraining model based on Recurrence Transformers.
• Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation
  • Yuanxin Liu, Fandong Meng, Zheng Lin, Weiping Wang, Jie Zhou
  • TLDR: We show that the rich information contained in the hidden layers of BERT is conducive to the student’s performance.
• Rational LAMOL: A Rationale-based Lifelong Learning Framework
  • Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, Peerapon Vateekul
  • TLDR: We propose Rational LAMOL, a novel end-to-end LL framework for language models.
• EnsLM: Ensemble Language Model for Data Diversity by Semantic Clustering
  • Zhibin Duan, Hao Zhang, Chaojie Wang, Zhengjue Wang, Bo Chen, Mingyuan Zhou
  • TLDR: We propose a new ensemble language model for NLP that learns the shared knowledge among diverse samples.
• LeeBERT: Learned Early Exit for BERT with cross-level optimization
  • Wei Zhu
  • TLDR: We propose a novel training scheme for BERT that learns from each exit of the model to improve the performance of the state-of-the-art early exit methods.
• Unsupervised Extractive Summarization-Based Representations for Accurate and Explainable Collaborative Filtering
  • Reinald Adrian Pugoy, Hung-Yu Kao
  • TLDR: We propose a novel extractive summarization-based collaborative filtering model that unifies representation and explanation.
• PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction
  • Shulin Liu, Tao Yang, Tianchi Yue, Feng Zhang, Di Wang
  • TLDR: We propose a Pre-trained masked Language model with Misspelled knowledgE (PLOME) for CSC, which jointly learns how to understand language and correct spelling errors.
• Competence-based Multimodal Curriculum Learning for Medical Report Generation
  • Fenglin Liu, Shen Ge, Xian Wu
  • TLDR: We propose a Competence-based Multimodal Curriculum Learning framework for medical report generation task.
• Learning Syntactic Dense Embedding with Correlation Graph for Automatic Readability Assessment
  • Xinying Qiu, Yuan Chen, Hanwu Chen, Jian-Yun Nie, Yuming Shen, Dawei Lu
  • TLDR: We propose to incorporate linguistic features into neural network models by learning syntactic dense embeddings based on linguistic features.
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
  • Haojie Pan, Chengyu Wang, Minghui Qiu, Yichang Zhang, Yaliang Li, Jun Huang
  • TLDR: We propose a meta-teacher model that captures transferable knowledge across domains and passes such knowledge to students.
• A Semantic-based Method for Unsupervised Commonsense Question Answering
  • Yilin Niu, Fei Huang, Jiaming Liang, Wenkai Chen, Xiaoyan Zhu, Minlie Huang
  • TLDR: We present a novel SEmantic-based Question Answering method for unsupervised commonsense question answering.
• Explanations for CommonsenseQA: New Dataset and Models
  • Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
  • TLDR: We present a new dataset for common-sense question answering task and propose a new explanation for each question.
• Few-Shot Question Answering by Pretraining Span Selection
  • Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy
  • TLDR: We propose a new pretraining scheme tailored for question answering: recurring span selection.
• UnitedQA: A Hybrid Approach for Open Domain Question Answering
  • Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
  • TLDR: We present a novel approach for combining extractive and generative readers for open-domain QA.
• Database reasoning over text
  • James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy
  • TLDR: We propose a modular architecture to answer database-style queries over multiple spans from text and aggregating these at scale.
• Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort
  • Vânia Mendonça, Ricardo Rei, Luisa Coheur, Alberto Sardinha, Ana Lúcia Santos
  • TLDR: We propose a novel online approach to evaluating the quality of Machine Translation systems by taking advantage of the human feedback available.
• How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
  • Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych
  • TLDR: We provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolandual task performance.
• Evaluating morphological typology in zero-shot cross-lingual transfer
  • Antonio Martínez-García, Toni Badia, Jeremy Barnes
  • TLDR: We address what effects morphological typology has on zero-shot cross-lingual transfer for two tasks: Part-of-speech tagging and sentiment analysis.
• From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text
  • Ishan Tarunesh, Syamantak Kumar, Preethi Jyothi
  • TLDR: We present a novel method for generating Hindi-English code-switched sentences from monolingual Hindi sentences.
• Fast and Accurate Neural Machine Translation with Translation Memory
  • Qiuxiang He, Guoping Huang, Qu Cui, Li Li, Lemao Liu
  • TLDR: We propose a fast and accurate approach to TM-based neural machine translation that significantly surpasses several strong baselines that use multiple TMs, in terms of BLEU and running time.
• Annotating Online Misogyny
  • Philine Zeinert, Nanna Inie, Leon Derczynski
  • TLDR: We present a comprehensive taxonomy of labels for annotating misogyny in natural written language, and a high-quality dataset of annotated posts sampled from social media posts.
• Few-NERD: A Few-shot Named Entity Recognition Dataset
  • Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, Zhiyuan Liu
  • TLDR: We present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine-grain entity types.
• MultiMET: A Multimodal Dataset for Metaphor Understanding
  • Dongyu Zhang, Minghao Zhang, Heting Zhang, Liang Yang, Hongfei Lin
  • TLDR: We present a multimodal metaphor dataset for understanding and expressing metaphor.
• Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech
  • Margherita Fanton, Helena Bonaldi, Serra Sinem Tekiroğlu, Marco Guerini
  • TLDR: We propose a novel human-in-the-loop data collection methodology for hate speech / counter narrative generation and show that it is scalable and cost-effective.
• Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA?
  • Cunxiang Wang, Pai Liu, Yue Zhang
  • TLDR: We present a new dataset of closed-book QA using SQuAD and show that it is challenging for BART to remember training facts in high precision, and also challenging to answer closed-books questions even if relevant knowledge is retained.
• Joint Models for Answer Verification in Question Answering Systems
  • Zeyu Zhang, Thuy Vu, Alessandro Moschitti
  • TLDR: We propose a new neural architecture for answer sentence selection modules that can exploit the interrelated information between pair of answers.
• Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction
  • Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
  • TLDR: We present a novel model that adaptively predicts a single answer or a set of question-answer pairs for ambiguous open-domain questions.
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance
  • Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua
  • TLDR: We present a novel QA model for both tabular and textual data, which is capable of reasoning over both tables and text.
• Modeling Transitions of Focal Entities for Conversational Knowledge Base Question Answering
  • Yunshi Lan, Jing Jiang
  • TLDR: Graph-based graph neural network for conversational KBQA.
• Evidence-based Factual Error Correction
  • James Thorne, Andreas Vlachos
  • TLDR: We present a novel method for generating factual error corrections from existing fact checking datasets which only contain labeled claims accompanied by evidence, but not the correction.
• Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments
  • Austin Blodgett, Nathan Schneider
  • TLDR: We present algorithms for aligning components of Abstract Meaning Representation (AMR) graphs to spans in English sentences.
• Meta-Learning to Compositionally Generalize
  • Henry Conklin, Bailin Wang, Kenny Smith, Ivan Titov
  • TLDR: Meta-learning for natural language synthesis by sub-sampling existing training data.
• Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation
  • Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song, Tong Zhang
  • TLDR: We present a novel domain-aware N-gram Adaptor for generic pretrained models that can improve the performance of a generic pretraining model on low-resource downstream tasks.
• ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
  • Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou
  • TLDR: We propose a novel contrastive learning framework ERICA to obtain a deep understanding of the entities and their relations in text.
• Position Bias Mitigation: A Knowledge-Aware Graph Model for Emotion Cause Extraction
  • Hanqi Yan, Lin Gui, Gabriele Pergola, Yulan He
  • TLDR: We propose a novel graph-based method to explicitly model the emotion triggering paths of emotion clauses and show that it is more robust against adversarial attacks than existing models.
• Every Bite Is an Experience: Key Point Analysis of Business Reviews
  • Roy Bar-Haim, Lilach Eden, Yoav Kantor, Roni Friedman, Noam Slonim
  • TLDR: We propose a novel summarization framework for review data that provides both textual and quantitative summary of the main points in the data.
• Structured Sentiment Analysis as Dependency Graph Parsing
  • Jeremy Barnes, Robin Kurtz, Stephan Oepen, Lilja Øvrelid, Erik Velldal
  • TLDR: We propose a new unified framework for structured sentiment analysis that uses graph-based dependency graph parsing to extract full opinion tuples from text.
• Consistency Regularization for Cross-Lingual Fine-Tuning
  • Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
  • TLDR: We propose to improve cross-lingual fine-tuning with consistency regularization.
• Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
  • Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei
  • TLDR: We propose a new cross-lingual pre-training task for cross-language language models that improves cross-linking transferability and improves performance on various tasks.
• Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation
  • Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu
  • TLDR: We propose reverse KD to rejuvenate more alignments for low-frequency target words in translation models.
• G-Transformer for Document-Level Machine Translation
  • Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, Weihua Luo
  • TLDR: We propose G-Transformer, a novel document-level translation model that can outperform Transformer by sticking around local minima during training.
• Prevent the Language Model from being Overconfident in Neural Machine Translation
  • Mengqi Miao, Fandong Meng, Yijin Liu, Xiao-Hua Zhou, Jie Zhou
  • TLDR: We propose a novel approach to improve translation adequacy and fluency of neural machine translation models by minimizing overconfidence of the Language Model.
• Towards Emotional Support Dialog Systems
  • Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, Minlie Huang
  • TLDR: We propose a new task and dataset for emotional support in dialog systems and provide a framework for training support strategies.
• Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System
  • Yanan Wu, Zhiyuan Zeng, Keqing He, Hong Xu, Yuanmeng Yan, Huixing Jiang, Weiran Xu
  • TLDR: We introduce a new task, Novel Slot Detection, in the task-oriented dialogue system.
• GTM: A Generative Triple-wise Model for Conversational Question Generation
  • Lei Shen, Fandong Meng, Jinchao Zhang, Yang Feng, Jie Zhou
  • TLDR: Generating appealing questions in open-domain conversations using generative triple-wise models with hierarchical variations.
• Diversifying Dialog Generation via Adaptive Label Smoothing
  • Yida Wang, Yinhe Zheng, Yong Jiang, Minlie Huang
  • TLDR: We propose a novel label smoothing approach for neural dialogue generation that adaptively estimates a target label distribution for different contexts and produces diverse responses.
• Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
  • Li-Ming Zhan, Haowen Liang, Bo Liu, Lu Fan, Xiao-Ming Wu, Albert Y.S. Lam
  • TLDR: We propose a simple yet effective method to train an out-of-scope intent classifier in a fully end-to-end manner by simulating the test scenario in training, which requires no assumption on data distribution and no additional post-processing or threshold setting.
• Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker
  • Runxin Xu, Tianyu Liu, Lei Li, Baobao Chang
  • TLDR: We propose a novel method for document-level event extraction that captures global interactions among different sentences and entity mentions.
• Nested Named Entity Recognition via Explicitly Excluding the Influence of the Best Path
  • Yiran Wang, Hiroyuki Shindo, Yuji Matsumoto, Taro Watanabe
  • TLDR: We propose a novel method for nested named entity recognition that uses hidden states to build a different potential function for recognition at each level.
• LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification
  • Xinyu Zuo, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Weihua Peng, Yuguang Chen
  • TLDR: We propose a novel approach to augment training data for event causality identification by iteratively generating new examples and classifying event causalities in a dual learning framework.
• Revisiting the Negative Data of Distantly Supervised Relation Extraction
  • Chenhao Xie, Jiaqing Liang, Jingping Liu, Chengsong Huang, Wenhao Huang, Yanghua Xiao
  • TLDR: We propose a pipeline approach for relation extraction that can be used to alleviate false negative problem.
• Knowing the No-match: Entity Alignment with Dangling Cases
  • Zequn Sun, Muhao Chen, Wei Hu
  • TLDR: We propose a multi-task learning framework for entity alignment and dangling entity detection for knowledge graphs.
• Superbizarre Is Not Superb: Derivational Morphology Improves BERT’s Interpretation of Complex Words
  • Valentin Hofmann, Janet Pierrehumbert, Hinrich Schütze
  • TLDR: We show that PLMs can be interpreted as serial dual-route models, i.e., the meanings of complex words are either stored or else need to be computed from the subwords, which implies that maximally meaningful input tokens should allow for the best generalization on new words.
• BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
  • Asahi Ushio, Luis Espinosa Anke, Steven Schockaert, Jose Camacho-Collados
  • TLDR: We analyze the ability of transformer-based language models to identify analogies and show that they are not as good as word embedding models.
• Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy
  • Marcos Garcia
  • TLDR: We present a multilingual study of word meaning representations in context.
• Measuring Fine-Grained Domain Relevance of Terms: A Hierarchical Core-Fringe Approach
  • Jie Huang, Kevin Chang, JinJun Xiong, Wen-mei Hwu
  • TLDR: We propose to measure fine-grained domain relevance– the degree that a term is relevant to a broad (e.g., computer science) or narrow (e.) domain.
• HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations
  • Weixin Liang, Kai-Hui Liang, Zhou Yu
  • TLDR: We propose HERALD, an efficient annotation framework that reframes the training data annotation process as a denoising problem.
• Value-Agnostic Conversational Semantic Parsing
  • Emmanouil Antonios Platanios, Adam Pauls, Subhro Roy, Yuchen Zhang, Alexander Kyte, Alan Guo, Sam Thomson, Jayant Krishnamurthy, Jason Wolfe, Jacob Andreas, Dan Klein
  • TLDR: We propose a model that abstracts over values to focus prediction on type- and function-level context in conversational semantic parsing.
• MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding
  • Jia-Chen Gu, Chongyang Tao, Zhenhua Ling, Can Xu, Xiubo Geng, Daxin Jiang
  • TLDR: We present MPC-BERT, a pre-trained model for multi-party conversation understanding that considers learning who says what to whom in a unified model with several elaborated self-supervised tasks.
• Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental
  • Morteza Rohanian, Julian Hough
  • TLDR: We present a novel method for incrementalising Transformer-based text classifiers without pre-segmentation and show that it improves the performance of existing Transformer models while maintaining their high non-incremental performance.
• NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation
  • Sungdong Kim, Minsuk Chang, Sang-Woo Lee
  • TLDR: We propose NeuralWOZ, a novel dialogue collection framework that uses model-based dialogue simulation.
• CDRNN: Discovering Complex Dynamics in Human Language Processing
  • Cory Shain
  • TLDR: We propose a deep neural extension of continuous-time deconvolutional regression that captures time-varying, non-linear, and delayed influences of predictors (e.g. word surprisal) on the response (e) of a language processing model.
• Structural Guidance for Transformer Language Models
  • Peng Qian, Tahira Naseem, Roger Levy, Ramón Fernandez Astudillo
  • TLDR: We propose a new way to train Transformer language models without needing to pre-training on large amounts of text data.
• Surprisal Estimators for Human Reading Times Need Character Models
  • Byung-Doh Oh, Christian Clark, William Schuler
  • TLDR: We present a novel character model that can be applied to a parser-based processing model to estimate surprisal estimates for sentence generation probabilities.
• CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals
  • Yuqi Ren, Deyi Xiong
  • TLDR: We propose a novel approach to align textual and cognitive features in neural models of natural language processing (NLP) by using a modality discriminator to capture their differences and commonalities.
• Self-Attention Networks Can Process Bounded Hierarchical Languages
  • Shunyu Yao, Binghui Peng, Christos Papadimitriou, Karthik Narasimhan
  • TLDR: We prove that self-attention networks can process Dyck-(k, D), a subset of Dyck-k, the language consisting of well-nested parentheses of k types.
• TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling
  • Parker Riley, Noah Constant, Mandy Guo, Girish Kumar, David Uthus, Zarana Parekh
  • TLDR: We present a novel approach to the problem of text style transfer that uses unlabeled text to condition the decoder to perform style transfer.
• H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences
  • Zhenhai Zhu, Radu Soricut
  • TLDR: We propose a new hierarchical method to compute attention in Transformer architecture that is efficient and effective in capturing the hierarchical structure in the sequences typical for natural language and vision tasks.
• Making Pre-trained Language Models Better Few-shot Learners
  • Tianyu Gao, Adam Fisch, Danqi Chen
  • TLDR: We present LM-BFF—better few-shot fine-tuning of language models—a suite of simple and complementary techniques for fine-tuning language models on a small number of annotated examples.
• A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks
  • Thai Le, Noseong Park, Dongwon Lee
  • TLDR: We propose DARCY, a honeypot-based defense framework against UniTrigger, a novel adversarial textual attack method that can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class.
• Towards Propagation Uncertainty: Edge-enhanced Bayesian Graph Convolutional Networks for Rumor Detection
  • Lingwei Wei, Dou Hu, Wei Zhou, Zhaojuan Yue, Songlin Hu
  • TLDR: We propose a novel Bayesian graph convolutional network for rumor detection that captures robust structural features and improves on the performance of existing methods.
• Label-Specific Dual Graph Neural Network for Multi-Label Text Classification
  • Qianwen Ma, Chunyuan Yuan, Wei Zhou, Songlin Hu
  • TLDR: We propose a novel label-specific dual graph neural network for multi-label text classification, which incorporates category information to learn label- specific components from documents, and employs dual Graph Convolution Network (GCN) to model complete and adaptive interactions among these components based on the statistical label co-occurrence and dynamic reconstruction graph in a joint way.
• TAN-NTM: Topic Attention Networks for Neural Topic Modeling
  • Madhur Panwar, Shashank Shailabh, Milan Aggarwal, Balaji Krishnamurthy
  • TLDR: We propose a novel attention mechanism which factors in topic-word distribution to enable the model to attend on relevant words that convey topic related cues.
• Cross-language Sentence Selection via Data Augmentation and Rationale Training
  • Yanda Chen, Chris Kedzie, Suraj Nair, Petra Galuscakova, Rui Zhang, Douglas Oard, Kathleen McKeown
  • TLDR: We propose a novel approach to cross-language sentence selection in a low-resource setting using data augmentation and negative sampling techniques on noisy parallel sentence data.
• A Neural Model for Joint Document and Snippet Ranking in Question Answering for Large Document Collections
  • Dimitris Pappas, Ion Androutsopoulos
  • TLDR: We present an architecture for joint document and snippet ranking, the two middle stages, which leverages the intuition that relevant documents have good snippets and good snippets come from relevant documents.
• W-RST: Towards a Weighted RST-style Discourse Framework
  • Patrick Huber, Wen Xiao, Giuseppe Carenini
  • TLDR: We propose a new approach to NLP that uses real-valued importance distributions to generate discourse trees from auxiliary tasks.
• ABCD: A Graph Framework to Convert Complex Sentences to a Covering Set of Simple Sentences
  • Yanjun Gao, Ting-Hao Huang, Rebecca J. Passonneau
  • TLDR: We propose a novel graph editing task for complex sentence decomposition and a novel algorithm for graph construction.
• Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering
  • Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, Deepak Ramachandran
  • TLDR: We present a novel framework for generating and verifying unanswerable questions and propose a new method for answering them.
• Adversarial Learning for Discourse Rhetorical Structure Parsing
  • Longyin Zhang, Fang Kong, Guodong Zhou
  • TLDR: We present a novel method to evaluate the pros and cons of the entire DRS tree for global optimization.
• Exploring Discourse Structures for Argument Impact Classification
  • Xin Liu, Jiefu Ou, Yangqiu Song, Xin Jiang
  • TLDR: We propose DisCOC, a novel approach to learn and analyze discourse relations among arguments along the context path of a debate conversation.
• Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation
  • Tong Zhang, Long Zhang, Wei Ye, Bo Li, Jinan Sun, Xiaoyu Zhu, Wen Zhao, Shikun Zhang
  • TLDR: We propose a novel neural architecture to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models.
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation
  • Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang, Luo Si
  • TLDR: We propose a new cross-lingual model for Transformer encoders that improves performance on language understanding and generation tasks.
• A unified approach to sentence segmentation of punctuated text in many languages
  • Rachel Wicks, Matt Post
  • TLDR: The sentence is a fundamental unit of text processing.
• Towards User-Driven Neural Machine Translation
  • Huan Lin, Liang Yao, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Degen Huang, Jinsong Su
  • TLDR: We propose a novel framework for neural machine translation that captures user traits from their historical inputs under zero-shot learning.
• End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages
  • Josef Jon, João Paulo Aires, Dusan Varis, Ondřej Bojar
  • TLDR: We propose a novel approach to improve translation of constrained terms in neural machine translation by reducing errors in agreement.
• Handling Extreme Class Imbalance in Technical Logbook Datasets
  • Farhad Akhbardeh, Cecilia Ovesdotter Alm, Marcos Zampieri, Travis Desell
  • TLDR: We propose a feedback strategy for dealing with extreme class imbalance in technical logbooks by resamples the training data based on its error in the prediction process.
• ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation
  • Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, Kripabandhu Ghosh, Shouvik Kumar Guha, Arnab Bhattacharya, Ashutosh Modi
  • TLDR: We propose a hierarchical occlusion based model for explainability of case predictions and a new algorithm for prediction and explanation of court judgments.
• Supporting Cognitive and Emotional Empathic Writing of Students
  • Thiemo Wambsganss, Christina Niklaus, Matthias Söllner, Siegfried Handschuh, Jan Marco Leimeister
  • TLDR: We present an annotation approach to capturing emotional and cognitive empathy in student-written peer reviews on business models in German.
• Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering
  • Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, Bing Xiang
  • TLDR: We propose a novel approach for generating SQL queries for open-domain question answering using textual evidences.
• Generation-Augmented Retrieval for Open-Domain Question Answering
  • Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen
  • TLDR: Generating diverse contexts for queries with sparse representations improves the semantics of queries and improves the performance of sparse retrieval methods.
• Check It Again:Progressive Visual Question Answering via Visual Entailment
  • Qingyi Si, Zheng Lin, Ming yu Zheng, Peng Fu, Weiping Wang
  • TLDR: We propose a select-and-rerank progressive framework based on Visual Entailment for Visual Question Answering.
• A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering
  • Zhihong Shao, Lifeng Shang, Qun Liu, Minlie Huang
  • TLDR: We propose to exploit the semantic correlations between question-answer pairs and predicted solutions to alleviate the spurious solution problem in weakly supervised question answering.
• Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy?
  • Abhilasha Ravichander, Alan W Black, Thomas Norton, Shomir Wilson, Norman Sadeh
  • TLDR: Language technologies can help users learn to read and understand privacy policies.
• Supporting Land Reuse of Former Open Pit Mining Sites using Text Classification and Active Learning
  • Christopher Schröder, Kim Bürgl, Yves Annanias, Andreas Niekler, Lydia Müller, Daniel Wiegreffe, Christian Bender, Christoph Mengs, Gerik Scheuermann, Gerhard Heyer
  • TLDR: We present and evaluate an automated workflow for supporting the post-mining management of former lignite open pit mines in the eastern part of Germany, where prior to any planned land reuse, aforementioned information has to be acquired to ensure the safety and validity of such an endeavor.
• Reliability Testing for Natural Language Processing Systems
  • Samson Tan, Shafiq Joty, Kathy Baxter, Araz Taeihagh, Gregory A. Bennett, Min-Yen Kan
  • TLDR: We propose a framework for developing adversarial attacks for reliability testing of neural network systems.
• Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
  • Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nick Allen, Randy Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency
  • TLDR: We show that language and multimodal representations of mobile typed text are predictive of daily mood, but that models trained to predict mood often also capture private user identities in their intermediate representations.
• Anonymisation Models for Text Data: State of the art, Challenges and Future Directions
  • Pierre Lison, Ildikó Pilán, David Sanchez, Montserrat Batet, Lilja Øvrelid
  • TLDR: We present a new approach to automated text anonymisation that incorporates explicit measures of disclosure risk into the text anonymization process.
• End-to-End AMR Coreference Resolution
  • Qiankun Fu, Linfeng Song, Wenyu Du, Yue Zhang
  • TLDR: We present the first end-to-end AMR coreference resolution model for parsing to abstract meaning representation and show that it can generate multi-sentence AMRs.
• How is BERT surprised? Layerwise detection of linguistic anomalies
  • Bai Li, Zining Zhu, Guillaume Thomas, Yang Xu, Frank Rudzicz
  • TLDR: We show that language models exhibit surprisal in earlier layers when the anomaly is morphosyntactic than when it is semantic, while commonsense anomalies do not exhibit surprusal at any intermediate layer.
• Psycholinguistic Tripartite Graph Network for Personality Detection
  • Tao Yang, Feifan Yang, Haolan Ouyang, Xiaojun Quan
  • TLDR: We propose a psycholinguistic knowledge-based tripartite graph network and a flow graph attention network for graph learning.
• Verb Metaphor Detection via Contextual Relation Learning
  • Wei Song, Shuhui Zhou, Ruiji Fu, Ting Liu, Lizhen Liu
  • TLDR: We propose a new verb metaphor detection model that explicitly models the relation between a verb and its grammatical, sentential and semantic contexts.
• Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
  • Yun Tang, Juan Pino, Xian Li, Changhan Wang, Dmitriy Genzel
  • TLDR: We propose three methods to improve the translation quality of speech translation models by improving the information sharing and parameter sharing between the tasks.
• Probing Toxic Content in Large Pre-Trained Language Models
  • Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, Dit-Yan Yeung
  • TLDR: We propose a method based on logistic regression classifiers to probe English, French, and Arabic PTLMs and quantify the potentially harmful content that they convey with respect to a set of templates.
• Societal Biases in Language Generation: Progress and Challenges
  • Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng
  • TLDR: We present a survey on societal biases in language generation and show promising directions for research and the importance of fairness and inclusivity considerations for language generation applications.
• Reservoir Transformers
  • Sheng Shen, Alexei Baevski, Ari Morcos, Kurt Keutzer, Michael Auli, Douwe Kiela
  • TLDR: We demonstrate that transformers obtain impressive performance even when some of the layers are randomly initialized and never updated.
• Subsequence Based Deep Active Learning for Named Entity Recognition
  • Puria Radmard, Yassir Fathullah, Aldo Lipani
  • TLDR: We propose a novel algorithm for querying subsequences within sentences and propagating their labels to other sentences.
• Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
  • Tyler Chang, Yifan Xu, Weijian Xu, Zhuowen Tu
  • TLDR: We propose composite attention, a new way of encoding relative position embeddings in Transformer self-attention layers that improves performance on multiple downstream tasks.
• BinaryBERT: Pushing the Limit of BERT Quantization
  • Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King
  • TLDR: We propose BinaryBERT, a new BERT quantization method that achieves state-of-the-art compression results on GLUE and SQuAD benchmarks.
• Are Pretrained Convolutions Better than Pretrained Transformers?
  • Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, Donald Metzler
  • TLDR: We show that CNN-based pre-trained language models are competitive to Transformers when pre-training.
• PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
  • Linlin Chao, Jianshan He, Taifeng Wang, Wei Chu
  • TLDR: Paired vectors for each relation representation enable a new state-of-the-art knowledge graph embedding model on link prediction tasks.
• Hierarchy-aware Label Semantics Matching Network for Hierarchical Text Classification
  • Haibin Chen, Qianli Ma, Zhenxi Lin, Jiangyue Yan
  • TLDR: We propose a hierarchical-aware label semantics matching network that captures the text-label semantics matching relationship among coarse-grained labels and fine-graining labels in a hierarchy-aware manner.
• HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability
  • Jiaao Chen, Dinghan Shen, Weizhu Chen, Diyi Yang
  • TLDR: We propose a simple yet effective data augmentation technique, HiddenCut, to better regularize the model and encourage it to learn more generalizable features.
• Neural Stylistic Response Generation with Disentangled Latent Variables
  • Qingfu Zhu, Wei-Nan Zhang, Ting Liu, William Yang Wang
  • TLDR: We propose to disentangle the content and style in latent space by diluting sentence-level information in style representations.
• Intent Classification and Slot Filling for Privacy Policies
  • Wasi Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, Kai-Wei Chang
  • TLDR: We propose PolicyIE, a new corpus of privacy policies of websites and mobile applications, which is a challenging real-world benchmark for understanding privacy policies.
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
  • Baolin Peng, Chunyuan Li, Zhu Zhang, Chenguang Zhu, Jinchao Li, Jianfeng Gao
  • TLDR: We present a robustness benchmark for task-oriented dialog systems and propose methods for evaluating robustness across a diverse set of domains.
• Semantic Representation for Dialogue Modeling
  • Xuefeng Bai, Yulong Chen, Linfeng Song, Yue Zhang
  • TLDR: We exploit abstract meaning representation to help dialogue modeling.
• A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations
  • Chongyang Tao, Changyu Chen, Jiazhan Feng, Ji-Rong Wen, Rui Yan
  • TLDR: We propose a novel method for knowledge-grounded response selection and comprehension based on a unified pre-trained language model and a large number of ungrounded multi-turn dialogues.
• Dependency-driven Relation Extraction with Attentive Graph Convolutional Networks
  • Yuanhe Tian, Guimin Chen, Yan Song, Xiang Wan
  • TLDR: We propose a dependency-driven approach for relation extraction with attentive graph convolutional networks, which outperforms previous studies and achieves state-of-the-art performance on both datasets.
• Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP
  • Anthony Chen, Pallavi Gudipati, Shayne Longpre, Xiao Ling, Sameer Singh
  • TLDR: We present a new evaluation benchmark for evaluating the entity disambiguation capabilities of open-domain NLP retrievers and show that popular retrieval systems exhibit popularity bias.
• Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?
  • Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, Jordan Boyd-Graber
  • TLDR: We propose a Bayesian model for evaluating NLP models and show that it can help to identify overfitting and over-fitting.
• Claim Matching Beyond English to Scale Global Fact-Checking
  • Ashkan Kazemi, Kiran Garimella, Devin Gaffney, Scott Hale
  • TLDR: We propose a novel method for annotating WhatsApp tipline and public group messages alongside fact-checked claims that are first annotated for containing “claim-like statements” and then matched with potentially similar items and annotated to match with potentially matching items and use this data to train a high-quality multilingual embedding model.
• SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation
  • Shuo Ren, Long Zhou, Shujie Liu, Furu Wei, Ming Zhou, Shuai Ma
  • TLDR: We propose a new pre-training method for neural machine translation by defining a semantic interface (SemFace) between the pre-trained encoder and the pre -trained decoder.
• Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models
  • Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum
  • TLDR: We present a re-ranking algorithm based on the samples drawn from autoregressive neural machine translation (NMT) and show that it improves the performance of Transformer-based NMT.
• Syntax-augmented Multilingual BERT for Cross-lingual Transfer
  • Wasi Ahmad, Haoran Li, Kai-Wei Chang, Yashar Mehdad
  • TLDR: We provide language syntax-augmented multilingual text encoders and show that syntax-algebra improves cross-lingual transfer on popular NLP tasks.
• How to Adapt Your Pretrained Multilingual Model to 1600 Languages
  • Abteen Ebrahimi, Katharina Kann
  • TLDR: Pretrained multilingual models for unseen languages with the New Testament.
• Weakly Supervised Named Entity Tagging with Learnable Logical Rules
  • Jiacheng Li, Haibo Ding, Jingbo Shang, Julian McAuley, Zhe Feng
  • TLDR: We propose a novel method TALLOR that bootstraps high-quality logical rules to train a neural tagger in a fully automated manner.
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
  • Xiang Lisa Li, Percy Liang
  • TLDR: We propose prefix-tuning, a lightweight alternative to fine-tuned language models for natural language generation tasks, which keeps language model parameters frozen and instead optimizes a sequence of continuous task-specific vectors, which we call the prefix.
• One2Set: Generating Diverse Keyphrases as a Set
  • Jiacheng Ye, Tao Gui, Yichao Luo, Yige Xu, Qi Zhang
  • TLDR: We propose a new training paradigm One2Set without predefining an order to concatenate the keyphrases.
• Continuous Language Generative Flow
  • Zineng Tang, Shiyue Zhang, Hyounghun Kim, Mohit Bansal
  • TLDR: We propose a flow-based language generation model for natural language generation and data augmentation for Question Answering.
• TWAG: A Topic-Guided Wikipedia Abstract Generator
  • Fangwei Zhu, Shangqing Tu, Jiaxin Shi, Juanzi Li, Lei Hou, Tong Cui
  • TLDR: We propose a two-stage model TWAG that guides the Wikipedia abstract generation with topical information.
• ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data
  • Woojeong Jin, Rahul Khanna, Suji Kim, Dong-Ho Lee, Fred Morstatter, Aram Galstyan, Xiang Ren
  • TLDR: We present a new QA task for event forecasting using restricted domain, multiple-choice, question-answering, and BERTbased models.
• Recursive Tree-Structured Self-Attention for Answer Sentence Selection
  • Khalil Mrini, Emilia Farcas, Ndapa Nakashole
  • TLDR: We propose a novel recursive, tree-structured self-attention model for Answer Sentence Selection that can learn syntactic structure and show strong performance on question-based question-answer tasks.
• How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction
  • Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua
  • TLDR: We propose a paradigm to quantitatively evaluate the effect of attention and knowledge graph on bag-level relation extraction and show that attention mechanism may exacerbate the issue of insufficient training data.
• Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction
  • Kaiwen Wei, Xian Sun, Zequn Zhang, Jingyuan Zhang, Guo Zhi, Li Jin
  • TLDR: We present a frame-aware event argument extraction framework that leverages related arguments of the expected one as clues to guide the reasoning process.
• Element Intervention for Open Relation Extraction
  • Fangchao Liu, Lingyong Yan, Hongyu Lin, Xianpei Han, Le Sun
  • TLDR: We propose a causal model for OpenRE, which can be used to solve the instability and spurious correlations in OpenRE.
• AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding
  • Jun Yan, Nasser Zalmout, Yan Liang, Christan Grant, Xiang Ren, Xin Luna Dong
  • TLDR: We present AdaTag, a multi-attribute attribute model that uses adaptive decoding to handle multi-attribution extraction.
• CoRI: Collective Relation Integration with Data Augmentation for Open Information Extraction
  • Zhengbao Jiang, Jialong Han, Bunyamin Sisman, Xin Luna Dong
  • TLDR: We propose a new method for relation integration that aims to align free-text relations in subject-relation-object extractions to relations in a target KG.
• Benchmarking Scalable Methods for Streaming Cross Document Entity Coreference
  • Robert L Logan IV, Andrew McCallum, Sameer Singh, Dan Bikel
  • TLDR: We present a comprehensive evaluation of the best ways to encode and classify mentions of named entities in streaming cross document entity coreference systems.
• Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs
  • Zixuan Li, Xiaolong Jin, Saiping Guan, Wei Li, Jiafeng Guo, Yuanzhuo Wang, Xueqi Cheng
  • TLDR: We propose CluSTeR, a novel method for temporal knowledge graph prediction based on clues from historical facts.
• Employing Argumentation Knowledge Graphs for Neural Argument Generation
  • Khalid Al Khatib, Lukas Trautner, Henning Wachsmuth, Yufang Hou, Benno Stein
  • TLDR: We use knowledge graphs to control the generation of arguments with better quality than those generated without knowledge.
• Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction
  • Lu Xu, Yew Ken Chia, Lidong Bing
  • TLDR: We propose a novel span-level approach for the span-based sentiment triplet extraction which improves the performance of the current models on multi-word triplet extractions.
• On Compositional Generalization of Neural Machine Translation
  • Yafu Li, Yongjing Yin, Yulong Chen, Yue Zhang
  • TLDR: We study NMT models from the perspective of compositional generalization by building a benchmark dataset, CoGnition, consisting of 216k clean and consistent sentence pairs.
• Mask-Align: Self-Supervised Neural Word Alignment
  • Chi Chen, Maosong Sun, Yang Liu
  • TLDR: We propose Mask-Align, a self-supervised word alignment model that takes advantage of the full context in the target sequence and predicts the most likely target token to be aligned.
• GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation
  • Huayang Li, Lemao Liu, Guoping Huang, Shuming Shi
  • TLDR: We propose a novel method for general word-level autocompletion in computer-aided translation and present a benchmark for it.
• De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention
  • Wenkai Zhang, Hongyu Lin, Xianpei Han, Le Sun
  • TLDR: We propose a causal invariance regularizer for dictionary-biased Deep Distant-Supervision Models and propose a new method to improve the robustness of the learned models.
• A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition
  • Fei Li, ZhiChao Lin, Meishan Zhang, Donghong Ji
  • TLDR: We propose a novel span-based model that can recognize both overlapped and discontinuous entities jointly.
• MLBiNet: A Cross-Sentence Collective Event Detection Network
  • Dongfang Lou, Zhilin Liao, Shumin Deng, Ningyu Zhang, Huajun Chen
  • TLDR: We propose a new task for detecting multiple events in cross-sentence settings.
• Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution
  • Hieu Minh Tran, Duy Phung, Thien Huu Nguyen
  • TLDR: We present a novel deep learning model for event coreference resolution that captures relevant objects for ECR and regularizes the consistencies of golden and predicted clusters for event mentions in documents.
• StereoRel: Relational Triple Extraction from a Stereoscopic Perspective
  • Xuetao Tian, Liping Jing, Lu He, Feng Liu
  • TLDR: We provide a revealing insight into relational triple extraction from a stereoscopic perspective, which rationalizes the occurrence of these issues and exposes the shortcomings of existing methods.
• Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks
  • Pengfei Cao, Xinyu Zuo, Yubo Chen, Kang Liu, Jun Zhao, Yuguang Chen, Weihua Peng
  • TLDR: We propose a novel Latent Structure Induction Network (LSIN) to incorporate the external structural knowledge into this task.
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
  • Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, Maosong Sun
  • TLDR: We present invisible backdoors that can be injected into neural NLP models and produce attacker-specified predictions when the backdoor is activated.
• Parameter-Efficient Transfer Learning with Diff Pruning
  • Demi Guo, Alexander Rush, Yoon Kim
  • TLDR: We propose a new method for parameter-efficient transfer learning that scales well with new tasks.
• R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling
  • Xiang Hu, Haitao Mi, Zujie Wen, Yafang Wang, Yi Su, Jing Zheng, Gerard de Melo
  • TLDR: We propose a recursive Transformer model based on differentiable CKY style binary trees to emulate this composition process, and we extend the bidirectional language model pre-training objective to this architecture, attempting to predict each word given its left and right abstraction nodes.
• Risk Minimization for Zero-shot Sequence Labeling
  • Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
  • TLDR: We propose a novel unified framework for zero-shot sequence labeling with minimum risk training and design a new decomposable risk function that models the relations between the predicted labels from the source models and the true labels.
• WARP: Word-level Adversarial ReProgramming
  • Karen Hambardzumyan, Hrant Khachatrian, Jonathan May
  • TLDR: We present an alternative approach based on adversarial reprogramming to transfer learning from pretrained language models that outperforms all existing methods with up to 25M trainable parameters on the GLUE benchmark.
• Lexicon Learning for Few Shot Sequence Modeling
  • Ekin Akyurek, Jacob Andreas
  • TLDR: We propose a lexical translation mechanism for sequence modeling that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules.
• Personalized Transformer for Explainable Recommendation
  • Lei Li, Yongfeng Zhang, Li Chen
  • TLDR: We present a simple and effective learning objective that utilizes the IDs to predict the words in the target explanation, so as to endow the IDs with linguistic meanings and to achieve personalized Transformer.
• Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques
  • Kundan Krishna, Sopan Khosla, Jeffrey Bigham, Zachary C. Lipton
  • TLDR: We propose Cluster2Sent, a novel algorithm for generating semi-structured clinical summaries based on transcripts of conversations between physicians and patients.
• Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction
  • Piji Li, Shuming Shi
  • TLDR: We investigate the problem of Chinese Grammatical Error Correction (CGEC) and present a new framework named Tail-to-Tail (
• Early Detection of Sexual Predators in Chats
  • Matthias Vogt, Ulf Leser, Alan Akbik
  • TLDR: We propose a new algorithm for early sexual predator detection in chats, which can be used to prevent grooming attempts.
• Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation
  • Xingyi Yang, Muchao Ye, Quanzeng You, Fenglong Ma
  • TLDR: We propose a novel hierarchical retrieval mechanism for medical report generation that uses both report and sentence-level templates for clinically accurate report generation.
• Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification
  • Xuepeng Wang, Li Zhao, Bing Liu, Tao Chen, Feng Zhang, Di Wang
  • TLDR: We propose a novel concept-based label embedding method that can explicitly represent the concept and model the sharing mechanism among classes for the hierarchical text classification.
• VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words
  • Xiaopeng Lu, Tiancheng Zhao, Kyusong Lee
  • TLDR: VisualSparta is a transformer-based text-to-image retrieval model that can achieve real-time searching for large-scale datasets, with significant accuracy improvement compared to previous state-of-the-art methods.
• Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
  • Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett
  • TLDR: MetaAdaptRank is a domain adaptive learning method for neural information retrieval that generalizes Neu-IR models from label-rich source domains to few-shot target domains.
• Semi-Supervised Text Classification with Balanced Deep Representation Distributions
  • Changchun Li, Ximing Li, Jihong Ouyang
  • TLDR: We propose a novel SSTC method, namely Semi-Supervised Text Classification with Balanced Deep representation Distributions (S2TC-BDD).
• Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval
  • Hongyin Tang, Xingwu Sun, Beihong Jin, Jingang Wang, Fuzheng Zhang, Wei Wu
  • TLDR: We present a novel method to mimic queries to each document by iterative clustering and improve the retrieval performance of sparse vector space models.
• ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer
  • Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, Weiran Xu
  • TLDR: We present ConSERT, a Contrastive Framework for Self-Supervised SEntence Representation Transfer, that adopts contrastive learning to fine-tune BERT in an unsupervised and effective way.
• Exploring Dynamic Selection of Branch Expansion Orders for Code Generation
  • Hui Jiang, Chulun Zhou, Fandong Meng, Biao Zhang, Jie Zhou, Degen Huang, Qingqiang Wu, Jinsong Su
  • TLDR: We propose a new algorithm for dynamically selecting optimal expansion orders of branches for multi-branch nodes in a Seq2Tree model.
• COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion
  • Debjit Paul, Anette Frank
  • TLDR: We present Coins, a recursive inference framework that iteratively reads context sentences, dynamically generates contextualized inference rules, encodes them, and uses them to guide task-specific output generation.
• Reasoning over Entity-Action-Location Graph for Procedural Text Understanding
  • Hao Huang, Xiubo Geng, Jian Pei, Guodong Long, Daxin Jiang
  • TLDR: We propose a novel approach (REAL) to procedural text understanding, where we build a general framework to systematically model the entity-entity, entity-action, and entity-location relations using a graph neural network.
• From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding
  • Shan Wu, Bo Chen, Chunlei Xin, Xianpei Han, Le Sun, Weipeng Zhang, Jiansong Chen, Fan Yang, Xunliang Cai
  • TLDR: We propose a novel unsupervised semantic parsing method which can simultaneously resolve the structure gap and the structure gaps by jointly leveraging paraphrasing and grammar-constrained decoding.
• Pre-training Universal Language Representation
  • Yian Li, Hai Zhao
  • TLDR: We propose a novel language representation learning objective for language models that can learn universal language representations.
• Structural Pre-training for Dialogue Comprehension
  • Zhuosheng Zhang, Hai Zhao
  • TLDR: We present SPIDER, Structural Pre-traIned DialoguE Reader, a novel novel method for pre-training dialogue models which captures dialogue exclusive features.
• AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
  • Yichun Yin, Cheng Chen, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
  • TLDR: We propose a new architecture hyper-parameter search algorithm for language models with tiny size and a new efficient development method for language model architectures.
• Data Augmentation with Adversarial Training for Cross-Lingual NLI
  • Xin Dong, Yaxin Zhu, Zuohui Fu, Dongkuan Xu, Gerard de Melo
  • TLDR: We propose a novel data augmentation strategy for better cross-lingual natural language inference by enriching the data to reflect more diversity in a semantically faithful way.
• Bootstrapped Unsupervised Sentence Representation Learning
  • Yan Zhang, Ruidan He, Zuozhu Liu, Lidong Bing, Haizhou Li
  • TLDR: We propose a novel framework for unsupervised sentence representation learning which maximizes the similarity between two augmented views of each sentence.
• Learning Event Graph Knowledge for Abductive Reasoning
  • Li Du, Xiao Ding, Ting Liu, Bing Qin
  • TLDR: We present a narrative text based abductive reasoning task for reading comprehension and question answering.
• A Cognitive Regularizer for Language Modeling
  • Jason Wei, Clara Meister, Ryan Cotterell
  • TLDR: We propose a new inductive bias for language modeling that improves perplexity in language models.
• Lower Perplexity is Not Always Human-Like
  • Tatsuki Kuribayashi, Yohei Oseki, Takumi Ito, Ryo Yoshida, Masayuki Asahara, Kentaro Inui
  • TLDR: We re-examine an established generalization of computational psycholinguistics that can be generalized across languages.
• Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives
  • Ming Wang, Yinglin Wang
  • TLDR: We propose a novel approach to learn sense embeddings in context for word sense disambiguation by learning sentences from the same document.
• A Knowledge-Guided Framework for Frame Identification
  • Xuefeng Su, Ru Li, Xiaoli Li, Jeff Z. Pan, Hu Zhang, Qinghua Chai, Xiaoqi Han
  • TLDR: We propose a knowledge-guided frame identification framework that learns better frame representation and significantly outperforms the state-of-the-art methods on two benchmark datasets.
• Obtaining Better Static Word Embeddings Using Contextual Embedding Models
  • Prakhar Gupta, Martin Jaggi
  • TLDR: We propose a novel method for distilling contextual embeddings from static embeddents and show that it improves computational efficiency and accuracy.
• Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation
  • Yingjun Du, Nithin Holla, Xiantong Zhen, Cees Snoek, Ekaterina Shutova
  • TLDR: We propose a model of semantic memory for few-shot word sense disambiguation using meta-learning and show its ability to learn new word senses from very few examples.
• LexFit: Lexical Fine-Tuning of Pretrained Language Models
  • Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš
  • TLDR: We show that it is possible to expose and enrich lexical knowledge from transformable language models and use it to serve as effective and universal word encoders even when fed input words “in isolation” (i.e., without any context).
• Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
  • Wei-Ning Hsu, David Harwath, Tyler Miller, Christopher Song, James Glass
  • TLDR: We present a novel model for synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision.
• CTFN: Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network
  • Jiajia Tang, Kang Li, Xuanyu Jin, Andrzej Cichocki, Qibin Zhao, Wanzeng Kong
  • TLDR: Coupled-translation fusion network for multimodal sentiment analysis.
• Positional Artefacts Propagate Through Masked Language Model Embeddings
  • Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
  • TLDR: We show that outliers in word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers.
• Language Model Evaluation Beyond Perplexity
  • Clara Meister, Ryan Cotterell
  • TLDR: We propose an alternate approach to quantifying how well language models learn natural language: we ask how well they match the statistical tendencies of natural language.
• Learning to Explain: Generating Stable Explanations Fast
  • Xuelin Situ, Ingrid Zukerman, Cecile Paris, Sameen Maruf, Gholamreza Haffari
  • TLDR: We propose a new algorithm for learning the behaviour of an explanation algorithm simultaneously from all training examples.
• StereoSet: Measuring stereotypical bias in pretrained language models
  • Moin Nadeem, Anna Bethke, Siva Reddy
  • TLDR: We present a large-scale natural English dataset to measure stereotypical biases in language models and show that these models exhibit strong stereotypical biases.
• Alignment Rationale for Natural Language Inference
  • Zhongtao Jiang, Yuanzhe Zhang, Zhao Yang, Jun Zhao, Kang Liu
  • TLDR: A post-hoc approach to generate alignment rationale explanations for co-attention based models in NLI.
• Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators
  • Peiyu Liu, Ze-Feng Gao, Wayne Xin Zhao, Zhi-Yuan Xie, Zhong-Yi Lu, Ji-Rong Wen
  • TLDR: We propose a novel pre-trained language models (PLM) compression approach based on the matrix product operator (short as MPO) from quantum many-body physics.
• On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation
  • Wei Zhang, Ziming Huang, Yada Zhu, Guangnan Ye, Xiaodong Cui, Fan Zhang
  • TLDR: We propose a new method for improving the interpretability of explanations by allowing arbitrary text sequences as the explanation unit.
• Syntax-Enhanced Pre-trained Model
  • Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan
  • TLDR: We present a model that utilizes the syntax of text in both pre-training and fine-tuning stages to enhance pre-trained models.
• Matching Distributions between Model and Data: Cross-domain Knowledge Distillation for Unsupervised Domain Adaptation
  • Bo Zhang, Xiaoming Zhang, Yun Liu, Lei Cheng, Zhoujun Li
  • TLDR: Unsupervised domain adaptation using only trained source model.
• Counterfactual Inference for Text Classification Debiasing
  • Chen Qian, Fuli Feng, Lijie Wen, Chunping Ma, Pengjun Xie
  • TLDR: We propose a novel model-agnostic text classification debiasing framework which can effectively avoid employing data manipulations or designing balancing mechanisms.
• HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation
  • Tao Qi, Fangzhao Wu, Chuhan Wu, Peiru Yang, Yang Yu, Xing Xie, Yongfeng Huang
  • TLDR: We propose a hierarchical user interest modeling method for personalized news recommendation.
• PP-Rec: News Recommendation with Personalized User Interest and Time-aware News Popularity
  • Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang
  • TLDR: We propose a new algorithm for personalized news recommendation based on personalized matching and popularity information.
• Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims
  • Qiang Sheng, Juan Cao, Xueyao Zhang, Xirong Li, Lei Zhong
  • TLDR: We propose a novel reranker for candidate fact-checking articles using key sentences selected with event (lexical and semantic) and pattern information.
• Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble
  • Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
  • TLDR: We propose Dirichlet Neighborhood Ensemble, a randomized method for training a robust model to defense synonym substitution-based attacks.
• Shortformer: Better Language Modeling using Shorter Inputs
  • Ofir Press, Noah A. Smith, Mike Lewis
  • TLDR: We show that short input lengths are not harmful to language modeling, and show how to improve the efficiency of recurrent models by training on short subsequences.
• BanditMTL: Bandit-based Multi-task Learning for Text Classification
  • Yuren Mao, Zekai Wang, Weiwei Liu, Xuemin Lin, Wenbin Hu
  • TLDR: We propose a novel multi-task learning method based on adversarial multi-armed bandit.
• Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding
  • Hidetaka Kamigaito, Katsuhiko Hayashi
  • TLDR: We provide a unified interpretation of the softmax cross-entropy and negative sampling loss functions and show that theoretical findings are valid in practical settings.
• De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation
  • Wenqing Chen, Jidong Tian, Yitian Li, Hao He, Yaohui Jin
  • TLDR: Logical table-to-text generation aims to automatically generate fluent and logically faithful text from tables.
• Rethinking Stealthiness of Backdoor Attack against NLP Models
  • Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun
  • TLDR: We propose two new stealthiness-based metrics to make the backdoor attacking evaluation more credible.
• Crowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition
  • Xin Zhang, Guangwei Xu, Yueheng Sun, Meishan Zhang, Pengjun Xie
  • TLDR: We propose a crowdsourcing method for learning named entity recognition that is highly similar to domain adaptation, and show that it can be used to improve the performance of supervised NER models.
• Exploring Distantly-Labeled Rationales in Neural Network Models
  • Quzhe Huang, Shengqi Zhu, Yansong Feng, Dongyan Zhao
  • TLDR: We propose two novel auxiliary loss functions to make better use of distantly-labeled rationales, which encourage models to maintain their focus on important words beyond labeled rationales.
• Learning to Perturb Word Embeddings for Out-of-distribution QA
  • Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang
  • TLDR: We propose a simple yet effective word embedding perturbation method for QA models based on data augmentation techniques.
• Maria: A Visual Experience Powered Conversational Agent
  • Zujie Liang, Huang Hu, Can Xu, Chongyang Tao, Xiubo Geng, Yining Chen, Fan Liang, Daxin Jiang
  • TLDR: We present a neural conversation agent that learns to perceive the visual world experiences of conversational agents and use them to ground the conversation.
• A Human-machine Collaborative Framework for Evaluating Malevolence in Dialogues
  • Yangjun Zhang, Pengjie Ren, Maarten de Rijke
  • TLDR: We present a new collaborative framework for dialogue evaluation that can guarantee reliability of the evaluation outcomes with reduced human effort.
• Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders
  • Bin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu, Kan Li
  • TLDR: We propose a new method for generating dialogue responses that are more coherent and diverse, while maintaining their diversity and informativeness.
• Learning to Ask Conversational Questions by Optimizing Levenshtein Distance
  • Zhongkun Liu, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Maarten de Rijke, Ming Zhou
  • TLDR: We propose a novel iterative sequence editing framework for conversational question simplification that optimizes Levenshtein distance through explicit editing actions.
• DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue
  • Hung Le, Chinnadhurai Sankar, Seungwhan Moon, Ahmad Beirami, Alborz Geramifard, Satwik Kottur
  • TLDR: We present DVD, a Diagnostic Dataset for Video-grounded Dialogue, a dataset for evaluating video-grounding dialogue systems.
• MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
  • Jingwen Hu, Yuchen Liu, Jinming Zhao, Qin Jin
  • TLDR: We propose a new multimodal fused graph convolutional network based on multimodality fused graph neural networks for emotion recognition in multimodally-constructed conversation.
• DynaEval: Unifying Turn and Dialogue Level Evaluation
  • Chen Zhang, Yiming Chen, Luis Fernando D’Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, Haizhou Li
  • TLDR: We propose DynaEval, a unified automatic dialogue evaluation framework which is not only capable of performing turn-level evaluation, but also holistically considers the quality of the entire dialogue.
• CoSQA: 20,000+ Web Queries for Code Search and Question Answering
  • Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, Nan Duan
  • TLDR: We introduce CoSQA dataset for text-code matching and introduce a contrastive learning method dubbed CoCLR to enhance text-coding matching.
• Rewriter-Evaluator Architecture for Neural Machine Translation
  • Yangming Li, Kaisheng Yao
  • TLDR: We propose a novel architecture of Rewriter-Evaluator for neural machine translation with multiple rewriting passes.
• Modeling Bilingual Conversational Characteristics for Neural Chat Translation
  • Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou
  • TLDR: We propose a novel approach to improve the translation quality of bilingual conversational text by modeling the above properties.
• Importance-based Neuron Allocation for Multilingual Neural Machine Translation
  • Wanying Xie, Yang Feng, Shuhao Gu, Dong Yu
  • TLDR: We propose to divide the model neurons into general and language-specific parts based on their importance across languages.
• Transfer Learning for Sequence Generation: from Single-source to Multi-source
  • Xuancheng Huang, Jingfang Xu, Maosong Sun, Yang Liu
  • TLDR: We propose a novel MSG model with a fine encoder to learn better representations in MSG tasks.
• A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters
  • Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schütze
  • TLDR: We show that few-shot crosslingual transfer is a highly sensitive crosslinguational transfer task, and provide a theoretical analysis of its performance and experimental results.
• Coreference Reasoning in Machine Reading Comprehension
  • Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych
  • TLDR: We present a methodology for creating MRC datasets that better reflect the challenges of coreference reasoning and use it to create a sample evaluation set.
• Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing
  • Liwen Zhang, Ge Wang, Wenjuan Han, Kewei Tu
  • TLDR: We propose a simple yet effective method to adapt unsupervised syntactic dependency parsing methodology for unsupervisioned discourse dependency parsing.
• A Conditional Splitting Framework for Efficient Constituency Parsing
  • Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, Xiaoli Li
  • TLDR: We present a new parsing framework that casts constituency parsing problems (syntactic and discourse parsing) into a series of conditional splitting decisions and show that it outperforms state-of-the-art methods that are more computationally expensive than ours.
• A Unified Generative Framework for Various NER Subtasks
  • Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, Xipeng Qiu
  • TLDR: We propose to solve all three kinds of NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
• An In-depth Study on Internal Structure of Chinese Words
  • Chen Gong, Saihao Huang, Houquan Zhou, Zhenghua Li, Min Zhang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan
  • TLDR: We propose to model the deep internal structures of Chinese words as dependency trees with 11 labels for distinguishing syntactic relationships.
• MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER
  • Linlin Liu, Bosheng Ding, Lidong Bing, Shafiq Joty, Luo Si, Chunyan Miao
  • TLDR: We propose a simple but effective zero-shot transfer method for cross-lingual NER for low-resource languages.
• Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter
  • Wei Liu, Xiyan Fu, Yue Zhang, Wenming Xiao
  • TLDR: We propose Lexicon Enhanced BERT for Chinese sequence labeling, which integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer.
• Math Word Problem Solving with Explicit Numerical Values
  • Qinzhuo Wu, Qi Zhang, Zhongyu Wei, Xuanjing Huang
  • TLDR: We propose a novel approach called NumS2T, which incorporates numerical values into a sequence-to-tree network for math word problem solving.
• Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks
  • Jinghui Qin, Xiaodan Liang, Yining Hong, Jianheng Tang, Liang Lin
  • TLDR: We propose Neural-Symbolic Solver (NS-Solver) to explicitly and seamlessly incorporate different levels of symbolic constraints by auxiliary tasks.
• SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining
  • Taolin Zhang, Zerui Cai, Chengyu Wang, Minghui Qiu, Bite Yang, Xiaofeng He
  • TLDR: We propose a medical PLM trained on large-scale medical corpora, incorporating deep structured semantic knowledge from neighbours of linked-entity.
• What is Your Article Based On? Inferring Fine-grained Provenance
  • Yi Zhang, Zachary Ives, Dan Roth
  • TLDR: When evaluating an article and the claims it makes, a critical reader must be able to assess where the information presented comes from, and whether the various claims are mutually consistent and support the conclusion.
• Cross-modal Memory Networks for Radiology Report Generation
  • Zhihong Chen, Yaling Shen, Yan Song, Xiang Wan
  • TLDR: We propose a cross-modal memory network for radiology report generation, which is able to better align information from radiology images and texts so as to help generating more accurate reports in terms of clinical indicators.
• Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection
  • Kamil Kanclerz, Alicja Figas, Marcin Gruza, Tomasz Kajdanowicz, Jan Kocon, Daria Puchalska, Przemyslaw Kazienko
  • TLDR: We propose novel personalized approaches that respect individual beliefs expressed by either user conformity-based measures or various embeddings of their previous text annotations.
• Multi-perspective Coherent Reasoning for Helpfulness Prediction of Multimodal Reviews
  • Junhao Liu, Zhen Hai, Min Yang, Lidong Bing
  • TLDR: We propose a novel Multi-perspective Coherent Reasoning method for review helpfulness prediction and a novel multi-persive coherent reasoning module for the task.
• Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding
  • Xin Sun, Tao Ge, Furu Wei, Houfeng Wang
  • TLDR: We propose Shallow Aggressive Aggressive Decoding for Grammatical Error Correction.
• Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism
  • Tong Zhou, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Kun Niu, Weifeng Chong, Shengping Liu
  • TLDR: We propose an interactive shared representation network with self-distillation mechanism for the automatic ICD coding task.
• PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Check
  • Li Huang, Junjie Li, Weiwei Jiang, Zhiyu Zhang, Minchuan Chen, Shaojun Wang, Jing Xiao
  • TLDR: We propose a novel end-to-end trainable model for Chinese spelling check with multi-modal information.
• Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting
  • Yi Cheng, Siyao Li, Bang Liu, Ruihui Zhao, Sujian Li, Chenghua Lin, Yefeng Zheng
  • TLDR: We propose a novel framework for generating questions with required difficulty levels and propose a method for improving the controllability of Question Generation systems.
• Improving Encoder by Auxiliary Supervision Tasks for Table-to-Text Generation
  • Liang Li, Can Ma, Yinliang Yue, Dayong Hu
  • TLDR: We propose to use reasoning modules to capture the relations between entities in the input tables and use them to generate natural text.
• POS-Constrained Parallel Decoding for Non-autoregressive Generation
  • Kexin Yang, Wenqiang Lei, Dayiheng Liu, Weizhen Qi, Jiancheng Lv
  • TLDR: We propose a novel method for sequence-level knowledge distillation in NAG that improves the performance of NAG models on text generation tasks.
• Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation
  • Xin Liu, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Min Zhang, Haiying Zhang, Jinsong Su
  • TLDR: We extend the vanilla pretrain-finetune pipeline with an extra embedding transfer step to enable us to transfer the vocabulary to downstream tasks.
• TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models
  • Jie He, Bo Peng, Yi Liao, Qun Liu, Deyi Xiong
  • TLDR: We present a comprehensive dataset for error annotation of text generation from pretrained language models and propose a series of diagnostic tasks for text generation.
• Long-Span Summarization via Local Attention and Content Selection
  • Potsawee Manakul, Mark Gales
  • TLDR: We exploit large pre-trained transformer-based models and address long-span dependencies in abstractive summarization using two methods: local self-attention; and explicit content selection.
• RepSum: Unsupervised Dialogue Summarization based on Replacement Strategy
  • Xiyan Fu, Yating Zhang, Tianyi Wang, Xiaozhong Liu, Changlong Sun, Zhenglu Yang
  • TLDR: We propose a novel unsupervised dialogue summarization algorithm based on the hypothetical foundation that a superior summary approximates a replacement of the original dialogue, and they are roughly equivalent for auxiliary (self-supervised) tasks, e.g., dialogue generation.
• BASS: Boosting Abstractive Summarization with Unified Semantic Graph
  • Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, Haifeng Wang
  • TLDR: We present BASS, a novel framework for Boosting Abstractive Summarization based on a unified Semantic graph, which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases.
• Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation
  • Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Xiangliang Zhang, Dongyan Zhao, Rui Yan
  • TLDR: We propose a relation-aware multi-document encoder that generates abstractive related work from the given multiple scientific papers in the same research area.
• Focus Attention: Promoting Faithfulness and Diversity in Summarization
  • Rahul Aralikatte, Shashi Narayan, Joshua Maynez, Sascha Rothe, Ryan McDonald
  • TLDR: We propose Focus Attention Mechanism, a simple yet effective method to encourage decoders to proactively generate tokens that are similar or topical to the input document.
• Generating Query Focused Summaries from Query-Free Resources
  • Yumo Xu, Mirella Lapata
  • TLDR: We propose a unified representation for query focused summarization and a Masked ROUGE Regression framework for evidence estimation and ranking which relies on a unified query representation for summaries and queries.
• Robustifying Multi-hop QA through Pseudo-Evidentiality Training
  • Kyungjae Lee, Seung-won Hwang, Sang-eun Han, Dohyeon Lee
  • TLDR: We propose a new approach to learn evidentiality, deciding whether the answer prediction is supported by correct evidences, without such annotations.
• xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering
  • Nan Yang, Furu Wei, Binxing Jiao, Daxing Jiang, Linjun Yang
  • TLDR: We propose a new contrastive learning method for question-passage matching, which efficiently uses separate encoders for questions and passages.
• Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering
  • Gangwoo Kim, Hyunjae Kim, Jungsoo Park, Jaewoo Kang
  • TLDR: We propose a novel framework for conversational question answering that improves the abilities of QA models in comprehending conversational context.
• PhotoChat: A Human-Human Dialogue Dataset With Photo Sharing Behavior For Joint Image-Text Modeling
  • Xiaoxue Zang, Lijuan Liu, Maria Wang, Yang Song, Hao Zhang, Jindong Chen
  • TLDR: We present a new human-human dialogue dataset that casts light on the photo sharing behavior in online messaging.
• Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
  • Zhiyong Wu, Lingpeng Kong, Wei Bi, Xiang Li, Ben Kao
  • TLDR: We propose a new multimodal machine translation system that learns to ignore multimodality, and show that the improvements achieved by the multimodals are in fact results of the regularization effect.
• Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering
  • Ahjeong Seo, Gi-Cheon Kang, Joonhan Park, Byoung-Tak Zhang
  • TLDR: We propose Motion-Appearance Synergistic Networks for video question answering, which embed two cross-modal features grounded on motion and appearance information and selectively utilize them depending on the question’s intentions.
• BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition
  • Yinghao Li, Pranav Shetty, Lucas Liu, Chao Zhang, Le Song
  • TLDR: We propose a conditional hidden Markov model that can effectively infer true labels from multi-source noisy labels in an unsupervised way.
• CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction
  • Tao Chen, Haizhou Shi, Siliang Tang, Zhigang Chen, Fei Wu, Yueting Zhuang
  • TLDR: We propose a novel contrastive instance learning framework for distant supervision training data.
• SENT: Sentence-level Distant Relation Extraction via Negative Training
  • Ruotian Ma, Tao Gui, Linyang Li, Qi Zhang, Xuanjing Huang, Yaqian Zhou
  • TLDR: We propose a sentence-level framework for distant relation extraction based on negative training and a sentence label framework for sentence- level evaluation.
• An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization
  • Baohang Zhou, Xiangrui Cai, Ying Zhang, Xiaojie Yuan
  • TLDR: We propose an end-to-end progressive multi-task learning model for jointly modeling medical NER and NEN in an effective way.
• PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction
  • Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, Yefeng Zheng
  • TLDR: We propose a novel approach to extract entities and relations from unstructured text using relational triple extraction and a novel method for aligning subjects and objects.
• Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition
  • Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, Juanzi Li
  • TLDR: We propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different unde- fined classes from the other class to improve few-shot NER.
• Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference
  • Tuan Lai, Heng Ji, ChengXiang Zhai, Quan Hung Tran
  • TLDR: We present a novel framework that utilizes external knowledge for joint entity and relation extraction from biomedical text.
• Fine-grained Information Extraction from Biomedical Literature based on Knowledge-enriched Abstract Meaning Representation
  • Zixuan Zhang, Nikolaus Parulian, Heng Ji, Ahmed Elsayed, Skatje Myers, Martha Palmer
  • TLDR: We propose a novel biomedical information extraction model for biomedical literature that uses abstract meaning representation to extract scientific entities and events from English research papers.
• Unleash GPT-2 Power for Event Detection
  • Amir Pouran Ben Veyseh, Viet Lai, Franck Dernoncourt, Thien Huu Nguyen
  • TLDR: We propose to exploit the powerful pre-trained language model GPT-2 to generate training samples for event detection and train a teacher-student architecture for event recognition.
• CLEVE: Contrastive Pre-training for Event Extraction
  • Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou
  • TLDR: We propose CLEVE, a contrastive pre-training framework for event extraction that improves both the conventional supervised and the unsupervised event extraction models.
• Document-level Event Extraction via Parallel Prediction Networks
  • Hang Yang, Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, Taifeng Wang
  • TLDR: We propose an end-to-end model for document-level event extraction, which can extract structured events from a document in a parallel manner.
• StructuralLM: Structural Pre-training for Form Understanding
  • Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si
  • TLDR: We propose a new pre-training approach to combine cell-level layout information from scanned documents with semantic information from text-only representation for form understanding and document image classification.
• Dual Graph Convolutional Networks for Aspect-based Sentiment Analysis
  • Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma, Xiaojie Wang, Eduard Hovy
  • TLDR: We propose a dual graph convolutional network over dependency trees that captures semantic correlations between aspects and opinion words.
• Multi-Label Few-Shot Learning for Aspect Category Detection
  • Mengting Hu, Shiwan Zhao, Honglei Guo, Chao Xue, Hang Gao, Tiegang Gao, Renhong Cheng, Zhong Su
  • TLDR: We propose a multi-label few-shot learning method based on the prototypical network for aspect category detection in the few-shots learning scenario.
• Argument Pair Extraction via Attention-guided Multi-Layer Multi-Cross Encoding
  • Liying Cheng, Tianyu Wu, Lidong Bing, Luo Si
  • TLDR: Attention-guided multi-layer multi-cross encoding scheme for argument pair extraction.
• A Neural Transition-based Model for Argumentation Mining
  • Jianzhu Bao, Chuang Fan, Jipeng Wu, Yixue Dang, Jiachen Du, Ruifeng Xu
  • TLDR: We propose a neural transition-based model for argumentation mining, which incrementally builds an argumentation graph by generating a sequence of actions, avoiding inefficient enumeration operations.
• Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text
  • Philippe Laban, Tobias Schnabel, Paul Bennett, Marti A. Hearst
  • TLDR: We propose a novel algorithm to optimize the reward for text simplification which learns to balance a reward across three properties: fluency, salience and simplicity.
• Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence
  • Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, Minlie Huang
  • TLDR: We propose a long text generation model that can represent the prefix sentences at sentence level and discourse level in the decoding process.
• OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics
  • Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, Minlie Huang
  • TLDR: We propose OpenMEVA, a benchmark for evaluating open-ended story generation metrics.
• DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation
  • Xinyu Hua, Ashwin Sreevatsa, Lu Wang
  • TLDR: We propose a novel generation framework for long-form opinion text generation based on a novel design of mixed language models.
• Controllable Open-ended Question Generation with A New Question Type Ontology
  • Shuyang Cao, Lu Wang
  • TLDR: We propose a novel question type-aware question generation framework and a novel model for generating open-ended questions that are typically answered by multiple sentences.
• BERTGen: Multi-task Generation through BERT
  • Faidon Mitzalis, Ozan Caglayan, Pranava Madhyastha, Lucia Specia
  • TLDR: We present BERTGen, a novel, generative, decoder-only model which extends BERT by fusing multimodal and multilingual pre-trained models VL-BERT and M-BERTs.
• Selective Knowledge Distillation for Neural Machine Translation
  • Fusheng Wang, Jianhao Yan, Fandong Meng, Jie Zhou
  • TLDR: We propose a novel protocol for knowledge distillation that can effectively analyze the different impacts and connections among different samples by comparing various samples’ partitions.
• Measuring and Increasing Context Usage in Context-Aware Machine Translation
  • Patrick Fernandes, Kayo Yin, Graham Neubig, André F. T. Martins
  • TLDR: We introduce a new metric, conditional cross-mutual information, to quantify usage of context by document-level machine translation systems, and show that including more context has a diminishing affect on results.
• Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring
  • Aitor Ormazabal, Mikel Artetxe, Aitor Soroa, Gorka Labaka, Eneko Agirre
  • TLDR: We propose an unsupervised method for cross-lingual word embeddings that uses a weak seed dictionary as the only form of supervision, and use skip-grams to learn new embeddable embeddents for the source language that are aligned with the target language.
• CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web
  • Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, Armand Joulin, Angela Fan
  • TLDR: We show that margin-based bitext mining in a multilingual sentence space can be successfully scaled to operate on monolingual corpora of billions of sentences.
• Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
  • Gyuwan Kim, Kyunghyun Cho
  • TLDR: We propose Length-Adaptive Transformer that can be used for various inference scenarios after one-shot training.
• GhostBERT: Generate More Features with Cheap Operations for BERT
  • Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
  • TLDR: We propose GhostBERT, a new method for generating more features from the remaining features of BERT models without sacrificing accuracy.
• Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
  • Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen
  • TLDR: We show that at certain compression ratios, the generalization performance of the winning tickets in extremely over-parametrized models can match but also exceed that of the full model.
• A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations
  • Pierre Colombo, Pablo Piantanida, Chloé Clavel
  • TLDR: We propose a novel variational upper bound to the mutual information between an attribute and the latent code of an encoder that allows for better disentangled representations of textual data.
• Determinantal Beam Search
  • Clara Meister, Martina Forster, Ryan Cotterell
  • TLDR: We propose a new algorithm for diverse subset selection in beam search that uses the string subsequence kernel to encourage n-gram coverage in text generated from a sequence model.
• Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
  • Shuoran Jiang, Qingcai Chen, Xin Liu, Baotian Hu, Lisai Zhang
  • TLDR: We propose a spectral graph convolutional network with high-order dynamic Chebyshev approximation for multi-hop graph reasoning.
• Accelerating Text Communication via Abbreviated Sentence Input
  • Jiban Adhikary, Jamie Berger, Keith Vertanen
  • TLDR: We show that noisy abbreviated input can be possible even if a third of characters are omitted.
• Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates
  • Yuqing Xie, Yi-An Lai, Yuanjun Xiong, Yi Zhang, Stefano Soatto
  • TLDR: We quantify, reduce and analyze regression errors in the NLP model updates.
• Detecting Propaganda Techniques in Memes
  • Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, Giovanni Da San Martino
  • TLDR: Propaganda techniques used in memes can appear in text, in image, or in both.
• On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study
  • Divyansh Kaushik, Douwe Kiela, Zachary C. Lipton, Wen-tau Yih
  • TLDR: We show that adversarial data collection produces more robust models than standard data, but not as well as other adversarial datasets.
• Learning Dense Representations of Phrases at Scale
  • Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, Danqi Chen
  • TLDR: We present a novel phrase retrieval model that learns dense phrase representations from reading comprehension tasks and use them to improve query-side fine-tuning and transfer learning.
• End-to-End Training of Neural Retrievers for Open-Domain Question Answering
  • Devendra Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
  • TLDR: We propose an approach of unsupervised pre-training of neural retrievers for open-domain question answering and a method of end-to-end training of the reader and retriever components in OpenQA models.
• Question Answering Over Temporal Knowledge Graphs
  • Apoorv Saxena, Soumen Chakrabarti, Partha Talukdar
  • TLDR: We present CRONQUESTIONS, the largest known Temporal KGQA dataset, and CRONKGQAA, a transformer-based solution that achieves performance superior to all baselines, with an increase of 120% in accuracy over the next best performing method.
• Language Model Augmented Relevance Score
  • Ruibo Liu, Jason Wei, Soroush Vosoughi
  • TLDR: We propose Language Model Augmented Relevance Score (MARS), a new context-aware metric for NLG evaluation.
• DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
  • Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi
  • TLDR: We propose DExperts: Decoding-time Experts, a decoding-time method for controlled text generation that combines a pretrained language model with “expert” LMs and/or “anti-expert”, and use it to control attributes of generated text.
• Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
  • Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel Weld
  • TLDR: We present Polyjuice, a general-purpose counterfactual generator that allows for control over perturbation types and locations, trained by finetuning GPT-2 on multiple datasets of paired sentences.
• Metaphor Generation with Conceptual Mappings
  • Kevin Stowe, Tuhin Chakrabarty, Nanyun Peng, Smaranda Muresan, Iryna Gurevych
  • TLDR: We propose to control the generation process by encoding conceptual mappings between cognitive domains to generate meaningful metaphoric expressions.
• Learning Latent Structures for Cross Action Phrase Relations in Wet Lab Protocols
  • Chaitanya Kulkarni, Jany Chan, Eric Fosler-Lussier, Raghu Machiraju
  • TLDR: We propose a new model that incrementally learns latent structures and is better suited to resolving inter-sentence relations and implicit arguments in wet laboratory protocols.
• Multimodal Multi-Speaker Merger & Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines
  • Ramit Sawhney, Mihir Goyal, Prakhar Goel, Puneet Mathur, Rajiv Ratn Shah
  • TLDR: We present a new baseline architecture for M&A calls that uses multimodal multi-speaker input to forecast the financial risk associated with the M&Ac calls.
• Mid-Air Hand Gestures for Post-Editing of Machine Translation
  • Rashad Albo Jamara, Nico Herbig, Antonio Krüger, Josef van Genabith
  • TLDR: We present a prototype for text editing in machine translation and post-editing using mid-air hand gestures and a keyboard.
• Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning
  • Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, Song-Chun Zhu
  • TLDR: Interpretable Geometry problem solving with formal language and symbolic reasoning.
• Joint Verification and Reranking for Open Fact Checking Over Tables
  • Michael Sejr Schlichtkrull, Vladimir Karpukhin, Barlas Oguz, Mike Lewis, Wen-tau Yih, Sebastian Riedel
  • TLDR: We propose a new method for verifying structured data in the open-domain setting, which achieves performance comparable to the closed-domain state-of-the-art on the TabFact dataset.
• Evaluation of Thematic Coherence in Microblogs
  • Iman Munire Bilal, Bo Wang, Maria Liakata, Rob Procter, Adam Tsakalidis
  • TLDR: We propose a new metric for evaluating thematic coherence in microblog clusters and provide annotation guidelines and human annotations of thematic content.
• Neural semi-Markov CRF for Monolingual Word Alignment
  • Wuwei Lan, Chao Jiang, Wei Xu
  • TLDR: We present a novel neural semi-Markov CRF alignment model for monolingual word alignment that unifies word and phrase alignments through variable-length spans.
• Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies
  • Mukund Srinath, Shomir Wilson, C Lee Giles
  • TLDR: We present a large-scale corpus of English language website privacy policies collected from the web and propose a new unsupervised topic modelling approach to interpret and simplify privacy policies.
• The statistical advantage of automatic NLG metrics at the system level
  • Johnny Wei, Robin Jia
  • TLDR: We show that metrics are not as good as humans in estimating system-level quality.
• Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion
  • Yixin Cao, Xiang Ji, Xin Lv, Juanzi Li, Yonggang Wen, Hanwang Zhang
  • TLDR: We present InferWiki, a Knowledge Graph Completion dataset that improves upon existing benchmarks in inferential ability, assumptions, and patterns.
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining
  • Alexander Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang, Haoran Li, Yashar Mehdad, Dragomir Radev
  • TLDR: We propose a novel method for summarizing online conversations by using issues–viewpoints–assertions and graph mining to model the issues, viewpoints, and assertions present in a conversation.
• Improving Factual Consistency of Abstractive Summarization via Question Answering
  • Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
  • TLDR: We present a novel algorithm for improving factual consistency in summarization models.
• EmailSum: Abstractive Email Thread Summarization
  • Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, Mohit Bansal
  • TLDR: We present a comprehensive empirical study on email thread summarization and show that current abstractive summarization models are weakly correlated with human judgments on this email thread summary generation task.
• Cross-Lingual Abstractive Summarization with Limited Parallel Resources
  • Yu Bai, Yang Gao, Heyan Huang
  • TLDR: We propose a novel multi-task framework for cross-lingual summarization that learns interactions between the discrete phrases of summaries in different languages, breaking the connections in order to transfer the knowledge of the high-resource languages to low-resource language.
• Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution
  • Jiacheng Xu, Greg Durrett
  • TLDR: We propose a two-step method to interpret summarization model decisions.
• Learning Prototypical Functions for Physical Artifacts
  • Tianyu Jiang, Ellen Riloff
  • TLDR: We introduce a new NLP task of learning the prototypical uses for human-made physical objects.
• Verb Knowledge Injection for Multilingual Event Processing
  • Olga Majewska, Ivan Vulić, Goran Glavaš, Edoardo Maria Ponti, Anna Korhonen
  • TLDR: We propose a novel approach to augment language knowledge obtained during language pretraining with verb knowledge, which improves the performance of pretrained Transformer-based language models in event extraction tasks.
• Dynamic Contextualized Word Embeddings
  • Valentin Hofmann, Janet Pierrehumbert, Hinrich Schütze
  • TLDR: We present dynamic contextualized word embeddings that represent words as a function of both linguistic and extralinguistic context.
• Lexical Semantic Change Discovery
  • Sinan Kurtyigit, Maike Park, Dominik Schlechtweg, Jonas Kuhn, Sabine Schulte im Walde
  • TLDR: We propose a shift of focus from change detection to change discovery, i.e., discovering novel word senses over time from the full corpus vocabulary.
• The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity
  • David Gros, Yu Li, Zhou Yu
  • TLDR: We explore how to identify non-human systems in dialogs and explore how we can make them confirm their non-humans identity.
• Using Meta-Knowledge Mined from Identifiers to Improve Intent Recognition in Conversational Systems
  • Claudio Pinhanez, Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel, Heloisa Candello, Julio Nogima, Mauro Pichiliani, Melina Guerra, Maira de Bayser, Gabriel Malfatti, Henrique Ferreira
  • TLDR: Meta-knowledge in chatbots improves intent recognition.
• Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer
  • Fabian Galetzka, Jewgeni Rose, David Schlangen, Jens Lehmann
  • TLDR: We propose a new way to encode background context structured in the form of knowledge graphs, which improves the coherence and knowledge retrieval capabilities of Transformer-based dialogue systems.
• DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations
  • Dou Hu, Lingwei Wei, Xiaoyong Huai
  • TLDR: We propose novel Contextual Reasoning Networks for Emotion Recognition in Conversations.
• Cross-replication Reliability - An Empirical Approach to Interpreting Inter-rater Reliability
  • Ka Wong, Praveen Paritosh, Lora Aroyo
  • TLDR: We present a new approach to interpreting IRR that is more empirical and contextualized.
• TIMEDIAL: Temporal Commonsense Reasoning in Dialog
  • Lianhui Qin, Aditya Gupta, Shyam Upadhyay, Luheng He, Yejin Choi, Manaal Faruqui
  • TLDR: We present the first study to investigate pre-trained language models for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TimeDial.
• RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English)
  • Sean Trott, Benjamin Bergen
  • TLDR: We present a dataset of graded, human relatedness judgments for 112 ambiguous words in context and a measure of cosine distance for contextualized embeddings.
• ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
  • Muhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi
  • TLDR: We present a new benchmark for multi-dialectal Arabic language understanding evaluation and propose a new deep bidirectional transformer-based model for pre-trained language models.
• Improving Paraphrase Detection with the Adversarial Paraphrasing Task
  • Animesh Nighojkar, John Licato
  • TLDR: We present a new adversarial method for generating sentence-level paraphrase datasets that are semantically equivalent but syntactically and lexically and syntactally disparate.
• ADEPT: An Adjective-Dependent Plausibility Task
  • Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
  • TLDR: We present a large-scale semantic plausibility task that uses adjective-noun pairs to assess the plausibility of given events.
• ReadOnce Transformers: Reusable Representations of Text for Transformers
  • Shih-Ting Lin, Ashish Sabharwal, Tushar Khot
  • TLDR: We present ReadOnce Transformers, an approach to convert a transformer-based model into one that can build an information-capturing, task-independent, and compressed representation of text.
• Conditional Generation of Temporally-ordered Event Sequences
  • Shih-Ting Lin, Nathanael Chambers, Greg Durrett
  • TLDR: We propose a novel model for narrative schema knowledge that captures both temporality and co-occurrence of events, and use it to generate new events that fit into an existing temporally-ordered sequence.
• Hate Speech Detection Based on Sentiment Knowledge Sharing
  • Xianbing Zhou, Yang Yong, Xiaochao Fan, Ge Ren, Yunfeng Song, Yufeng Diao, Liang Yang, Hongfei Lin
  • TLDR: We propose a novel framework for hate speech detection based on sentiment knowledge sharing and propose a new feature extraction unit for the task.
• Transition-based Bubble Parsing: Improvements on Coordination Structure Prediction
  • Tianze Shi, Lillian Lee
  • TLDR: We propose a transition-based bubble parser to perform coordination structure identification and dependency-based syntactic analysis simultaneously.
• SpanNER: Named Entity Re-/Recognition as Span Prediction
  • Jinlan Fu, Xuanjing Huang, Pengfei Liu
  • TLDR: We show that span prediction can serve as a system combiner and a span prediction model for named entity recognition.
• StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling
  • Yikang Shen, Yi Tay, Che Zheng, Dara Bahri, Donald Metzler, Aaron Courville
  • TLDR: We propose a new parsing framework that can induce dependency and constituency structure at the same time.
• Language Embeddings for Typology and Cross-lingual Transfer Learning
  • Dian Yu, Taiqi He, Kenji Sagae
  • TLDR: We explore whether language representations that capture relationships among languages can be learned and subsequently leveraged in cross-lingual tasks without the use of parallel data.
• Can Sequence-to-Sequence Models Crack Substitution Ciphers?
  • Nada Aldarrab, Jonathan May
  • TLDR: We propose an end-to-end multilingual model for solving simple substitution ciphers.
• Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation
  • Eleftheria Briakou, Marine Carpuat
  • TLDR: We analyze the impact of different types of semantic divergences on Transformer models.
• Discriminative Reranking for Neural Machine Translation
  • Ann Lee, Michael Auli, Marc’Aurelio Ranzato
  • TLDR: We present a new approach to discriminative reranking for neural machine translation models by training a large transformer architecture.
• Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering
  • Siddharth Karamcheti, Ranjay Krishna, Li Fei-Fei, Christopher Manning
  • TLDR: We identify a general phenomenon responsible for degrading pool-based active learning and show that it is a general feature of active learning that is responsible for the observed decline in sample efficiency.
• All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
  • Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, Noah A. Smith
  • TLDR: We investigate the role untrained human evaluations play in natural language generation and provide recommendations for improving human evaluations of text generated from state-of-the-art models.
• Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers
  • Benjamin Marie, Atsushi Fujita, Raphael Rubino
  • TLDR: Meta-evaluation of machine translation.
• Neural Machine Translation with Monolingual Translation Memory
  • Deng Cai, Yan Wang, Huayang Li, Wai Lam, Lemao Liu
  • TLDR: We propose a new framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner.
• Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
  • Armen Aghajanyan, Sonal Gupta, Luke Zettlemoyer
  • TLDR: We show that pre-trained language models have a very low intrinsic dimension and that this dimensionality can be used to fine-tuned to achieve state-of-the-art results on language understanding tasks.
• UnNatural Language Inference
  • Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams
  • TLDR: We show that state-of-the-art NLU models are word order invariant to permuted examples, and that this invariance is not limited to Transformer-based models.
• Including Signed Languages in Natural Language Processing
  • Kayo Yin, Amit Moryossef, Julie Hochgesang, Yoav Goldberg, Malihe Alikhani
  • TLDR: We call on the NLP community to explore and leverage the linguistic organization of signed languages.
• Vocabulary Learning via Optimal Transport for Neural Machine Translation
  • Jingjing Xu, Hao Zhou, Chun Gan, Zaixiang Zheng, Lei Li
  • TLDR: We propose a novel vocabularization algorithm that can find the optimal vocabulary without trial training.