ACL 2022

• German
• Italian
• Chinese
• Japanese

TLDRs

• AdapLeR: Speeding up Inference by Adaptive Length Reduction
  • Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar
  • TLDR: We propose a novel approach for reducing the computational cost of BERT with minimal loss in downstream performance.
• Quantified Reproducibility Assessment of NLP Results
  • Anya Belz, Maja Popovic, Simon Mille
  • TLDR: We propose a method for quantifying reproducibility assessment based on concepts and definitions from metrology.
• Rare Tokens Degenerate All Tokens: Improving Neural Text Generation via Adaptive Gradient Gating for Rare Token Embeddings
  • Sangwon Yu, Jongyoon Song, Heeseung Kim, Seongmin Lee, Woo-Jong Ryu, Sungroh Yoon
  • TLDR: We analyze the training dynamics of token embeddings focusing on rare token embedding and propose a novel method to address the degeneration problem.
• AlephBERT: Language Model Pre-training and Evaluation from Sub-Word to Sentence Level
  • Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, Reut Tsarfaty
  • TLDR: We present AlephBERT, a large pre-trained language model for Modern Hebrew, trained on larger vocabulary and a larger dataset than any Hebrew PLM before.
• Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning
  • Moxin Li, Fuli Feng, Hanwang Zhang, Xiangnan He, Fengbin Zhu, Tat-Seng Chua
  • TLDR: We propose a novel approach to counterfactual thinking in neural discrete reasoning by using the imagination of unseen counterfactuality.
• Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification
  • George-Eduard Zaharia, Răzvan-Alexandru Smădu, Dumitru Cercel, Mihai Dascalu
  • TLDR: We propose a novel training technique for complex word identification based on domain adaptation to improve the target character and context representations.
• JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection
  • Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, Ruifeng Xu
  • TLDR: We propose a novel target-aware prototypical graph contrastive learning and stance contrastive training framework for zero-shot stance detection.
• [CASPI] Causal-aware Safe Policy Improvement for Task-oriented Dialogue
  • Govardana Sachithanandam Ramachandran, Kazuma Hashimoto, Caiming Xiong
  • TLDR: We propose a batch-based RL framework for Task-oriented Dialogues that captures intention behind human response and also offers guarantee on dialogue policy’s performance against a baseline.
• UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog System
  • Zhiyuan Ma, Jianjun Li, Guohui Li, Yongjing Cheng
  • TLDR: Unified Transformer Semantic Representation for multimodal task-oriented dialog system.
• Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking
  • Yue Feng, Aldo Lipani, Fanghua Ye, Qiang Zhang, Emine Yilmaz
  • TLDR: We propose a novel approach to modelling the relations among domains and slots in dialogue state tracking.
• Attention Temperature Matters in Abstractive Summarization Distillation
  • Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, Furu Wei
  • TLDR: We propose a method for pseudo-labeling based text summarization that improves vanilla pseudo-labeling based methods and improves abstractive text summarizations.
• Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation
  • Guanhua Chen, Shuming Ma, Yun Chen, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei
  • TLDR: We present a multilingual NMT model that can transfer cross-linguals unseen during translation and demonstrate its performance on many-to-English translation and cross-language abstractive summarization.
• TopWORDS-Seg: Simultaneous Text Segmentation and Word Discovery for Open-Domain Chinese Texts via Bayesian Inference
  • Changzai Pan, Maosong Sun, Ke Deng
  • TLDR: We propose a novel method for efficient and transparent translation of open-domain Chinese texts based on Bayesian inference.
• An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition
  • Zhuoran Li, Chunming Hu, Xiaohui Guo, Junfan Chen, Wenyi Qin, Richong Zhang
  • TLDR: We introduce the similarity metric model as an auxiliary task to improve the cross-lingual NER performance on the target domain.
• Discriminative Marginalized Probabilistic Neural Method for Multi-Document Summarization of Medical Literature
  • Gianluca Moro, Luca Ragazzi, Lorenzo Valgimigli, Davide Freddi
  • TLDR: We propose a novel discriminative marginalized probabilistic method for multi-document summarization of biomedical literature reviews.
• Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
  • Shaoyi Huang, Dongkuan Xu, Ian Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu, Caiwen Ding
  • TLDR: We show that reducing the risk of overfitting can help the effectiveness of pruning Transformer-based language models under the pretrain-and-finetune paradigm.
• CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation
  • Nishant Kambhatla, Logan Born, Anoop Sarkar
  • TLDR: We propose a novel data-augmentation technique for neural machine translation based on ROT-
• Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages
  • Vaidehi Patil, Partha Talukdar, Sunita Sarawagi
  • TLDR: We propose Overlap BPE, a simple yet effective modification to the BPE vocabulary generation algorithm which enhances overlap across related languages.
• Long-range Sequence Modeling with Predictable Sparse Attention
  • Yimeng Zhuang, Jing Zhang, Mei Tu
  • TLDR: We provide a new approach for sparse attention matrix for fast long-range sequence modeling.
• Improving Personalized Explanation Generation through Visualization
  • Shijie Geng, Zuohui Fu, Yingqiang Ge, Lei Li, Gerard de Melo, Yongfeng Zhang
  • TLDR: We propose a visually-enhanced approach named METER with the help of visualization generation and text–image matching discrimination to improve the explainable recommendation model’s ability to generate plausible explanations.
• New Intent Discovery with Pre-training and Contrastive Learning
  • Yuwei Zhang, Haode Zhang, Li-Ming Zhan, Xiao-Ming Wu, Albert Lam
  • TLDR: We propose a novel method for new intent discovery that uses unlabeled utterances for representation learning and clustering.
• Modeling U.S. State-Level Policies by Extracting Winners and Losers from Legislative Texts
  • Maryam Davoodi, Eric Waltenburg, Dan Goldwasser
  • TLDR: We use data to predict winners/losers of bills and analyze legislators’ votes.
• Structural Characterization for Dialogue Disentanglement
  • Xinbei Ma, Zhuosheng Zhang, Hai Zhao
  • TLDR: We propose a novel model for dialogue disentangling that incorporates structural information of dialogues in two aspects: speaker property that indicates whom a message is from, and reference dependency that shows whom a sentence may refer to.
• Multi-Party Empathetic Dialogue Generation: A New Task for Dialog Systems
  • Ling.Yu Zhu, Zhengkun Zhang, Jun Wang, Hongbin Wang, Haiying Wu, Zhenglu Yang
  • TLDR: We propose a novel task for multi-party empathetic dialogue generation that combines static sensibility and dynamic emotion for the multi-part dialogue generation.
• MISC: A Mixed Strategy-Aware Model integrating COMET for Emotional Support Conversation
  • Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, Rui Yan
  • TLDR: We propose a novel model for emotional support conversation that uses empathy to reduce user distress.
• GLM: General Language Model Pretraining with Autoregressive Blank Infilling
  • Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang
  • TLDR: We propose a general language model based on autoregressive blank infilling to address the challenge of pretraining for natural language understanding and unconditional generation.
• QuoteR: A Benchmark of Quote Recommendation for Writing
  • Fanchao Qi, Yanhui Yang, Jing Yi, Zhili Cheng, Zhiyuan Liu, Maosong Sun
  • TLDR: We present a large and fully open quote recommendation dataset for use in the research on the task of quote recommendation.
• Towards Comprehensive Patent Approval Predictions:Beyond Traditional Document Classification
  • Xiaochen Gao, Zhaoyi Hou, Yifei Ning, Kewen Zhao, Beilei He, Jingbo Shang, Vish Krishnan
  • TLDR: Predicting the approval chance of a patent application is a challenging problem involving multiple facets.
• Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering
  • Yu-Jung Heo, Eun-Sol Kim, Woo Suk Choi, Byoung-Tak Zhang
  • TLDR: We propose a new method for knowledge-based visual question answering by encoding high-order semantics of multi-hop knowledge facts and learning high-level associations between them.
• Cross-Utterance Conditioned VAE for Non-Autoregressive Text-to-Speech
  • Yang Li, Cheng Yu, Guangzhi Sun, Hua Jiang, Fanglei Sun, Weiqin Zu, Ying Wen, Yang Yang, Jun Wang
  • TLDR: We propose a novel method for modelling prosody variation in end-to-end text-to speech systems by conditioning on utterance-specific utterance information.
• Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
  • Fatemehsadat Mireshghallah, Kartik Goyal, Taylor Berg-Kirkpatrick
  • TLDR: We propose a global score-based alternative for controllable text generation that combines arbitrary pre-trained black-box models for achieving the desired attributes in the generated text without involving any fine-tuning or structural assumptions about the black-boxes.
• So Different Yet So Alike! Constrained Unsupervised Text Style Transfer
  • Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, Roger Zimmermann, Soujanya Poria
  • TLDR: We introduce complementary cooperative losses to GANs for unsupervised text style transfer and show that the complementary cooperative loss improves text quality.
• e-CARE: a New Dataset for Exploring Explainable Causal Reasoning
  • Li Du, Xiao Ding, Kai Xiong, Ting Liu, Bing Qin
  • TLDR: We present a new dataset for explainable CAusal REasoning questions and explanations.
• Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension
  • Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
  • TLDR: We present FairytaleQA, a dataset focusing on narrative comprehension of kindergarten to eighth-grade students.
• KaFSP: Knowledge-Aware Fuzzy Semantic Parsing for Conversational Question Answering over a Large-Scale Knowledge Base
  • Junzhuo Li, Deyi Xiong
  • TLDR: We propose a knowledge-aware fuzzy semantic parsing framework for uncertain reasoning in conversational question answering over a large-scale knowledge base.
• Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment
  • Zijie Huang, Zheng Li, Haoming Jiang, Tianyu Cao, Hanqing Lu, Bing Yin, Karthik Subbian, Yizhou Sun, Wei Wang
  • TLDR: We propose a novel self-supervised adaptive graph alignment method for multilingual knowledge graph completion that leverages limited seed alignment as a bridge, to embrace the collective knowledge from multiple languages.
• Modeling Hierarchical Syntax Structure with Triplet Position for Source Code Summarization
  • Juncai Guo, Jin Liu, Yao Wan, Li Li, Pingyi Zhou
  • TLDR: We propose CODESCRIBE to model the hierarchical syntax structure of code by introducing a novel triplet position for code summarization.
• FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
  • Yanan Zheng, Jing Zhou, Yujie Qian, Ming Ding, Chonghua Liao, Li Jian, Ruslan Salakhutdinov, Jie Tang, Sebastian Ruder, Zhilin Yang
  • TLDR: We present a new evaluation framework for few-shot natural language understanding that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability.
• Learn to Adapt for Generalized Zero-Shot Text Classification
  • Yiwen Zhang, Caixia Yuan, Xiaojie Wang, Ziwei Bai, Yongbin Liu
  • TLDR: We propose a novel Learn to Adapt (LTA) network for generalized zero-shot text classification that learns to adapt its parameters to incoming unseen classes.
• TableFormer: Robust Transformer Modeling for Table-Text Encoding
  • Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, Shachi Paul
  • TLDR: We propose a robust and structurally aware table-text encoding architecture TableFormer, where tabular structural biases are incorporated completely through learnable attention biases.
• Perceiving the World: Question-guided Reinforcement Learning for Text-based Games
  • Yunqiu Xu, Meng Fang, Ling Chen, Yali Du, Joey Zhou, Chengqi Zhang
  • TLDR: We propose a novel method to decompose tasks and prune actions in text-based games by answering questions about the environment.
• Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization
  • Ruipeng Jia, Xingxing Zhang, Yanan Cao, Zheng Lin, Shi Wang, Furu Wei
  • TLDR: We propose NLSSum (Neural Label Search for Summarization), a novel neural label search algorithm for zero-shot multilingual extractive text summarization.
• Few-Shot Class-Incremental Learning for Named Entity Recognition
  • Rui Wang, Tong Yu, Handong Zhao, Sungchul Kim, Subrata Mitra, Ruiyi Zhang, Ricardo Henao
  • TLDR: We study class-incremental learning for Named Entity Recognition.
• Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation
  • Yingxiu Zhao, Zhiliang Tian, Huaxiu Yao, Yinhe Zheng, Dongkyu Lee, Yiping Song, Jian Sun, Nevin Zhang
  • TLDR: Memory imitation meta-learning for NLP.
• Quality Controlled Paraphrase Generation
  • Elron Bandel, Ranit Aharonov, Michal Shmueli-Scheuer, Ilya Shnayderman, Noam Slonim, Liat Ein-Dor
  • TLDR: We propose a quality-guided controlled controlled paraphrase generation model that allows directly controlling the quality dimensions of the generated paraphrase.
• Controllable Dictionary Example Generation: Generating Example Sentences for Specific Targeted Audiences
  • Xingwei He, Siu Ming Yiu
  • TLDR: We propose a controllable target-word-aware model for dictionary example sentence generation, which can generate suitable examples for targeted words with specific definitions while meeting the desired readability.
• AraT5: Text-to-Text Transformers for Arabic Language Generation
  • El Moatez Billah Nagoudi, AbdelRahim Elmadany, Muhammad Abdul-Mageed
  • TLDR: We present a novel benchmark for Arabic language GENeration and show that T5-style models can perform significantly better than mT5 on ARGEN.
• Legal Judgment Prediction via Event Extraction with Constraints
  • Yi Feng, Chuanyi Li, Vincent Ng
  • TLDR: We propose EPM, an Event-based Prediction Model with constraints, which surpasses existing SOTA models in performance on a standard LJP dataset.
• Answer-level Calibration for Free-form Multiple Choice Question Answering
  • Sawan Kumar
  • TLDR: We propose answer-level calibration for language models that can remove context-independent biases in terms of the probability of a choice without the associated context and use an unsupervised estimate of similarity with the full context to remove them.
• Learning When to Translate for Streaming Speech
  • Qian Dong, Yaoming Zhu, Mingxuan Wang, Lei Li
  • TLDR: We propose a simple yet effective method for translating streaming speech content.
• Compact Token Representations with Contextual Quantization for Efficient Document Re-ranking
  • Yingrui Yang, Yifan Qiao, Tao Yang
  • TLDR: We propose a novel approach to quantify token embeddings in a transformable re-ranking model by quantifying document-independent ranking contributions during codebook-based compression.
• Early Stopping Based on Unlabeled Samples in Text Classification
  • HongSeok Choi, Dongha Choi, Hyunju Lee
  • TLDR: We propose an early stopping method that uses unlabeled samples and show that it is better than existing methods.
• Meta-learning via Language Model In-context Tuning
  • Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, He He
  • TLDR: Meta-learning with only a few labeled examples.
• It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books
  • Bingsheng Yao, Dakuo Wang, Tongshuang Wu, Zheng Zhang, Toby Li, Mo Yu, Ying Xu
  • TLDR: We propose a novel question answering algorithm for story-telling that can generate pairs of QA pairs that can be used to test a student’s comprehension skills.
• Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
  • Rongzhi Zhang, Yue Yu, Pranav Shetty, Le Song, Chao Zhang
  • TLDR: We propose a novel method for iteratively and automatically discovering novel labeling rules from data to improve the WSL model.
• Constrained Multi-Task Learning for Bridging Resolution
  • Hideo Kobayashi, Yufang Hou, Vincent Ng
  • TLDR: We propose a novel constrained multi-task learning framework for bridging resolution, within which we (1) design cross-task consistency constraints to guide the learning process; (2) pre-train the entity coreference model in the multi- task framework on the large amount of publicly available coreference data; and (3) integrate prior knowledge encoded in rule-based resolvers.
• DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations
  • Sarik Ghazarian, Nuan Wen, Aram Galstyan, Nanyun Peng
  • TLDR: We propose DEAM, a Dialogue coherence Evaluation metric that relies on Abstract Meaning Representation to apply semantic-level Manipulations for incoherent (negative) data generation.
• HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarization
  • Shuyang Cao, Lu Wang
  • TLDR: We present a new task for hierarchical question-summary generation and a new dataset for hierarchical hierarchical question structure.
• De-Bias for Generative Extraction in Unified NER Task
  • Shuai Zhang, Yongliang Shen, Zeqi Tan, Yiquan Wu, Weiming Lu
  • TLDR: We propose a novel method to identify and eliminate the incorrect biases in the generation process of named entity recognition.
• An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels
  • Taylor Sorensen, Joshua Robinson, Christopher Rytting, Alexander Shaw, Kyle Rogers, Alexia Delorey, Mahmoud Khalil, Nancy Fulda, David Wingate
  • TLDR: We present a new method for selecting prompt templates for language models that are trained to perform specific tasks.
• Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation
  • Xinyi Wang, Sebastian Ruder, Graham Neubig
  • TLDR: We study strategies to synthesize textual or labeled data using lexicons, and how this data can be combined with monolingual or parallel text when available.
• Language-agnostic BERT Sentence Embedding
  • Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, Wei Wang
  • TLDR: We present a novel multilingual sentence embedding model that achieves 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba, well above the 65.5% achieved by LASER, while still performing competitively on monolingual transfer learning benchmarks.
• Nested Named Entity Recognition with Span-level Graphs
  • Juncheng Wan, Dongyu Ru, Weinan Zhang, Yong Yu
  • TLDR: We propose a retrieval-based span-level graph representation for nested named entity recognition.
• CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP
  • Yifei Luo, Minghui Xu, Deyi Xiong
  • TLDR: Cognitively inspired framework for NLP transfer learning.
• RoCBert: Robust Chinese Bert with Multimodal Contrastive Pretraining
  • Hui Su, Weiwei Shi, Xiaoyu Shen, Zhou Xiao, Tuo Ji, Jiarui Fang, Jie Zhou
  • TLDR: We propose a robust language model that is robust to various forms of adversarial attacks like word perturbation, synonyms, typos, etc.
• Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues
  • Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Zhongyu Wei, Weidong Zhan, Baobao Chang, Sujian Li, Tianyu Liu, Zhifang Sui
  • TLDR: We propose a new task for multi-modal reasoning in vision language cross-modality where a textual premise is the background presumption on each source image.
• Parallel Instance Query Network for Named Entity Recognition
  • Yongliang Shen, Xiaobin Wang, Zeqi Tan, Guangwei Xu, Pengjun Xie, Fei Huang, Weiming Lu, Yueting Zhuang
  • TLDR: We propose Parallel Instance Query Network (PIQN), a novel approach to named entity recognition that learns query semantics and uses label assignment to extract entities from a sentence in parallel.
• ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation
  • Chang Liu, Xu Tan, Chongyang Tao, Zhenxin Fu, Dongyan Zhao, Tie-Yan Liu, Rui Yan
  • TLDR: Proposed a novel dialogue generation framework that incorporates the simulated dialogue futures in the inference phase to enhance response generation.
• Modeling Multi-hop Question Answering as Single Sequence Prediction
  • Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou, Nitish Shirish Keskar, Caiming Xiong
  • TLDR: We propose a simple generative question answering model for multi-hop QA that is more interpretable and more accurate than the current model.
• Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension
  • Linjuan Wu, Shaojuan Wu, Xiaowang Zhang, Deyi Xiong, Shizhan Chen, Zhiqiang Zhuang, Zhiyong Feng
  • TLDR: We propose a novel multilingual MRC framework equipped with a Siamese Semantic Disentanglement Model (S2DM) to disassociate semantics from syntax in representations learned by multilingual pre-trained models.
• Multi-Granularity Structural Knowledge Distillation for Language Model Compression
  • Chang Liu, Chongyang Tao, Jiazhan Feng, Dongyan Zhao
  • TLDR: We present a novel knowledge distillation framework that gathers intermediate representations from multiple semantic granularities (e.g., tokens, spans and samples) and forms the knowledge as more sophisticated structural relations specified as the pair-wise interactions and the triplet-wise geometric angles based on multi-granularity representations.
• Auto-Debias: Debiasing Masked Language Models with Automated Biased Prompts
  • Yue Guo, Yi Yang, Ahmed Abbasi
  • TLDR: We propose an automatic method to mitigate the biases in pretrained language models.
• Where to Go for the Holidays: Towards Mixed-Type Dialogs for Clarification of User Goals
  • Zeming Liu, Jun Xu, Zeyang Lei, Haifeng Wang, Zheng-Yu Niu, Hua Wu
  • TLDR: We propose a new mixed-type dialog model with a novel Prompt-based continual learning mechanism.
• Semi-supervised Domain Adaptation for Dependency Parsing with Dynamic Matching Network
  • Ying Li, Shuaike Li, Min Zhang
  • TLDR: We propose a dynamic matching network on the shared-private model for semi-supervised cross-domain dependency parsing and show that it can outperform various domain-specific parsing models.
• A Closer Look at How Fine-tuning Changes BERT
  • Yichu Zhou, Vivek Srikumar
  • TLDR: We propose a new theory of how fine-tuning contextualized representations in NLP improves classification performance by increasing the distances between examples associated with different labels.
• Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
  • Wu Hong, Zhuosheng Zhang, Jinyuan Wang, Hai Zhao
  • TLDR: We present a novel approach to capture passage with internal representation conflicts from improper modeling granularity.
• FaiRR: Faithful and Robust Deductive Reasoning over Natural Language
  • Soumya Sanyal, Harman Singh, Xiang Ren
  • TLDR: We propose a modular deductive reasoning model that is robust to language perturbations and faster at inference than previous works on existing reasoning datasets.
• HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation
  • Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang
  • TLDR: We present a new dataset, HiTab, to study question answering (QA) and natural language generation (NLG) over hierarchical tables.
• Doctor Recommendation in Online Health Forums via Expertise Learning
  • Xiaoxin Lu, Yubo Zhang, Jing Li, Shi Zong
  • TLDR: We propose a novel algorithm for doctor recommendation based on user profiles and previous dialogues to enable automatic pairing of a patient to a doctor with relevant expertise.
• Continual Prompt Tuning for Dialog State Tracking
  • Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, Minlie Huang
  • TLDR: We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
• There’s a Time and Place for Reasoning Beyond the Image
  • Xingyu Fu, Ben Zhou, Ishaan Chandratreya, Carl Vondrick, Dan Roth
  • TLDR: We present a novel architecture for deep segment-wise reasoning of images with information about their time and location.
• FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
  • Zhoujun Cheng, Haoyu Dong, Ran Jia, Pengfei Wu, Shi Han, Fan Cheng, Dongmei Zhang
  • TLDR: We propose FORTAP, a new method for spreadsheet table pretraining that uses formulas to train spreadsheet table encoders.
• Multimodal fusion via cortical network inspired losses
  • Shiv Shankar
  • TLDR: We propose a novel approach to integrate information from different modalities in neural networks by introducing neural dependencies in the loss functions.
• Modeling Temporal-Modal Entity Graph for Procedural Multimodal Machine Comprehension
  • Huibin Zhang, Zhengkun Zhang, Yao Zhang, Jun Wang, Yufan Li, Ning Jiang, Xin Wei, Zhenglu Yang
  • TLDR: We propose a novel graph structure for procedural multi-modal machine comprehension and propose a graph aggregation module for graph-based reasoning.
• Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning
  • Swarnadeep Saha, Prateek Yadav, Mohit Bansal
  • TLDR: We analyze graph generation by pre-trained language models and propose simple yet effective ways of graph perturbations via node and edge edit operations that lead to structurally and semantically positive and negative graphs.
• Unsupervised Extractive Opinion Summarization Using Sparse Coding
  • Somnath Basu Roy Chowdhury, Chao Zhao, Snigdha Chaturvedi
  • TLDR: We present a novel algorithm for extractive opinion summarization in an unsupervised manner.
• LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution
  • George Michalopoulos, Ian McKillop, Alexander Wong, Helen Chen
  • TLDR: We introduce LexSubCon, an end-to-end lexical substitution framework based on contextual embedding models that can identify highly-accurate substitute candidates.
• Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation
  • Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, Dilek Hakkani-Tur
  • TLDR: We present Think-Before-Speaking (TBS), a generative approach to first externalize implicit commonsense knowledge (
• Flow-Adapter Architecture for Unsupervised Machine Translation
  • Yihong Liu, Haris Jabbar, Hinrich Schuetze
  • TLDR: We propose a flow-adapter architecture for unsupervised NMT that captures language-specific sentence representations separately for each language using normalizing flows and use a simple transformation of these latent representations for translating from one language to another.
• Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning
  • Demian Ghalandari, Chris Hokamp, Georgiana Ifrim
  • TLDR: We use reinforcement learning to train efficient sentence compression models that are also fast when generating predictions.
• Tracing Origins: Coreference-aware Machine Reading Comprehension
  • Baorong Huang, Zhuosheng Zhang, Hai Zhao
  • TLDR: We use the coreference information of entities to enhance the word embeddings from the pre-trained language model, in order to highlight the corelanguage features of the entities that must be identified for coreference-intensive question answering in QUOREF.
• WatClaimCheck: A new Dataset for Claim Entailment and Inference
  • Kashif Khan, Ruizhe Wang, Pascal Poupart
  • TLDR: We present a new dataset for automated fact checking and an evaluation of state of the art algorithms.
• FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metrics for Automatic Text Generation
  • Moussa Kamal Eddine, Guokan Shang, Antoine Tixier, Michalis Vazirgiannis
  • TLDR: We propose FrugalScore, a new metric for natural language generation that is fast and reliable while retaining most of its original performance.
• A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation
  • Shashi Narayan, Gonçalo Simões, Yao Zhao, Joshua Maynez, Dipanjan Das, Michael Collins, Mirella Lapata
  • TLDR: We propose Composition Sampling, a simple but effective method to generate diverse outputs for conditional generation of higher quality compared to previous stochastic decoding strategies.
• Synthetic Question Value Estimation for Domain Adaptation of Question Answering
  • Xiang Yue, Ziyu Yao, Huan Sun
  • TLDR: We propose a novel question value estimator that directly estimates the usefulness of synthetic questions for improving the target-domain QA performance.
• Better Language Model with Hypernym Class Prediction
  • He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
  • TLDR: Class-based language models (LMs) have been long devised to address context sparsity in
• Tackling Fake News Detection by Continually Improving Social Context Representations using Graph Neural Networks
  • Nikhil Mehta, Maria Leonor Pacheco, Dan Goldwasser
  • TLDR: We propose a graph framework for fake news detection and show that inference operators can help to explain the relations between sources, articles they publish, and engaging users on social media in a graph.
• Understanding Gender Bias in Knowledge Base Embeddings
  • Yupei Du, Qi Zheng, Yuanbin Wu, Man Lan, Yan Yang, Meirong Ma
  • TLDR: We develop two novel bias measures for knowledge base embeddings and trace their origins in KB.
• Computational Historical Linguistics and Language Diversity in South Asia
  • Aryaman Arora, Adam Farris, Samopriya Basu, Suresh Kolichala
  • TLDR: We present a new approach to tackling the data scarcity problem in language technology in South Asia, and propose a new way to study language history to overcome it.
• Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization
  • Faisal Ladhak, Esin Durmus, He He, Claire Cardie, Kathleen McKeown
  • TLDR: We present a framework for evaluating the effective faithfulness of summarization systems, by generating a faithfulness-abstractiveness trade-off curve that serves as a control at different operating points on the abstractiveness spectrum.
• Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang
  • Daphna Keidar, Andreas Opedal, Zhijing Jin, Mrinmaya Sachan
  • TLDR: We study the role of word type in language evolution and show that slang words undergo less semantic change but tend to have larger frequency shifts over time.
• Spurious Correlations in Reference-Free Evaluation of Text Generation
  • Esin Durmus, Faisal Ladhak, Tatsunori Hashimoto
  • TLDR: We show that the current evaluation metrics of summarization and dialog generation are relying on spurious features.
• On The Ingredients of an Effective Zero-shot Semantic Parser
  • Pengcheng Yin, John Wieting, Avirup Sil, Graham Neubig
  • TLDR: We analyze zero-shot parsers through the lenses of the language and logical gaps (Herzig and Berant, 2019) and propose a new model for zero-labeling and efficient zero-shooting.
• Bias Mitigation in Machine Translation Quality Estimation
  • Hanna Behnke, Marina Fomicheva, Lucia Specia
  • TLDR: We present a new approach to reduce the partial input bias in machine translation quality estimation by training auxiliary tasks that mitigate the bias.
• Unified Speech-Text Pre-training for Speech Translation and Recognition
  • Yun Tang, Hongyu Gong, Ning Dong, Changhan Wang, Wei-Ning Hsu, Jiatao Gu, Alexei Baevski, Xian Li, Abdelrahman Mohamed, Michael Auli, Juan Pino
  • TLDR: We propose a novel method to jointly pre-train speech and text in an encoder-decoder modeling framework for speech translation and recognition.
• Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability
  • Yoshinari Fujinuma, Jordan Boyd-Graber, Katharina Kann
  • TLDR: We show that increasing the number of pretraining languages improves zero-shot performance on unseen target languages, but not the number.
• Structured Pruning Learns Compact and Accurate Models
  • Mengzhou Xia, Zexuan Zhong, Danqi Chen
  • TLDR: We propose a task-specific structured structured pruning method that outperforms distillation methods in both accuracy and latency, and also improves the performance of unpruned models.
• How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language
  • Shiyue Zhang, Ben Frey, Mohit Bansal
  • TLDR: We present a case study of Cherokee, a severely-endangered Native American language, and propose a few ways in which NLP can help revitalize it.
• Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization
  • Sanjeev Kumar Karn, Ning Liu, Hinrich Schuetze, Oladimeji Farri
  • TLDR: We propose a novel two-step approach for report summarization that leads to a more precise and explainable summary.
• Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue
  • Jiawei Zhou, Jason Eisner, Michael Newman, Emmanouil Antonios Platanios, Sam Thomson
  • TLDR: We propose a general framework for online semantic parsing that learns prefix-to-program prediction and executes function calls while the user is still speaking.
• Few-Shot Tabular Data Enrichment Using Fine-Tuned Transformer Architectures
  • Asaf Harari, Gilad Katz
  • TLDR: We propose Few-Shot Transformer based Enrichment, a generic and robust framework for the enrichment of tabular datasets using external unstructured data.
• Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
  • Yusen Zhang, Ansong Ni, Ziming Mao, Chen Henry Wu, Chenguang Zhu, Budhaditya Deb, Ahmed Awadallah, Dragomir Radev, Rui Zhang
  • TLDR: We propose a novel method for summarizing long text that can be used for many summarization tasks.
• Open Domain Question Answering with A Unified Knowledge Interface
  • Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao
  • TLDR: We propose a unified interface for open-domain question answering over data and text where verbalized tables from Wikipedia and graphs from Wikidata are used as augmented knowledge sources.
• Principled Paraphrase Generation with Parallel Corpora
  • Aitor Ormazabal, Mikel Artetxe, Aitor Soroa, Gorka Labaka, Eneko Agirre
  • TLDR: We propose a new adversarial term for machine translation that allows for more principled and efficient paraphrase generation.
• GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems
  • Bosheng Ding, Junjie Hu, Lidong Bing, Mahani Aljunied, Shafiq Joty, Luo Si, Chunyan Miao
  • TLDR: We present a novel data curation method for multilingual task-oriented dialogue systems that generates a large-scale multilingual ToD dataset globalized from an English ToD datasets for three unexplored use cases of multilingual toD systems.
• Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation
  • Dongha Choi, HongSeok Choi, Hyunju Lee
  • TLDR: We propose a domain knowledge transferring framework for PLMs without additional in-domain pretraining.
• Retrieval-guided Counterfactual Generation for QA
  • Bhargavi Paranjape, Matthew Lamm, Ian Tenney
  • TLDR: We develop a Retrieve-Generate-Filter-based counterfactual evaluation and training method for question answering that improves performance on out-of-domain and challenging evaluation sets over and above existing methods, in both the reading comprehension and open-domain QA settings.
• DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization
  • Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed Awadallah, Dragomir Radev
  • TLDR: We present a novel dynamic latent extraction approach for abstractive long-input summarization.
• Searching for fingerspelled content in American Sign Language
  • Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu
  • TLDR: We propose a novel search and retrieval model for sign language video that detects fingerspelling and matches it to text sequences.
• Skill Induction and Planning with Latent Language
  • Pratyusha Sharma, Antonio Torralba, Jacob Andreas
  • TLDR: We present a framework for learning hierarchical policies from demonstrations, using sparse natural language annotations to guide the discovery of reusable skills for autonomous decision-making.
• Fully-Semantic Parsing and Generation: the BabelNet Meaning Representation
  • Abelardo Carlos Martínez Lorenzo, Marco Maru, Roberto Navigli
  • TLDR: We present the BabelNet Meaning Representation (BMR), an interlingual formalism that abstracts away from language-specific constraints by taking advantage of the multilingual semantic resources of BabelNet and VerbAtlas.
• Leveraging Similar Users for Personalized Language Modeling with Limited Data
  • Charles Welch, Chenxi Gu, Jonathan K. Kummerfeld, Veronica Perez-Rosas, Rada Mihalcea
  • TLDR: We propose a new approach to find and train personalized language models that are similar to existing users and use the data from existing users to train them.
• DEEP: DEnoising Entity Pre-training for Neural Machine Translation
  • Junjie Hu, Hiroaki Hayashi, Kyunghyun Cho, Graham Neubig
  • TLDR: We propose DEEP, a DEnoising Entity Pre-training method that leverages large amounts of monolingual data and a knowledge base to improve named entity translation accuracy within sentences.
• Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
  • Bin Liang, Chenwei Lou, Xiang Li, Min Yang, Lin Gui, Yulan He, Wenjie Pei, Ruifeng Xu
  • TLDR: We propose a novel multi-modal sarcasm detection method by constructing a cross-modality graph for each instance to explicitly draw the ironic relations between textual and visual modalities.
• Composable Sparse Fine-Tuning for Cross-Lingual Transfer
  • Alan Ansell, Edoardo Ponti, Anna Korhonen, Ivan Vulić
  • TLDR: We present a new method for sparse fine-tuning the entire set of parameters of a large pretrained model that outperforms adapters in zero-shot cross-lingual transfer learning.
• Toward Annotator Group Bias in Crowdsourcing
  • Haochen Liu, Joseph Thekinen, Sinem Mollaoglu, Da Tang, Ji Yang, Youlong Cheng, Hui Liu, Jiliang Tang
  • TLDR: We show that annotators within the same demographic group tend to show consistent group bias in annotation tasks and thus we conduct an initial study on annotator group bias.
• Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation
  • Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, Marco Turchi
  • TLDR: Gender bias in speech translation is a problem in natural language translation, and we investigate its impact on different lexical categories and agreement phenomena.
• Answering Open-Domain Multi-Answer Questions via a Recall-then-Verify Framework
  • Zhihong Shao, Minlie Huang
  • TLDR: We propose a novel approach to answer open-domain multi-answer questions with a recall-then-verify framework, which separates the reasoning process of each answer so that we can make better use of retrieved evidence while also leveraging large models under the same memory constraint.
• Probing as Quantifying Inductive Bias
  • Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin, Ryan Cotterell
  • TLDR: We propose a Bayesian framework for quantifying the amount of inductive bias that contextual representations encode on a variety of NLP tasks.
• Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
  • Yanyang Li, Fuli Luo, Runxin Xu, Songfang Huang, Fei Huang, Liwei Wang
  • TLDR: We investigate three aspects of structured pruning on multilingual pre-trained language models: settings, algorithms, and efficiency.
• GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models
  • Changye Li, David Knopman, Weizhe Xu, Trevor Cohen, Serguei Pakhomov
  • TLDR: We propose a novel method for generalizing deep learning models to generalize beyond the small reference sets that are publicly available for research.
• An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models
  • Nicholas Meade, Elinor Poole-Dayan, Siva Reddy
  • TLDR: We present empirical evidence on the effectiveness of five recently proposed bias mitigation techniques and their impact on language modeling ability.
• Exploring and Adapting Chinese GPT to Pinyin Input Method
  • Minghuan Tan, Yong Dai, Duyu Tang, Zhangyin Feng, Guoping Huang, Jing Jiang, Jiwei Li, Shuming Shi
  • TLDR: We make the first exploration to leverage Chinese GPT for pinyin input method.
• Enhancing Cross-lingual Natural Language Inference by Prompt-learning from Cross-lingual Templates
  • Kunxun Qi, Hai Wan, Jianfeng Du, Haolan Chen
  • TLDR: We propose a novel prompt-learning based framework for enhancing cross-lingual natural language inference and show that it significantly outperforms existing methods.
• Sense Embeddings are also Biased – Evaluating Social Biases in Static and Contextualised Sense Embeddings
  • Yi Zhou, Masahiro Kaneko, Danushka Bollegala
  • TLDR: We propose novel sense-specific bias evaluation measures for evaluating the social biases in sense embeddings and show that even in cases where no biases are found at word-level, there still exist worrying levels of social biases at sense-level.
• Hybrid Semantics for Goal-Directed Natural Language Generation
  • Connor Baumler, Soumya Ray
  • TLDR: We propose a novel goal-directed sentence generation system that uses distributional semantics to generate sentences with better quality than S-STRUCT.
• Predicting Intervention Approval in Clinical Trials through Multi-Document Summarization
  • Georgios Katsimpras, Georgios Paliouras
  • TLDR: We propose a new method to predict the effectiveness of an intervention in a clinical trial using evidence sentences extracted from PubMed articles.
• BiTIIMT: A Bilingual Text-infilling Method for Interactive Machine Translation
  • Yanling Xiao, Lemao Liu, Guoping Huang, Qu Cui, Shujian Huang, Shuming Shi, Jiajun Chen
  • TLDR: We propose a novel BiTIIMT system, Bilingual Text-Infilling for Interactive Neural Machine Translation.
• Distributionally Robust Finetuning BERT for Covariate Drift in Spoken Language Understanding
  • Samuel Broscheit, Quynh Do, Judith Gaspers
  • TLDR: We propose a method that exploits natural variations in data to create covariate drift in spoken language understanding datasets and show that it improves robustness under covariate drifting.
• Enhancing Chinese Pre-trained Language Model via Heterogeneous Linguistics Graph
  • Yanzeng Li, Jiangxia Cao, Xin Cong, Zhenyu Zhang, Bowen Yu, Hongsong Zhu, Tingwen Liu
  • TLDR: We propose a task-free enhancement module termed as Heterogeneous Linguistics Graph (HLG) to enhance Chinese pre-trained language models by integrating linguistics knowledge.
• Divide and Denoise: Learning from Noisy Labels in Fine-Grained Entity Typing with Cluster-Wise Loss Correction
  • Kunyuan Pang, Haoyu Zhang, Jie Zhou, Ting Wang
  • TLDR: We propose a clustering-based loss correction framework for fine-grained entity typeting that achieves competitive performance over existing competitive systems.
• Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation
  • Xinyu Pi, Bing Wang, Yan Gao, Jiaqi Guo, Zhoujun Li, Jian-Guang Lou
  • TLDR: We propose the Adversarial Table Perturbation (ATP) as a new adversarial perturbation benchmark for robustness of Text-to-SQL parsers.
• Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation
  • Chenze Shao, Yang Feng
  • TLDR: Neural networks tend to gradually forget the previously learned knowledge when learning multiple tasks sequentially from dynamic data distributions.
• Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages
  • Ehsan Aghazadeh, Mohsen Fayyaz, Yadollah Yaghoobzadeh
  • TLDR: We investigate the metaphoricity of pre-trained language models and show that they do encode metaphorical knowledge.
• Discrete Opinion Tree Induction for Aspect-based Sentiment Analysis
  • Chenhua Chen, Zhiyang Teng, Zhongqing Wang, Yue Zhang
  • TLDR: We propose an aspect-specific and language-agnostic discrete latent opinion tree model as an alternative structure to explicit dependency trees for aspect-based sentiment classification.
• Investigating Non-local Features for Neural Constituency Parsing
  • Leyang Cui, Sen Yang, Yue Zhang
  • TLDR: We propose a novel method for injecting non-local features into the training process of a local span-based parser, by predicting constituent features of the input data.
• Learning from Sibling Mentions with Scalable Graph Inference in Fine-Grained Entity Typing
  • Yi Chen, Jiayang Cheng, Haiyun Jiang, Lemao Liu, Haisong Zhang, Shuming Shi, Ruifeng Xu
  • TLDR: We propose to exploit sibling mentions for enhancing the mention representations.
• A Variational Hierarchical Model for Neural Cross-Lingual Summarization
  • Yunlong Liang, Fandong Meng, Chulun Zhou, Jinan Xu, Yufeng Chen, Jinsong Su, Jie Zhou
  • TLDR: We propose a hierarchical model for cross-lingual summarization task based on the conditional variational auto-encoder.
• On the Robustness of Question Rewriting Systems to Questions of Varying Hardness
  • Hai Ye, Hwee Tou Ng, Wenjuan Han
  • TLDR: We propose a novel learning framework for question rewriting in context that improves the robustness of a QR system to questions varying in rewriting hardness or difficulty.
• OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages
  • Prem Selvaraj, Gokul Nc, Pratyush Kumar, Mitesh Khapra
  • TLDR: We present a novel approach to sign language recognition by using pose extracted through pretrained models as the standard modality of data in this work to reduce training time and enable efficient inference.
• bert2BERT: Towards Reusable Pretrained Language Models
  • Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu
  • TLDR: We propose bert2BERT, a novel method for pre-training large language models by using advanced knowledge for parameter initialization and significantly improve the pre-train efficiency of the large model.
• Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis
  • Yan Ling, Jianfei Yu, Rui Xia
  • TLDR: We propose a unified multimodal encoder-decoder architecture for MABSA and a task-specific Vision-LanguagePre-training framework for MabSA.
• “You might think about slightly revising the title”: Identifying Hedges in Peer-tutoring Interactions
  • Yann Raphalen, Chloé Clavel, Justine Cassell
  • TLDR: We present a computational framework for identifying hedges in peer-tutoring conversations and show that it outperforms existing models.
• Efficient Cluster-Based k-Nearest-Neighbor Machine Translation
  • Dexin Wang, Kai Fan, Boxing Chen, Deyi Xiong
  • TLDR: Weird News Photos:
• Headed-Span-Based Projective Dependency Parsing
  • Songlin Yang, Kewei Tu
  • TLDR: We propose a new method for projective dependency parsing based on headed spans.
• Decoding Part-of-Speech from Human EEG Signals
  • Alex Murphy, Bernd Bohnet, Ryan McDonald, Uta Noppeney
  • TLDR: We show that information about word length, frequency and word class is encoded by the brain at different post-stimulus latencies.
• Robust Lottery Tickets for Pre-trained Language Models
  • Rui Zheng, Bao Rong, Yuhao Zhou, Di Liang, Sirui Wang, Wei Wu, Tao Gui, Qi Zhang, Xuanjing Huang
  • TLDR: We propose a novel adversarial adversarial robustness evaluation method based on learning binary weight masks to identify robust tickets hidden in the original PLMs.
• Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification
  • Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, Maosong Sun
  • TLDR: We propose a new method for improving and stabilizing prompttuning of pre-trained language models with external knowledge bases.
• Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages
  • Xu Han, Yuqi Luo, Weize Chen, Zhiyuan Liu, Maosong Sun, Zhou Botong, Hao Fei, Suncong Zheng
  • TLDR: We propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
• MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER
  • Ran Zhou, Xin Li, Ruidan He, Lidong Bing, Erik Cambria, Luo Si, Chunyan Miao
  • TLDR: We propose a novel data augmentation framework for low-resource neural network evaluation, which improves NER performance and provides rich entity regularity knowledge.
• Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings
  • Shib Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Li, Andrew McCallum
  • TLDR: We provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective.
• IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks
  • Liying Cheng, Lidong Bing, Ruidan He, Qian Yu, Yan Zhang, Luo Si
  • TLDR: We present a comprehensive and large dataset for argument mining and propose two new integrated argument mining tasks for AI debate preparation.
• PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation
  • Zhe Hu, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Hua Wu, Lifu Huang
  • TLDR: Autoregressive self-attention for long-form text generation.
• CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation
  • Pei Ke, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Xiaoyan Zhu, Minlie Huang
  • TLDR: We propose an unsupervised metric for evaluating controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks.
• Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking
  • Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang, Yixuan Liu
  • TLDR: We propose a new approach to dynamically select the relevant dialogue contents corresponding to each slot for state updating.
• Are Prompt-based Models Clueless?
  • Pride Kavumba, Ryo Takahashi, Yusuke Oda
  • TLDR: We show that few-shot prompt-based models also exploit superficial cues.
• Learning Confidence for Transformer-based Neural Machine Translation
  • Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, Mu Li
  • TLDR: We propose an unsupervised confidence estimate for neural machine translation that can estimate the confidence of the model prediction and assess underlying risk in real-world settings.
• Things not Written in Text: Exploring Spatial Commonsense from Visual Signals
  • Xiao Liu, Da Yin, Yansong Feng, Dongyan Zhao
  • TLDR: We propose a spatial commonsense benchmark that focuses on the relative scales of objects, and the positional relationship between people and objects under different actions.
• Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation
  • Songming Zhang, Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu, Jian Liu, Jie Zhou
  • TLDR: We propose a target-context-aware metric, named conditional bilingual mutual information (CBMI), which makes it feasible to supplement target context information for statistical metrics.
• ClusterFormer: Neural Clustering Attention for Efficient and Effective Transformer
  • Ningning Wang, Guobing Gan, Peng Zhang, Shuai Zhang, Junqiu Wei, Qun Liu, Xin Jiang
  • TLDR: We propose a new sparse method for Transformer that uses neural clustering to improve the efficiency of Transformer.
• Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks
  • Songlin Yang, Kewei Tu
  • TLDR: We propose a novel pointing mechanism for constituency parsing and named entity recognition that uses a pointer network to predict the next boundary of a constituency tree.
• Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
  • Liang Ding, Longyue Wang, Shuming Shi, Dacheng Tao, Zhaopeng Tu
  • TLDR: Knowledge distillation is a promising alternative to knowledge distillation for non-autoregressive translation.
• Dependency Parsing as MRC-based Span-Span Prediction
  • Leilei Gan, Yuxian Meng, Kun Kuang, Xiaofei Sun, Chun Fan, Fei Wu, Jiwei Li
  • TLDR: We propose a new method for dependency parsing that constructs dependency trees by directly modeling span-span (in other words, subtree-subtree) relations.
• Adversarial Soft Prompt Tuning for Cross-Domain Sentiment Analysis
  • Hui Wu, Xiaodong Shi
  • TLDR: We present a new approach for cross-domain sentiment analysis using GPT-3 and GPT3-3-2.
• Generating Scientific Claims for Zero-Shot Scientific Fact Checking
  • Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isabelle Augenstein, Lucy Wang
  • TLDR: We propose scientific claim generation and zero-shot fact checking for biomedical claims.
• Modeling Dual Read/Write Paths for Simultaneous Machine Translation
  • Shaolei Zhang, Yang Feng
  • TLDR: We propose a method of dual-path SiMT which introduces duality constraints to direct the read/write path.
• ExtEnD: Extractive Entity Disambiguation
  • Edoardo Barba, Luigi Procopio, Roberto Navigli
  • TLDR: We propose ExtEnD, a novel local formulation for Entity Disambiguation that outperforms all its competitors in terms of both data efficiency and raw performance.
• Hierarchical Sketch Induction for Paraphrase Generation
  • Tom Hosking, Hao Tang, Mirella Lapata
  • TLDR: We propose a generative model of paraphrase generation, that encourages syntactic diversity by conditioning on an explicit syntactic sketch.
• Alignment-Augmented Consistent Translation for Multilingual Open Information Extraction
  • Keshav Kolluru, Muqeeth Mohammed, Shubham Mittal, Soumen Chakrabarti, Mausam .
  • TLDR: We present a novel two-stage generative OpenIE model that outputs for each sentence: 1) relations in the first stage and 2) all extractions containing the relation in the second stage.
• Text-to-Table: A New Way of Information Extraction
  • Xueqing Wu, Jiacheng Zhang, Hang Li
  • TLDR: We formalize text-to-table as a sequence-to sequence (seq2seq) problem and develop a new method for extracting text-based table generation and table generation.
• Accelerating Code Search with Deep Hashing and Code Classification
  • Wenchao Gu, Yanlin Wang, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Michael Lyu
  • TLDR: We propose a novel method CoSHC to accelerate code search with deep hashing and code classification, aiming to perform efficient code search without sacrificing too much accuracy.
• Other Roles Matter! Enhancing Role-Oriented Dialogue Summarization via Role Interactions
  • Haitao Lin, Junnan Zhu, Lu Xiang, Yu Zhou, Jiajun Zhang, Chengqing Zong
  • TLDR: We propose a novel role interaction enhanced method for role-oriented dialogue summarization that uses cross attention and decoder self-attention interactions to interactively acquire other roles’ critical information.
• ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
  • Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, Daxin Jiang
  • TLDR: We propose a general Correlation-aware context-to-event Transformer for event-centric reasoning, which can be used for a wide range of event-based reasoning tasks.
• Measuring and Mitigating Name Biases in Neural Machine Translation
  • Jun Wang, Benjamin Rubinstein, Trevor Cohn
  • TLDR: We show that given name gender bias in neural machine translation is a common source of bias in NMT systems, and propose a simple data augmentation method to mitigate it.
• Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation
  • Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, Michael Lyu
  • TLDR: We present a substantial step in better understanding the SOTA sequence-to-sequence (Seq2Seq) pretraining for neural machine translation (NMT).
• MSCTD: A Multimodal Sentiment Chat Translation Dataset
  • Yunlong Liang, Fandong Meng, Jinan Xu, Yufeng Chen, Jie Zhou
  • TLDR: We introduce a new multimodal chat translation task with multimodality and sentiment features and provide a new benchmark for multimodally chat translation.
• Learning Disentangled Textual Representations via Statistical Measures of Similarity
  • Pierre Colombo, Guillaume Staerman, Nathan Noiry, Pablo Piantanida
  • TLDR: We present a novel family of regularizers for learning disentangled textual representations that do not require training.
• On the Sensitivity and Stability of Model Interpretations in NLP
  • Fan Yin, Zhouxing Shi, Cho-Jui Hsieh, Kai-Wei Chang
  • TLDR: We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existing removal-based criteria.
• Down and Across: Introducing Crossword-Solving as a New NLP Benchmark
  • Saurabh Kulshreshtha, Olga Kovaleva, Namrata Shivagunde, Anna Rumshisky
  • TLDR: We introduce solving crossword puzzles as a new natural language understanding task and propose a framework for evaluating performance.
• Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets
  • Yuxiang Wu, Matt Gardner, Pontus Stenetorp, Pradeep Dasigi
  • TLDR: We propose to tackle this problem by generating a debiased version of a dataset, which can then be used to train a debuased, off-the-shelf model, by simply replacing its training data.
• GL-CLeF: A Global–Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding
  • Libo Qin, Qiguang Chen, Tianbao Xie, Qixin Li, Jian-Guang Lou, Wanxiang Che, Min-Yen Kan
  • TLDR: We present Global-Local Contrastive Learning Framework for zero-shot cross-lingual spoken language understanding, which achieves to explicitly align representations of similar sentences across languages.
• Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER
  • Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
  • TLDR: We present a simple demonstration-based learning method for named entity recognition which uses tasks to demonstrate entity types and demonstrate their properties.
• Contextual Representation Learning beyond Masked Language Modeling
  • Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu, Hao Zhou, Lei Li
  • TLDR: We propose TACO, a simple yet effective representation learning approach to directly model global semantics.
• Efficient Hyper-parameter Search for Knowledge Graph Embedding
  • Yongqi Zhang, Zhanke Zhou, Quanming Yao, Yong Li
  • TLDR: We propose an efficient two-stage search algorithm for hyperparameters in knowledge graph learning, which can consistently find better HPs than the baseline algorithms within the same time budget.
• A Meta-framework for Spatiotemporal Quantity Extraction from Text
  • Qiang Ning, Ben Zhou, Hao Wu, Haoruo Peng, Chuchu Fan, Matt Gardner
  • TLDR: We propose a meta-framework for spatiotemporal quantity extraction and demonstrate its general and extensible applicability to news events.
• Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer
  • Woojeong Jin, Dong-Ho Lee, Chenguang Zhu, Jay Pujara, Xiang Ren
  • TLDR: We study whether integrating visual knowledge into a language model can fill the gap.
• A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models
  • Woojeong Jin, Yu Cheng, Yelong Shen, Weizhu Chen, Xiang Ren
  • TLDR: We study prompt-based low-resource learning of vision-language models with few-shot learning and show that it improves few-shots performance and improves captioning performance.
• Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation
  • Chengwei Qin, Shafiq Joty
  • TLDR: We propose a novel method for continual few-shot relation learning based on embedding space regularization and data augmentation.
• Variational Graph Autoencoding as Cheap Supervision for AMR Coreference Resolution
  • Irene Li, Linfeng Song, Kun Xu, Dong Yu
  • TLDR: We propose a general pretraining method using variational graph autoencoder for AMR coreference resolution, which can leverage any general AMR corpus and even automatically parsed AMR data.
• Identifying Chinese Opinion Expressions with Extremely-Noisy Crowdsourcing Annotations
  • Xin Zhang, Guangwei Xu, Yueheng Sun, Meishan Zhang, Xiaobin Wang, Min Zhang
  • TLDR: We propose to generate synthetic training samples by a pertinent mixup strategy to make the training and testing highly consistent.
• Sequence-to-Sequence Knowledge Graph Completion and Question Answering
  • Apoorv Saxena, Adrian Kochsiek, Rainer Gemulla
  • TLDR: We present a scalable and versatile knowledge graph embedding model that outperforms previous models on incomplete KG link prediction and question answering over incomplete KGs.
• Learning to Mediate Disparities Towards Pragmatic Communication
  • Yuwei Bao, Sayan Ghosh, Joyce Chai
  • TLDR: We propose a novel rational reasoning framework for improving language communication by learning the speaker-listener disparity and adjusting the speech accordingly.
• Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
  • Luyu Gao, Jamie Callan
  • TLDR: We propose a novel dense retrieval architecture that learns to condense information into the dense vector through LM pre-training and a novel contrastive loss to warm up the passage embedding space.
• Multimodal Dialogue Response Generation
  • Qingfeng Sun, Yujing Wang, Can Xu, Kai Zheng, Yaming Yang, Huang Hu, Fei Xu, Jessica Zhang, Xiubo Geng, Daxin Jiang
  • TLDR: We present a novel multimodal dialogue response generation method based on multimodality and a novel conversational agent.
• CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph Completion
  • Guanglin Niu, Bo Li, Yongfei Zhang, Shiliang Pu
  • TLDR: We propose a novel and scalable Commonsense-Aware knowledge graph embedding framework to automatically extract commonsense from factual triples with entity concepts.
• Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation
  • Chulun Zhou, Fandong Meng, Jie Zhou, Min Zhang, Hongji Wang, Jinsong Su
  • TLDR: We propose a new training framework for neural machine translation models that incorporates bidirectional global context into the model.
• BRIO: Bringing Order to Abstractive Summarization
  • Yixin Liu, Pengfei Liu, Dragomir Radev, Graham Neubig
  • TLDR: We propose a novel training paradigm for abstractive summarization models that uses a non-deterministic distribution to estimate probabilities of candidate summaries according to their quality.
• Leveraging Relaxed Equilibrium by Lazy Transition for Sequence Modeling
  • Xi Ai, Bin Fang
  • TLDR: We present a lazy transition mechanism to adjust the significance of iterative refinements for each token representation in sequence modeling.
• FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Framework
  • Santiago Castro, Ruoyao Wang, Pingxuan Huang, Ian Stewart, Oana Ignat, Nan Liu, Jonathan Stroud, Rada Mihalcea
  • TLDR: We propose fill-in-the-blanks as a video understanding evaluation framework and introduce FIBER – a novel dataset consisting of 28,000 videos and descriptions in support of this evaluation framework.
• KenMeSH: Knowledge-enhanced End-to-end Biomedical Text Labelling
  • Xindi Wang, Robert Mercer, Frank Rudzicz
  • TLDR: We propose KenMeSH, an end-to-end model that combines new text features and a dynamic knowledge-enhanced mask attention to index biomedical Subject Headings.
• A Taxonomy of Empathetic Questions in Social Dialogs
  • Ekaterina Svikhnushina, Iuliana Voinea, Anuradha Welivita, Pearl Pu
  • TLDR: We propose a new empathetic question taxonomy for conversational chatbots that captures communicative acts and their emotion-regulation intents.
• Enhanced Multi-Channel Graph Convolutional Network for Aspect Sentiment Triplet Extraction
  • Hao Chen, Zepeng Zhai, Fangxiang Feng, Ruifan Li, Xiaojie Wang
  • TLDR: We propose an Enhanced Multi-Channel Graph Convolutional Network model for aspect sentiment triplet extraction task, which outperforms state-of-the-art methods significantly.
• ProtoTEx: Explaining Model Decisions with Prototype Tensors
  • Anubrata Das, Chitrank Gupta, Venelin Kovatchev, Matthew Lease, Junyi Jessy Li
  • TLDR: We present ProtoTEx, a novel white-box NLP classification architecture based on prototype networks that faithfully explains model decisions based on the distances between the input text and the prototype tensors.
• Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
  • Shuyan Zhou, Li Zhang, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch, Graham Neubig
  • TLDR: We propose a method for constructing a hierarchical knowledge-base of procedures based on wikiHow, a website containing more than 110k instructional articles, each documenting the steps to carry out a complex procedure.
• Cross-Modal Discrete Representation Learning
  • Alexander Liu, SouYoung Jin, Cheng-I Lai, Andrew Rouditchenko, Aude Oliva, James Glass
  • TLDR: We present a self-supervised multi-modal fine-grained representation learning framework that captures finer levels of granularity across different modalities such as concepts or events represented by visual objects or spoken words.
• Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering
  • Jun Gao, Wei Wang, Changlong Yu, Huan Zhao, Wilfred Ng, Ruifeng Xu
  • TLDR: We present a novel event representation learning framework that learns event representations by making better use of co-occurrence information of events.
• Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations
  • Robert Wolfe, Aylin Caliskan
  • TLDR: We show that contrastive visual semantic pretraining mitigates the anisotropy found in contextualized English language representations formed by GPT-2 and CLIP, a zero-shot multimodal image classifier which adapts the GPT2 architecture to encode image captions.
• ConTinTin: Continual Learning from Task Instructions
  • Wenpeng Yin, Jia Li, Caiming Xiong
  • TLDR: We propose a new learning paradigm for NLP that learns new tasks from task instructions.
• Automated Crossword Solving
  • Eric Wallace, Nicholas Tomlin, Albert Xu, Kevin Yang, Eshaan Pathak, Matthew Ginsberg, Dan Klein
  • TLDR: We present the Berkeley Crossword Solver, a state-of-the-art approach for automatically solving crossword puzzles.
• Learned Incremental Representations for Parsing
  • Nikita Kitaev, Thomas Lu, Dan Klein
  • TLDR: We present an incremental syntactic representation that consists of assigning a single discrete label to each word in a sentence, where the label is predicted using strictly incremental processing of a prefix of the sentence, and the sequence of labels for a sentence fully determines a parse tree.
• Knowledge Enhanced Reflection Generation for Counseling Dialogues
  • Siqi Shen, Veronica Perez-Rosas, Charles Welch, Soujanya Poria, Rada Mihalcea
  • TLDR: We present a new method for combining commonsense and domain knowledge for counseling conversations using retrieval and generative methods for knowledge integration.
• Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines
  • Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi, Yejin Choi
  • TLDR: We propose Misinfo Reaction Frames, a pragmatic formalism for modeling how readers might react to a news headline.
• On Continual Model Refinement in Out-of-Distribution Data Streams
  • Bill Yuchen Lin, Sida Wang, Xi Lin, Robin Jia, Lin Xiao, Xiang Ren, Scott Yih
  • TLDR: We propose a new continual learning problem formulation for dynamic OOD data streams and show its promise and challenges.
• Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection
  • Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, Julian McAuley
  • TLDR: We propose a novel knowledge-injection technique for neural dialog models that improves the quality of dialog responses by extracting relevant knowledge from external sources and incorporating it into the dialog response.
• Generated Knowledge Prompting for Commonsense Reasoning
  • Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
  • TLDR: We develop generated knowledge prompting, a method for generating knowledge from a language model and providing the knowledge as additional input when answering a question.
• Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  • Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  • TLDR: We propose a novel method for REtrieving from traINing datA to generate output from labeled training instances.
• Life after BERT: What do Other Muppets Understand about Language?
  • Vladislav Lialin, Kevin Zhao, Namrata Shivagunde, Anna Rumshisky
  • TLDR: We present a new approach to zero-shot transformer analysis that is not based on existing pre-training objectives and show that the results are not predictive of a model’s linguistic capabilities.
• Tailor: Generating and Perturbing Text with Semantic Controls
  • Alexis Ross, Tongshuang Wu, Hao Peng, Matthew Peters, Matt Gardner
  • TLDR: We present Tailor, a semantically-controlled text generation system that produces textual outputs conditioned on control codes derived from semantic representations.
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
  • Stephanie Lin, Jacob Hilton, Owain Evans
  • TLDR: We propose a benchmark to measure whether a language model is truthful in generating answers to questions.
• Adaptive Testing and Debugging of NLP Models
  • Marco Tulio Ribeiro, Scott Lundberg
  • TLDR: We present AdaTest, a process which uses large scale language models (LMs) in partnership with human feedback to automatically write unit tests highlighting bugs in a target model.
• Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning
  • Vivek Gupta, Shuo Zhang, Alakananda Vempala, Yujie He, Temma Choji, Vivek Srikumar
  • TLDR: We propose a new task for trustworthy inference in structured tabular data, where a model needs to extract evidence to be used for reasoning, in addition to predicting the label.
• Interactive Word Completion for Plains Cree
  • William Lane, Atticus Harrigan, Antti Arppe
  • TLDR: We present a new approach to morph-based auto-completion based on a finite state morphological analyzer of Plains Cree (nêhiyawêwin) and show its portability to a much larger, more complete morphological transducer.
• LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing
  • Dora Jambor, Dzmitry Bahdanau
  • TLDR: We propose LAGr, a general framework for semantic parsers that produces meaning representations for natural language sentences by independently predicting node and edge labels for a complete multi-layer input-aligned graph.
• ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
  • Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, Ece Kamar
  • TLDR: We present a large-scale and machine-generated dataset of toxic and benign statements about 13 minority groups.
• Direct Speech-to-Speech Translation With Discrete Units
  • Ann Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu, Sravya Popuri, Xutai Ma, Adam Polyak, Yossi Adi, Qing He, Yun Tang, Juan Pino, Wei-Ning Hsu
  • TLDR: We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation.
• Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
  • Meng Cao, Yue Dong, Jackie Cheung
  • TLDR: We propose a novel detection approach that separates factual from non-factual hallucinations of entities.
• EntSUM: A Data Set for Entity-Centric Extractive Summarization
  • Mounica Maddela, Mayank Kulkarni, Daniel Preotiuc-Pietro
  • TLDR: We introduce a human-annotated data set EntSUM for controllable summarization with a focus on named entities as the aspects to control.
• Sentence-level Privacy for Document Embeddings
  • Casey Meehan, Khalil Mrini, Kamalika Chaudhuri
  • TLDR: We propose SentDP, pure local differential privacy at the sentence level for a single user document.
• Dataset Geography: Mapping Language Data to Language Users
  • Fahim Faisal, Yinkai Wang, Antonios Anastasopoulos
  • TLDR: We study the geographical representativeness of NLP datasets, aiming to quantify if and by how much do NLP dataset match the expected needs of the language speakers.
• ILDAE: Instance-Level Difficulty Analysis of Evaluation Data
  • Neeraj Varshney, Swaroop Mishra, Chitta Baral
  • TLDR: We present a novel method for evaluating evaluation datasets using instance difficulty and show its applications in NLP.
• Image Retrieval from Contextual Descriptions
  • Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy
  • TLDR: We present Image Retrieval from Contextual Descriptions, a multimodal challenge that measures how well current vision-and-language models can integrate visual and temporal context into their representations.
• Multilingual Molecular Representation Learning via Contrastive Pre-training
  • Zhihui Guo, Pramod Sharma, Andy Martinez, Liang Du, Robin Abraham
  • TLDR: We propose a multilingual molecular embedding generation approach called MM-Deacon (multilingual molecular domain embedding analysis via contrastive learning).
• [Investigating Failures of Automatic Translation

in the Case of Unambiguous Gender](https://aclanthology.org/2022.acl-long.243)

• Adithya Renduchintala, Adina Williams

• TLDR: We present a new evaluation scheme and dataset for neural machine translation that measures the ability of NMT models to translate gender morphology correctly in unambiguous contexts across syntactically diverse sentences.

• Cross-Task Generalization via Natural Language Crowdsourcing Instructions
  • Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh Hajishirzi
  • TLDR: We present a meta-dataset of 61 distinct tasks, their human-authored instructions, and 193k task instances (input-output pairs) and measure generalization to unseen tasks.
• Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost
  • Lihu Chen, Gael Varoquaux, Fabian Suchanek
  • TLDR: We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT) and makes it robust to OOV with few additional parameters.
• NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks
  • Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Sachdeva, Peter Clark, Chitta Baral, Ashwin Kalyan
  • TLDR: We propose NumGLUE, a multi-task benchmark that evaluates the performance of AI systems on eight different tasks, that at their core require simple arithmetic understanding.
• Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models
  • Ryan Steed, Swetasudha Panda, Ari Kobren, Michael Wick
  • TLDR: We investigate the role of pre-trained models in machine learning and find that they can be harmful.
• Improving Multi-label Malevolence Detection in Dialogues through Multi-faceted Label Correlation Enhancement
  • Yangjun Zhang, Pengjie Ren, Wentao Deng, Zhumin Chen, Maarten Rijke
  • TLDR: We propose a multi-label dialogue malevolence detection task and a multi label dialogue malevolent dialogue response detection model.
• How Do We Answer Complex Questions: Discourse Structure of Long-form Answers
  • Fangyuan Xu, Junyi Jessy Li, Eunsol Choi
  • TLDR: We develop an ontology of six sentence-level functional roles for long-form answers, and annotate 3.9k sentences in 640 answer paragraphs.
• Understanding Iterative Revision from Human-Written Text
  • Wanyu Du, Vipul Raheja, Dhruv Kumar, Zae Myung Kim, Melissa Lopez, Dongyeop Kang
  • TLDR: We propose a new framework for iterative text revision that generalizes to a variety of domains, edit intentions, revision depths, and granularities.
• Making Transformers Solve Compositional Tasks
  • Santiago Ontanon, Joshua Ainslie, Zachary Fisher, Vaclav Cvicek
  • TLDR: We show that Transformer models can generalize compositionally, a key type of generalization in many NLP tasks such as semantic parsing and string editing.
• Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation
  • Verna Dankers, Christopher Lucas, Ivan Titov
  • TLDR: We investigate the role of idioms in neural machine translation and show that they are more likely to be translated as compositional expressions than as literal expressions.
• ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers
  • Haitian Sun, William Cohen, Ruslan Salakhutdinov
  • TLDR: We present a dataset that contains complex questions with conditional answers, i.e. the answers are only applicable when certain conditions apply.
• Prompt-free and Efficient Few-shot Learning with Language Models
  • Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Lambert Mathias, Marzieh Saeidi, Veselin Stoyanov, Majid Yazdani
  • TLDR: We propose Perfect, a simple and efficient method for few-shot fine-tuning of pretrained masked language models without relying on any such handcrafting, which is highly effective given as few as 32 data points.
• Continual Sequence Generation with Adaptive Compositional Modules
  • Yanzhe Zhang, Xuezhi Wang, Diyi Yang
  • TLDR: We propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks.
• An Investigation of the (In)effectiveness of Counterfactually Augmented Data
  • Nitish Joshi, He He
  • TLDR: We show that the lack of perturbation diversity in counterfactually-augmented data can prevent the model from learning robust features that are invariant under distribution shift.
• Inducing Positive Perspectives with Text Reframing
  • Caleb Ziems, Minzhi Li, Anthony Zhang, Diyi Yang
  • TLDR: We present a novel text style transfer task that uses positive reframing to generate a more positive perspective for the author without contradicting the original meaning.
• VALUE: Understanding Dialect Disparity in NLU
  • Caleb Ziems, Jiaao Chen, Camille Harris, Jessica Anderson, Diyi Yang
  • TLDR: We present a new benchmark for African American Vernacular English that can help to improve the quality of existing NLU systems.
• From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer
  • John Pavlopoulos, Leo Laugier, Alexandros Xenos, Jeffrey Sorensen, Ion Androutsopoulos
  • TLDR: We study the task of toxic spans detection, which concerns the detection of the spans that make a text toxic, when detecting such spans is possible.
• FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
  • Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas Pfister
  • TLDR: We propose FormNet, a structure-aware sequence model to mitigate the suboptimal serialization of forms.
• The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
  • Caleb Ziems, Jane Yu, Yi-Chia Wang, Alon Halevy, Diyi Yang
  • TLDR: We present a new moral integrity corpus that captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs).
• Token Dropping for Efficient BERT Pretraining
  • Le Hou, Richard Yuanzhe Pang, Tianyi Zhou, Yuexin Wu, Xinying Song, Xiaodan Song, Denny Zhou
  • TLDR: We develop a simple but effective method to accelerate the pretraining of transformer models, such as BERT, without degrading its performance on downstream tasks.
• DialFact: A Benchmark for Fact-Checking in Dialogue
  • Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, Caiming Xiong
  • TLDR: We present a novel fact-checking task for dialogue that uses Wikipedia snippets as evidence and a simple yet data-efficient solution to effectively improve fact-check in dialogue.
• The Trade-offs of Domain Adaptation for Neural Language Models
  • David Grangier, Dan Iter
  • TLDR: We present a new approach to language model adaptation based on data selection and theory.
• Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Go
  • Ife Adebara, Muhammad Abdul-Mageed
  • TLDR: We present a typological framework for language diversity and discuss the major linguistic and sociopolitical challenges facing development of NLP technologies for African languages.
• Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction
  • Maksym Tarnavskyi, Artem Chernodub, Kostiantyn Omelianchuk
  • TLDR: Ensembling Transformer-based encoders in Large configurations with majority votes.
• Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching
  • Alissa Ostapenko, Shuly Wintner, Melinda Fricke, Yulia Tsvetkov
  • TLDR: We show that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases.
• Detecting Unassimilated Borrowings in Spanish: An Annotated Corpus and Approaches to Modeling
  • Elena Álvarez-Mellado, Constantine Lignos
  • TLDR: We present a new corpus of Spanish newswire rich in unassimilated lexical borrowings and evaluate how several sequence labeling models (CRF, BiLSTM-CRF) perform.
• Is Attention Explanation? An Introduction to the Debate
  • Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, Patrick Watrin
  • TLDR: We provide a clear overview of the insights on the debate on the explanatory power of attention in neural networks and propose a holistic vision for future work.
• There Are a Thousand Hamlets in a Thousand People’s Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memory
  • Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji-Rong Wen, Rui Yan
  • TLDR: We introduce personal memory into knowledge selection in KGC to address the personalization issue.
• Neural Pipeline for Zero-Shot Data-to-Text Generation
  • Zdeněk Kasner, Ondrej Dusek
  • TLDR: We propose to generate text from RDF triples by training PLMs on general-domain text-based operations.
• Not always about you: Prioritizing community needs when developing endangered language technology
  • Zoey Liu, Crystal Richardson, Richard Hatcher, Emily Prud’hommeaux
  • TLDR: We present a new perspective on the challenges of language technology and language documentation and revitalization in low-resource languages and propose a new approach to address them.
• Automatic Identification and Classification of Bragging in Social Media
  • Mali Jin, Daniel Preotiuc-Pietro, A. Seza Doğruöz, Nikolaos Aletras
  • TLDR: We present the first large scale study of bragging in computational linguistics, building on previous research in linguistics and pragmatics.
• Automatic Error Analysis for Document-level Information Extraction
  • Aliva Das, Xinya Du, Barry Wang, Kejian Shi, Jiayuan Gu, Thomas Porter, Claire Cardie
  • TLDR: We propose a transformation-based framework for automating error analysis in document-level document-layer information extraction tasks.
• Learning Functional Distributional Semantics with Visual Data
  • Yinhong Liu, Guy Emerson
  • TLDR: We propose a method to train a Functional Distributional Semantics model with grounded visual data.
• ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding
  • Sayan Ghosh, Shashank Srivastava
  • TLDR: We present a high-quality crowdsourced dataset of narratives for employing proverbs in context as a benchmark for abstract language understanding.
• Chart-to-Text: A Large-Scale Benchmark for Chart Summarization
  • Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, Shafiq Joty
  • TLDR: We present a large-scale dataset and a large dataset of 44,096 charts for generating natural language summaries from charts.
• Characterizing Idioms: Conventionality and Contingency
  • Michaela Socolof, Jackie Cheung, Michael Wagner, Timothy O’Donnell
  • TLDR: We show that idioms are not correlated with other words in the idiom, and that the dimensions themselves are not related.
• Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP
  • Lukas Galke, Ansgar Scherp
  • TLDR: We show that a graph-based wide multi-layer perceptron using a Bag-of-Words outperforms the recent graph- based models TextGCN and HeteGCN in an inductive text classification setting and is comparable with HyperGAT.
• Generative Pretraining for Paraphrase Evaluation
  • Jack Weston, Raphael Lenain, Udeepa Meepegama, Emil Fristed
  • TLDR: We introduce ParaBLEU, a paraphrase representation learning model and evaluation metric for text generation.
• Incorporating Stock Market Signals for Twitter Stance Detection
  • Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, Nigel Collier
  • TLDR: We propose a novel multi-task neural architecture for stance detection in the financial domain which combines textual input with high-frequency intra-day time series from stock market prices.
• Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation
  • Yong Cheng, Ankur Bapna, Orhan Firat, Yuan Cao, Pidong Wang, Wolfgang Macherey
  • TLDR: We propose a novel crossover encoder-decoder for multilingual neural machine translation models that improves quality on translation tasks by interpolating instances from different language pairs into joint crossover examples.
• Word Segmentation as Unsupervised Constituency Parsing
  • Raquel G. Alhama
  • TLDR: We propose a novel approach to word identification in artificial languages that is isomorphic to unsupervised constituency parsing.
• SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
  • Emily Dinan, Gavin Abercrombie, A. Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, Verena Rieser
  • TLDR: We present a taxonomy of three observed phenomena and empirically assess the extent to which current tools can measure these effects and current systems display them.
• Zero-Shot Cross-lingual Semantic Parsing
  • Tom Sherborne, Mirella Lapata
  • TLDR: We present a novel multi-task encoder-decoder model for cross-lingual semantic parsing that can transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora in each new language.
• The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study
  • Verna Dankers, Elia Bruni, Dieuwke Hupkes
  • TLDR: We present a new compositionality test for neural machine translation and show that models are not always compositional, but sometimes more compositional than expected.
• Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
  • Biao Zhang, Ankur Bapna, Melvin Johnson, Ali Dabirmoghaddam, Naveen Arivazhagan, Orhan Firat
  • TLDR: We study the transfer of document-level neural machine translation from teacher languages with document level data to student languages with no documents but sentence level data, and for the first time treat document- level translation as a transfer learning problem.
• Cross-Lingual Phrase Retrieval
  • Heqi Zheng, Xiao Zhang, Zewen Chi, Heyan Huang, Yan Tan, Tian Lan, Wei Wei, Xian-Ling Mao
  • TLDR: We propose a novel cross-lingual phrase retriever that extracts phrase representations from unlabeled example sentences and perform zero-shot transferability.
• Improving Compositional Generalization with Self-Training for Data-to-Text Generation
  • Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, Emma Strubell
  • TLDR: We systematically study the compositional generalization of the state-of-the-art T5 models in few-shot data-to-text generation tasks.
• MMCoQA: Conversational Question Answering over Text, Tables, and Images
  • Yongqi Li, Wenjie Li, Liqiang Nie
  • TLDR: We propose a novel research task for multimodal conversational question answering, which aims to answer users’ questions with multimodality of different modalities via multi-turn conversations.
• Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis
  • Wenxuan Shi, Fei Li, Jingye Li, Hao Fei, Donghong Ji
  • TLDR: We propose a novel labeling strategy for structured sentiment analysis, which is able to capture various token relations and solve the imbalance problem in the span prediction and span relation prediction.
• PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks
  • Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng, Daxin Jiang
  • TLDR: We propose Prompt-based Data Augmentation for low-resource Natural Language Understanding tasks which improves the performance of NLU models.
• Disentangled Sequence to Sequence Learning for Compositional Generalization
  • Hao Zheng, Mirella Lapata
  • TLDR: We propose an extension to sequence-to-sequence models which encourage disentanglement by adaptively re-encoding (at each time step) the source input.
• RST Discourse Parsing with Second-Stage EDU-Level Pre-training
  • Nan Yu, Meishan Zhang, Guohong Fu, Min Zhang
  • TLDR: We propose a second-stage EDU-level pre-training approach in this work, which presents two novel tasks to learn effective EDU representations continually based on well pre-trained language models.
• SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models
  • Liang Wang, Wei Zhao, Zhuoyu Wei, Jingming Liu
  • TLDR: We propose a novel graph embedding-based knowledge graph completion algorithm that can outperform existing graph embeddings on several benchmark datasets.
• Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?
  • Oliver Eberle, Stephanie Brandl, Jonas Pilot, Anders Søgaard
  • TLDR: We show that large-scale pre-trained language models are as predictive of human eye fixation patterns during task-reading as classical cognitive models of human attention.
• LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
  • Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, Nikolaos Aletras
  • TLDR: We present a benchmark for evaluating and evaluating the generalization of legal NLU models and provide a new evaluation and analysis of several generative models.
• DiBiMT: A Novel Benchmark for Measuring Word Sense Disambiguation Biases in Machine Translation
  • Niccolò Campolungo, Federico Martelli, Francesco Saina, Roberto Navigli
  • TLDR: We present DiBiMT, a new evaluation benchmark for Machine Translation which enables an extensive study of semantic biases in Machine Translation of nominal and verbal words in five different language combinations, namely, English and one or other of the following languages: Chinese, German, Italian, Russian and Spanish.
• Improving Word Translation via Two-Stage Contrastive Learning
  • Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić
  • TLDR: We propose a robust and effective two-stage contrastive learning framework for the bilingual lexicon induction task.
• Scheduled Multi-task Learning for Neural Chat Translation
  • Yunlong Liang, Fandong Meng, Jinan Xu, Yufeng Chen, Jie Zhou
  • TLDR: We propose a scheduled multi-task learning framework for neural chat translation and show its effectiveness and superiority over existing methods.
• FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing
  • Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Schwemer, Anders Søgaard
  • TLDR: We present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks.
• Towards Abstractive Grounded Summarization of Podcast Transcripts
  • Kaiqiang Song, Chen Li, Xiaoyang Wang, Dong Yu, Fei Liu
  • TLDR: We propose a novel abstractive summarization method for podcast summarization that learns to produce an abstractive summary while grounding summary segments in specific regions of the transcript to allow for full inspection of summary details.
• FiNER: Financial Numeric Entity Recognition for XBRL Tagging
  • Lefteris Loukas, Manos Fergadiotis, Ilias Chalkidis, Eirini Spyropoulou, Prodromos Malakasiotis, Ion Androutsopoulos, Georgios Paliouras
  • TLDR: We propose two simple and effective solutions that replace numeric expressions with pseudo-tokens reflecting original token shapes and numeric magnitudes.
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation
  • Mingzhe Li, XieXiong Lin, Xiuying Chen, Jinxiong Chang, Qishen Zhang, Feng Wang, Taifeng Wang, Zhongyi Liu, Wei Chu, Dongyan Zhao, Rui Yan
  • TLDR: We propose a hierarchical contrastive learning mechanism for novel text generation tasks by constructing intra-contrasts between instance-level and keyword-level, which can unify hybrid granularities semantic meaning in the input text.
• EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers
  • Bugeun Kim, Kyung Seo Ki, Sangkyu Rhim, Gahgene Gweon
  • TLDR: We propose a neural model EPT-X (Expression-Pointer Transformer with Explanations), which utilizes natural language explanations to solve an algebraic word problem.
• Identifying the Human Values behind Arguments
  • Johannes Kiesel, Milad Alshomary, Nicolas Handke, Xiaoni Cai, Henning Wachsmuth, Benno Stein
  • TLDR: We propose a multi-level taxonomy of human values that is in line with psychological research.
• BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
  • Kiril Gashteovski, Mingying Yu, Bhushan Kotnis, Carolin Lawrence, Mathias Niepert, Goran Glavaš
  • TLDR: BenchIE: a benchmark and evaluation framework for comprehensive evaluation of OIE systems for English, Chinese, and German.
• Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
  • Xichen Pan, Peiyu Chen, Yichen Gong, Helong Zhou, Xinbing Wang, Zhouhan Lin
  • TLDR: We propose a multimodal multimodality speech recognition framework which learns to recognize parallel audio-visual data into characters through a combination of CTC and seq2seq decoding.
• SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization
  • Mathieu Ravaut, Shafiq Joty, Nancy Chen
  • TLDR: We present a new method for generating a new summary by re-ranking a set of summary candidates.
• Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals
  • Te-Lin Wu, Alex Spangher, Pegah Alipoormolabashi, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
  • TLDR: We benchmark models’ capability of reasoning over and sequencing unordered multimodal instructions by curating datasets from online instructional manuals and collecting comprehensive human annotations.
• Zoom Out and Observe: News Environment Perception for Fake News Detection
  • Qiang Sheng, Juan Cao, Xueyao Zhang, Rundong Li, Danding Wang, Yongchun Zhu
  • TLDR: We propose a novel framework for detecting fake news by observing the external news environment and propose a new module to perceive useful signals.
• Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models
  • Lorenzo Lupo, Marco Dinarelli, Laurent Besacier
  • TLDR: We propose to pre-train the contextual parameters over split sentence pairs, which makes efficient use of the available data for two reasons.
• Saliency as Evidence: Event Detection with Trigger Saliency Attribution
  • Jian Liu, Yufeng Chen, Jinan Xu
  • TLDR: We present a new concept for event detection that can identify the underlying patterns of events and show that there are many distinct types of events.
• SRL4E – Semantic Role Labeling for Emotions: A Unified Evaluation Framework
  • Cesare Campagnano, Simone Conia, Roberto Navigli
  • TLDR: We present a unified evaluation framework for semantic role labeling for emotions and semantic roles in sentiment analysis.
• Context Matters: A Pragmatic Study of PLMs’ Negation Understanding
  • Reto Gubelmann, Siegfried Handschuh
  • TLDR: We present a new study of transformer-based PLMs on negation understanding and show that transformer-derived PLMs are much better than transformer-only PLMs.
• Probing for Predicate Argument Structures in Pretrained Language Models
  • Simone Conia, Roberto Navigli
  • TLDR: We investigate the inner workings of modern pretrained language models and show that they encode semantic structures directly into the contextualized representation of a predicate, and provide insights into the correlation between predicate senses and their structures, the degree of transferability between nominal and verbal structures, and how such structures are encoded across languages.
• Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction
  • Kuan-Hao Huang, I-Hung Hsu, Prem Natarajan, Kai-Wei Chang, Nanyun Peng
  • TLDR: We present a language generation task for zero-shot cross-lingual event argument extraction and show that the proposed model outperforms the current state-of-the-art models on zero-Shot cross-shot event argument transfer.
• Identifying Moments of Change from Longitudinal User Text
  • Adam Tsakalidis, Federico Nanni, Anthony Hills, Jenny Chim, Jiayu Song, Maria Liakata
  • TLDR: We propose a new task for detecting changes in individuals’ behaviour and mood, as observed via content shared on online platforms, and show that the best performance is obtained through context aware sequential modelling.
• Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System
  • Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, Yi Zhang
  • TLDR: We present a unified plug-and-play model for task-oriented dialogue that achieves new state of the art results on all three benchmark TOD tasks.
• Graph Enhanced Contrastive Learning for Radiology Findings Summarization
  • Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, Tsung-Hui Chang
  • TLDR: We propose a unified framework for exploiting both extra knowledge and the original findings in an integrated way so that the critical information (i.e., key words and their relations) can be extracted in an appropriate way to facilitate impression generation.
• Semi-Supervised Formality Style Transfer with Consistency Training
  • Ao Liu, An Wang, Naoaki Okazaki
  • TLDR: We propose a simple yet effective semi-supervised framework to better utilize source-side unlabeled sentences based on consistency training.
• Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure
  • Yuan Chai, Yaobo Liang, Nan Duan
  • TLDR: We study the contribution of language properties in cross-linguistic transfer and show that composition is more crucial to the success of cross-language transfer.
• Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting
  • Ying Su, Hongming Zhang, Yangqiu Song, Tong Zhang
  • TLDR: We propose a Z-reweighting method on the word level to adjust the training on the imbalanced dataset.
• Nibbling at the Hard Core of Word Sense Disambiguation
  • Marco Maru, Simone Conia, Michele Bevilacqua, Roberto Navigli
  • TLDR: We provide evidence showing why the F1 score metric for Word Sense Disambiguation is not as accurate as it is given credit for and present a collection of new test sets and model predictions for the task.
• Large Scale Substitution-based Word Sense Induction
  • Matan Eyal, Shoval Sadde, Hillel Taub-Tabib, Yoav Goldberg
  • TLDR: We present a word-sense induction method based on pre-trained masked language models, which can cheaply scale to large vocabularies and large corpora.
• Can Synthetic Translations Improve Bitext Quality?
  • Eleftheria Briakou, Marine Carpuat
  • TLDR: Synthetic translations can improve bitext quality without any additional bilingual supervision when they replace the originals based on a semantic equivalence classifier that helps mitigate NMT noise.
• Unsupervised Dependency Graph Network
  • Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
  • TLDR: We propose a new competitive mechanism for training pretrained self-attention heads that can induce dependency structures from raw corpora and the masked language modeling task.
• WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types
  • Xuwu Wang, Junfeng Tian, Min Gui, Zhixu Li, Rui Wang, Ming Yan, Lihan Chen, Yanghua Xiao
  • TLDR: We present WikiDiverse, a high-quality human-annotated MEL dataset with diversified contextual topics and entity types from Wikinews, which uses Wikipedia as the corresponding knowledge base.
• Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
  • Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi, Yixuan Su, Charlotte Collins, Nigel Collier
  • TLDR: We propose Contrastive-Probe, a novel self-supervised contrastive probing approach for biomedical knowledge probing, which improves the underlying PLMs without using any probing data.
• Fine- and Coarse-Granularity Hybrid Self-Attention for Efficient BERT
  • Jing Zhao, Yifan Wang, Junwei Bao, Youzheng Wu, Xiaodong He
  • TLDR: We propose a new method for improving the performance of Transformer-based pre-trained models by reducing the computational cost of the input sequence length in self-attention.
• Compression of Generative Pre-trained Language Models via Quantization
  • Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong
  • TLDR: We compress generative PLMs by quantization, and propose a module-wise dynamic scaling to make quantizers adaptive to different modules.
• Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
  • Xiwen Liang, Fengda Zhu, Li Lingling, Hang Xu, Xiaodan Liang
  • TLDR: We propose Prompt-based Environmental Self-exploration for vision-language navigation, which can self-explore the environments by sampling trajectories and automatically generates structured instructions via a large-scale cross-modal pretrained model.
• DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation
  • Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
  • TLDR: We propose a new dialog pre-training framework called DialogVED, which introduces continuous latent variables into the enhanced encoder-decoder pre-trained framework to increase the relevance and diversity of responses.
• Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  • Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang, Nan Duan
  • TLDR: We propose a Contextual Fine-to-Coarse response selection model for retrieval-based dialogue systems.
• Textomics: A Dataset for Genomics Data Summary Generation
  • Mu-Chun Wang, Zixuan Liu, Sheng Wang
  • TLDR: We present a novel dataset of genomics data description and their summaries.
• A Contrastive Framework for Learning Sentence Representations from Pairwise and Triple-wise Perspective in Angular Space
  • Yuhao Zhang, Hongji Zhu, Yongliang Wang, Nan Xu, Xiaobo Li, Binqiang Zhao
  • TLDR: We propose ArcCSE, a novel contrastive learning method for sentence representations, which improves the pairwise discriminative power and model the entailment relation of triplet sentences.
• Packed Levitated Marker for Entity and Relation Extraction
  • Deming Ye, Yankai Lin, Peng Li, Maosong Sun
  • TLDR: We propose a novel span representation approach, named Packed Levitated Markers (PL-Marker), to consider the interrelation between the spans (pairs) by strategically packing the markers in the encoder.
• An Interpretable Neuro-Symbolic Reasoning Framework for Task-Oriented Dialogue Generation
  • Shiquan Yang, Rui Zhang, Sarah Erfani, Jey Han Lau
  • TLDR: We introduce neuro-symbolic to perform explicit reasoning that justifies model decisions by reasoning chains.
• Impact of Evaluation Methodologies on Code Summarization
  • Pengyu Nie, Jiyang Zhang, Junyi Jessy Li, Ray Mooney, Milos Gligoric
  • TLDR: We introduce the time-segmented evaluation methodology for code summarization tasks and show that it leads to conflicting evaluation results.
• KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  • Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  • TLDR: We propose a knowledge-enhanced version of the Fusion-in-Decoder framework for ODQA, which can achieve comparable or better performance in answer prediction than FiD, with less than 40% of the computation cost.
• Which side are you on? Insider-Outsider classification in conspiracy-theoretic social media
  • Pavan Holur, Tianyi Wang, Shadi Shahsavari, Timothy Tangherlini, Vwani Roychowdhury
  • TLDR: Social media is a breeding ground for threat narratives and related conspiracy theories. In these, an
• Learning From Failure: Data Capture in an Australian Aboriginal Community
  • Eric Le Ferrand, Steven Bird, Laurent Besacier
  • TLDR: We present a novel approach to language data capture in an Aboriginal community which generates data while avoiding the transcription bottleneck.
• Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension
  • Wenya Wang, Sinno Pan
  • TLDR: We propose a novel inductive logic reasoning method for multi-hop reading comprehension that learns to infer the target relation of query-related information.
• CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues
  • Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, Soujanya Poria
  • TLDR: We present a dataset of dialogue-centric commonsense inference inferences from 5,672 dialogues.
• A Comparative Study of Faithfulness Metrics for Model Interpretability Methods
  • Chun Sik Chan, Huanqi Kong, Liang Guanqing
  • TLDR: We present a comprehensive and comparative study of the widely adopted faithfulness metrics and show that different metrics show conflicting preferences when comparing different interpretations.
• SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
  • Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou’, Daniel Cer
  • TLDR: We propose a novel prompt-based transfer learning approach for NLP tasks that improves the performance of Prompt Tuning and improves transferability across many tasks.
• Pass off Fish Eyes for Pearls: Attacking Model Selection of Pre-trained Models
  • Biru Zhu, Yujia Qin, Fanchao Qi, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu
  • TLDR: We show that current feature-based model selection methods are vulnerable to the vulnerability of the backdoor attack and evaluation data selection.
• Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-centric Summarization
  • Zhenjie Zhao, Yufang Hou, Dakuo Wang, Mo Yu, Chengzhong Liu, Xiaojuan Ma
  • TLDR: We propose a novel question generation method that first learns the question type distribution of an input story paragraph, and then summarizes salient events which can be used to generate high-cognitive-demand questions.
• HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations
  • Jia-Chen Gu, Chao-Hong Tan, Chongyang Tao, Zhen-Hua Ling, Huang Hu, Xiubo Geng, Daxin Jiang
  • TLDR: We present HeterMPC, a graph-based neural network for response generation in multi-party conversations which models the semantics of utterances and interlocutors simultaneously with two types of nodes in a graph.
• The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature
  • Yulia Otmakhova, Karin Verspoor, Timothy Baldwin, Jey Han Lau
  • TLDR: We examine the quality of biomedical summaries generated by two current models in order to understand the deficiencies of existing evaluation approaches in the context of the challenges that arise in the MDS task.
• A Multi-Document Coverage Reward for RELAXed Multi-Document Summarization
  • Jacob Parnell, Inigo Jauregi Unanue, Massimo Piccardi
  • TLDR: We propose a novel reward reward for multi-document summarization that balances ROUGE and METEOR and show significant improvements over the baseline and the input documents.
• KNN-Contrastive Learning for Out-of-Domain Intent Classification
  • Yunhua Zhou, Peiju Liu, Xipeng Qiu
  • TLDR: We propose a novel method for OOD intent classification that uses k-nearest neighbors of In-domain intents to learn discriminative semantic features that are more conducive to OOD detection.
• A Neural Network Architecture for Program Understanding Inspired by Human Behaviors
  • Renyu Zhu, Lei Yuan, Xiang Li, Ming Gao, Wenyuan Cai
  • TLDR: We present a graph neural network model that is based on partitioning-based graph neural networks and propose a novel method for code comprehension and code clone detection.
• FaVIQ: FAct Verification from Information-seeking Questions
  • Jungsoo Park, Sewon Min, Jaewoo Kang, Luke Zettlemoyer, Hannaneh Hajishirzi
  • TLDR: We construct a large-scale challenging fact verification dataset with realistic real-world claims.
• Simulating Bandit Learning from User Feedback for Extractive Question Answering
  • Ge Gao, Eunsol Choi, Yoav Artzi
  • TLDR: We study learning from user feedback for extractive question answering by simulating feedback using supervised data.
• Beyond Goldfish Memory: Long-Term Open-Domain Conversation
  • Jing Xu, Arthur Szlam, Jason Weston
  • TLDR: We show how existing models trained on existing datasets perform poorly in the long-term conversation setting and show long-context models that can perform much better.
• ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
  • Sanjay Subramanian, William Merrill, Trevor Darrell, Matt Gardner, Sameer Singh, Anna Rohrbach
  • TLDR: We present ReCLIP, a simple but strong ReC model for visual domain training.
• Dynamic Prefix-Tuning for Generative Template-based Event Extraction
  • Xiao Liu, Heyan Huang, Ge Shi, Bo Wang
  • TLDR: Generative template-based event extraction with dynamic prefixes.
• E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models
  • Mohammad Akbari, Amin Banitalebi-Dehkordi, Yong Zhang
  • TLDR: We propose a dynamic inference approach for language models that can be applied to black-box pre-trained models without re-training.
• PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization
  • Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan
  • TLDR: We introduce PRIMERA, a pre-trained model for multi-document representation with a focus on summarization that reduces the need for dataset-specific architectures and large amounts of fine-tuning labeled data.
• Dynamic Global Memory for Document-level Argument Extraction
  • Xinya Du, Sha Li, Heng Ji
  • TLDR: We propose a novel document memory store-based framework for document-level event argument extraction and use it to implicitly and explicitly help with decoding of arguments for later events.
• Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns
  • Daniel Wiechmann, Yu Qiao, Elma Kerz, Justus Mattern
  • TLDR: We show that both the features included and the architecture of the transformer-based language models play a role in predicting multiple eye-tracking measures during naturalistic reading.
• Alternative Input Signals Ease Transfer in Multilingual Machine Translation
  • Simeng Sun, Angela Fan, James Cross, Vishrav Chaudhary, Chau Tran, Philipp Koehn, Francisco Guzmán
  • TLDR: We propose a novel multi-source self-ensemble method for efficient multilingual machine translation that improves the transfer of supervision signals between languages with different writing systems.
• Phone-ing it in: Towards Flexible Multi-Modal Language Model Training by Phonetic Representations of Data
  • Colin Leong, Daniel Whitenack
  • TLDR: We propose a multi-modal approach to train language models using whatever text and/or audio data might be available in a language.
• Noisy Channel Language Model Prompting for Few-Shot Text Classification
  • Sewon Min, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer
  • TLDR: We introduce a noisy channel approach for language model prompting in few-shot text classification, which significantly outperforms the direct models.
• Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages
  • C. Downey, Shannon Drizin, Levon Haroutunian, Shivin Thukral
  • TLDR: We show that unsupervised sequence-segmentation performance can be transferred to extremely low-resource languages by pre-training a Masked Segmental Language Model (Downey et al., 2021) multilingually.
• KinyaBERT: a Morphology-aware Kinyarwanda Language Model
  • Antoine Nzeyimana, Andre Niyongabo Rubungo
  • TLDR: We propose a simple yet effective two-tier BERT architecture that leverages a morphological analyzer and explicitly represents morphological compositionality.
• On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency
  • Seo Yeon Park, Cornelia Caragea
  • TLDR: We propose a novel mixup strategy for pre-trained language models that improves model calibration further.
• IMPLI: Investigating NLI Models’ Performance on Figurative Language
  • Kevin Stowe, Prasetya Utama, Iryna Gurevych
  • TLDR: We present a new dataset for evaluating and evaluating NLI models for understanding figurative language and show that they are not able to detect entailment relationship between figurative phrases with their literal counterparts.
• QAConv: Question Answering on Informative Conversations
  • Chien-Sheng Wu, Andrea Madotto, Wenhao Liu, Pascale Fung, Caiming Xiong
  • TLDR: We present a new question answering dataset for informative conversations that uses conversations as a knowledge source.
• Prix-LM: Pretraining for Multilingual Knowledge Base Construction
  • Wenxuan Zhou, Fangyu Liu, Ivan Vulić, Nigel Collier, Muhao Chen
  • TLDR: We propose a unified representation model for multilingual knowledge bases and tune a multilingual language encoder XLM-R via a causal language modeling objective.
• Semantic Composition with PSHRG for Derivation Tree Reconstruction from Graph-Based Meaning Representations
  • Chun Hei Lo, Wai Lam, Hong Cheng
  • TLDR: We introduce a data-driven approach to generating derivation trees from meaning representation graphs with probabilistic synchronous hyperedge replacement grammar (PSHRG).
• HOLM: Hallucinating Objects with Language Models for Referring Expression Recognition in Partially-Observed Scenes
  • Volkan Cirik, Louis-Philippe Morency, Taylor Berg-Kirkpatrick
  • TLDR: We propose a novel approach to explain the relationship between language and the environment by infering hallucinations of unseen objects in the unobserved part of the environment.
• Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models
  • Kabir Ahuja, Shanu Kumar, Sandipan Dandapat, Monojit Choudhury
  • TLDR: We propose a multi-task learning approach for predicting zero-shot transfer across languages, and show that it is effective for tasks where we have test data in very few languages to measure the actual performance of the model.
• \infty-former: Infinite Memory Transformer
  • Pedro Henrique Martins, Zita Marinho, Andre Martins
  • TLDR: We propose a new efficient transformer for long-term memory.
• Systematic Inequalities in Language Technology Performance across the World’s Languages
  • Damian Blasi, Antonios Anastasopoulos, Graham Neubig
  • TLDR: We present a new approach to improving natural language processing systems.
• CaMEL: Case Marker Extraction without Labels
  • Leonie Weissweiler, Valentin Hofmann, Masoud Jalili Sabet, Hinrich Schuetze
  • TLDR: We introduce
• Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectors
  • Isar Nejadgholi, Kathleen Fraser, Svetlana Kiritchenko
  • TLDR: We show that general abusive language classifiers tend to be fairly reliable in detecting out-of-domain explicitly abusive utterances but fail to detect new types of more subtle, implicit abuse.
• Reports of personal experiences and stories in argumentation: datasets and analysis
  • Neele Falk, Gabriella Lapesa
  • TLDR: We present a new method for cross-domain classification of personal experiences and stories in argumentation, and show that it is possible to identify documents containing personal experiences.
• Non-neural Models Matter: a Re-evaluation of Neural Referring Expression Generation Systems
  • Fahime Same, Guanyi Chen, Kees Van Deemter
  • TLDR: We show that rule-based and classic machine learning approaches outperform neural models on a simple sentence generation task.
• Bridging the Generalization Gap in Text-to-SQL Parsing with Schema Expansion
  • Chen Zhao, Yu Su, Adam Pauls, Emmanouil Antonios Platanios
  • TLDR: We propose to address this problem by incorporating prior domain knowledge by preprocessing table schemas, and design a method that consists of two components: schema expansion and schema pruning.
• Predicate-Argument Based Bi-Encoder for Paraphrase Identification
  • Qiwei Peng, David Weir, Julie Weeds, Yekun Chai
  • TLDR: We propose a new bi-encoder for paraphrase identification that outperforms SBERT/SRoBERTa and outperforms SRoBERT.
• MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective
  • Xiao Wang, Shihan Dou, Limao Xiong, Yicheng Zou, Qi Zhang, Tao Gui, Liang Qiao, Zhanzhan Cheng, Xuanjing Huang
  • TLDR: We propose MINER, a novel NER learning framework, which improves representation and predicts OOV entities.
• Leveraging Wikipedia article evolution for promotional tone detection
  • Christine De Kock, Andreas Vlachos
  • TLDR: We propose WikiEvolve, a dataset for document-level promotional tone detection.
• From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology
  • Mark Dingemanse, Andreas Liesenfeld
  • TLDR: We show how interactional data from 63 languages (26 families) harbours insights about turn-taking, timing, sequential structure and social action, with implications for language technology, natural language understanding, and the design of conversational interfaces.
• Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning
  • Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, Xuanjing Huang
  • TLDR: We propose a novel adversarial robustness method which uses hyper-parameter-dependent flooding to improve adversarial adversarial attacks.
• RoMe: A Robust Metric for Evaluating Natural Language Generation
  • Md Rashad Al Hasan Rony, Liubov Kovriguina, Debanjan Chaudhuri, Ricardo Usbeck, Jens Lehmann
  • TLDR: We propose an automatic evaluation metric incorporating several core aspects of natural language understanding (language competence, syntactic and semantic variation) to evaluate system-generated sentences.
• Finding Structural Knowledge in Multimodal-BERT
  • Victor Milewski, Miryam de Lhoneux, Marie-Francine Moens
  • TLDR: We investigate the knowledge learned in the embeddings of multimodal-BERT models and their ability to encode the structure of language and visuals.
• Fully Hyperbolic Neural Networks
  • Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
  • TLDR: We propose a fully hyperbolic framework to build hyperbolics based on the Lorentz model by adapting the Lorendz transformations (including boost and rotation) to formalize essential operations of neural networks.
• Neural Machine Translation with Phrase-Level Universal Visual Representations
  • Qingkai Fang, Yang Feng
  • TLDR: We propose a phrase-level retrieval-based method for MMT to get visual information for the source input from existing sentence-image data sets so that MMT can break the limitation of paired sentence-Image input.
• M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database
  • Jinming Zhao, Tenggan Zhang, Jingwen Hu, Yuchen Liu, Qin Jin, Xinchao Wang, Haizhou Li
  • TLDR: We propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset for multimodal affective analysis in dialogues.
• Few-shot Named Entity Recognition with Self-describing Networks
  • Jiawei Chen, Qing Liu, Hongyu Lin, Xianpei Han, Le Sun
  • TLDR: We propose a self-describing mechanism for few-shot NER, which can effectively leverage illustrative instances and transfer useful knowledge from external resources by describing both entity types and mentions using a universal concept set.
• SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
  • Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
  • TLDR: We propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.
• Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation
  • Francesco Moramarco, Alex Papadopoulos Korfiatis, Mark Perera, Damir Juric, Jack Flann, Ehud Reiter, Anya Belz, Aleksandar Savkov
  • TLDR: We present an extensive human evaluation study of consultation notes and show that a simple, character-based Levenshtein distance metric performs on par if not better than common model-based metrics like BertScore.
• Unified Structure Generation for Universal Information Extraction
  • Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, Hua Wu
  • TLDR: We propose a unified text-to-structure generation framework for information extraction that can universally model different IE tasks, adaptively generate targeted structures, and collaboratively learn general IE abilities from different knowledge sources.
• Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering
  • Jing Zhang, Xiaokang Zhang, Jifan Yu, Jian Tang, Jie Tang, Cuiping Li, Hong Chen
  • TLDR: We propose a trainable subgraph retriever for knowledge base question answering, which improves the reasoning bias of existing KBQA methods.
• Pre-training to Match for Unified Low-shot Relation Extraction
  • Fangchao Liu, Hongyu Lin, Xianpei Han, Boxi Cao, Le Sun
  • TLDR: We propose Multi-Choice Matching Networks to unify low-shot relation extraction and meta-learning for few-shot and few-shots RE tasks.
• Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View
  • Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu, Le Sun
  • TLDR: We investigate the biases in the prompt-based probing and propose to debiase the results and improve the criteria for evaluating and applying PLMs.
• Evaluating Extreme Hierarchical Multi-label Classification
  • Enrique Amigo, Agustín Delgado
  • TLDR: We propose a new metric for evaluating the state of the art in NLP evaluation metrics based on a set of formal properties and show its suitability for multi-label Hierarchical Extreme classification.
• What does the sea say to the shore? A BERT based DST style approach for speaker to dialogue attribution in novels
  • Carolina Cuesta-Lazaro, Animesh Prasad, Trevor Wood
  • TLDR: We present a complete pipeline to extract characters in a novel and link them to their direct-speech utterances.
• Measuring Fairness of Text Classifiers via Prediction Sensitivity
  • Satyapriya Krishna, Rahul Gupta, Apurv Verma, Jwala Dhamala, Yada Pruksachatkun, Kai-Wei Chang
  • TLDR: We propose a new metric for fairness in machine learning models based on the model’s prediction sensitivity to perturbations in input features.
• RotateQVS: Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion
  • Kai Chen, Ye Wang, Yitong Li, Aiping Li
  • TLDR: We propose a novel temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space (RotateQVS) and relations as complex vectors in Hamilton’s quaternion space.
• Feeding What You Need by Understanding What You Learned
  • Xiaoqiang Wang, Bang Liu, Fangli Xu, Bo Long, Siliang Tang, Lingfei Wu
  • TLDR: Machine Reading Comprehension: Understanding text passage and answering questions based on it.
• Probing Simile Knowledge from Pre-trained Language Models
  • Weijie Chen, Yongzhu Chang, Rongsheng Zhang, Jiashu Pu, Guandan Chen, Le Zhang, Yadong Xi, Yijiang Chen, Chang Su
  • TLDR: We propose a unified framework for simile triple completion and mask training based on pre-trained language models for similes interpretation and simile generation tasks.
• An Effective and Efficient Entity Alignment Decoding Algorithm via Third-Order Tensor Isomorphism
  • Xin Mao, Meirong Ma, Hao Yuan, Jianchao Zhu, ZongYu Wang, Rui Xie, Wei Wu, Man Lan
  • TLDR: We propose a new and efficient entity alignment algorithm via third-order tensor isomorphism equations.
• Entailment Graph Learning with Textual Entailment and Soft Transitivity
  • Zhibin Chen, Yansong Feng, Dongyan Zhao
  • TLDR: We propose a two-stage method for entailment graph with textual entailment and logical transitivity.
• Logic Traps in Evaluating Attribution Scores
  • Yiming Ju, Yuanzhe Zhang, Zhao Yang, Zhongtao Jiang, Kang Liu, Jun Zhao
  • TLDR: We present a systematic analysis of the logic traps in existing methods for evaluating attribution scores and show that they are often ignored in most works.
• Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network
  • Zheng Gong, Kun Zhou, Xin Zhao, Jing Sha, Shijin Wang, Ji-Rong Wen
  • TLDR: We propose a new approach to continually pre-train language models for improving the understanding of math problems.
• Multitasking Framework for Unsupervised Simple Definition Generation
  • Cunliang Kong, Yun Chen, Hengyuan Zhang, Liner Yang, Erhong Yang
  • TLDR: SimpDefiner is a novel multi-tasker for simple definition generation in language learners and low literacy readers.
• Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction
  • Zhanming Jie, Jierui Li, Wei Lu
  • TLDR: We propose a novel approach to solve math word problems that presents explainable deductive reasoning steps to iteratively construct target expressions.
• When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues
  • Shivani Kumar, Atharva Kulkarni, Md Shad Akhtar, Tanmoy Chakraborty
  • TLDR: We propose a novel task for analysing and generating explanations of sarcastic conversations.
• Toward Interpretable Semantic Textual Similarity via Optimal Transport-based Contrastive Sentence Learning
  • Seonghyeon Lee, Dongha Lee, Seongbo Jang, Hwanjo Yu
  • TLDR: We propose a novel contrastive learning framework for sentence similarity and interpretable sentence similarity.
• Pre-training and Fine-tuning Neural Topic Model: A Simple yet Effective Approach to Incorporating External Knowledge
  • Linhai Zhang, Xuemeng Hu, Boyu Wang, Deyu Zhou, Qian-Wen Zhang, Yunbo Cao
  • TLDR: We propose a novel strategy to incorporate external knowledge into neural topic modeling where the neural topic model is pre-trained on a large corpus and then fine-tuned on the target dataset.
• Multi-View Document Representation Learning for Open-Domain Dense Retrieval
  • Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, Nan Duan
  • TLDR: We propose a novel multi-view document representation learning framework, aiming to produce multi-way embeddings to represent documents and enforce them to align with different queries.
• Graph Pre-training for AMR Parsing and Generation
  • Xuefeng Bai, Yulong Chen, Yue Zhang
  • TLDR: We propose graph self-supervised graph pre-training for graph-to-graph graph parsing and graph-based graph-level generation of abstract meaning representation.
• Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills
  • Ori Yoran, Alon Talmor, Jonathan Berant
  • TLDR: We propose to generate question-paragraph pairs that require reasoning skills, and show that our model substantially outperforms T5, a popular pre-trained encoder-decoder model.
• RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
  • Xi Ye, Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou, Caiming Xiong
  • TLDR: We present RnG-KBQA, a Rank-and-Generate approach for KBQA that improves generalization and generalizes to unseen schema items.
• Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling
  • Prathyusha Jwalapuram, Shafiq Joty, Xiang Lin
  • TLDR: We propose a new neural coherence model that improves the quality of text generation and evaluation on downstream tasks.
• Just Rank: Rethinking Evaluation with Word and Sentence Similarities
  • Bin Wang, C.-C. Jay Kuo, Haizhou Li
  • TLDR: We present a new intrinsic evaluation method for word and sentence embedding models that is much stronger than the current gold standard.
• MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding
  • Junlong Li, Yiheng Xu, Lei Cui, Furu Wei
  • TLDR: We propose MarkupLM for document understanding tasks with markup languages as the backbone, such as HTML/XML-based documents, where text and markup information is jointly pre-trained.
• CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment
  • Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, Furu Wei
  • TLDR: We empirically show that CLIP can be a strong vision-language few-shot learner by leveraging the power of language.
• KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base
  • Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, Hanwang Zhang
  • TLDR: We present a dataset for Complex Knowledge Base Question answering over Knowledge Base and provide a compositional and interpretable programming language KoPL to represent the reasoning process of complex questions.
• Debiased Contrastive Learning of Unsupervised Sentence Representations
  • Kun Zhou, Beichen Zhang, Xin Zhao, Ji-Rong Wen
  • TLDR: We present a new framework for contrastive learning that aims to improve the alignment of sentence representations and reduce the sampling bias.
• MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators
  • Zhixing Tan, Xiangwen Zhang, Shuo Wang, Yang Liu
  • TLDR: We present Multi-Stage Prompting, a simple and automatic approach for leveraging pre-trained language models to translation tasks.
• SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialogues
  • Ssu Chiu, Maolin Li, Yen-Ting Lin, Yun-Nung Chen
  • TLDR: We propose a framework to automatically generate dialogues without human involvement in the open-domain dialogue generation and show that the generated dialogues have a natural flow and can be used for guiding future research directions and commercial activities.
• UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining
  • Jiacheng Li, Jingbo Shang, Julian McAuley
  • TLDR: We propose a novel unsupervised contrastive learning framework for context-aware phrase representations and topic mining.
• XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
  • Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
  • TLDR: We present ELECTRA-style tasks to cross-lingual language model pre-training and show that XLM-E outperforms the baseline models on various cross-language understanding tasks with much less computation cost.
• Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing
  • Chao Lou, Songlin Yang, Kewei Tu
  • TLDR: We propose a novel method for nested named entity recognition using lexicalized constituency trees and a new masking algorithm for partial marginalization and inference.
• Can Explanations Be Useful for Calibrating Black Box Models?
  • Xi Ye, Greg Durrett
  • TLDR: We propose a new method for improving black box models by leveraging explanations of the model’s behavior.
• OIE@OIA: an Adaptable and Efficient Open Information Extraction Framework
  • Xin Wang, Minlong Peng, Mingming Sun, Ping Li
  • TLDR: We propose a new adaptable and efficient OIE system that can adapt to different OIE tasks with simple rules.
• ReACC: A Retrieval-Augmented Code Completion Framework
  • Shuai Lu, Nan Duan, Hojae Han, Daya Guo, Seung-won Hwang, Alexey Svyatkovskiy
  • TLDR: We propose a retrieval-augmented code completion framework that uses external context to learn to predict the following code token(s) according to the code context.
• Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED
  • Quzhe Huang, Shibo Hao, Yuan Ye, Shengqi Zhu, Yansong Feng, Dongyan Zhao
  • TLDR: We show that the recommend-revise scheme for document-level relation extraction is not only biased towards popular entities and relations, but also discourages annotators from supplementing adequate instances in the revision phase.
• UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
  • Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Scott Yih, Madian Khabsa
  • TLDR: Unified framework for language model tuning that incorporates different PELT methods as submodules and learns to activate the ones that best suit the current data or task setup via gating mechanism.
• An Empirical Study of Memorization in NLP
  • Xiaosen Zheng, Jing Jiang
  • TLDR: We empirically show that top-ranked memorized training instances are likely atypical, and we develop an attribution method to better understand why a training instance is memorized.
• AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
  • Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, Vishrav Chaudhary, Luis Chiruzzo, Angela Fan, John Ortega, Ricardo Ramos, Annette Rios, Ivan Vladimir Meza Ruiz, Gustavo Giménez-Lugo, Elisabeth Mager, Graham Neubig, Alexis Palmer, Rolando Coto-Solano, Thang Vu, Katharina Kann
  • TLDR: We present AmericasNLINLI, a new multilingual model for unseen languages, and show that it is possible to perform cross-lingual transfer in a zero-shot setting, even for languages unseen during pretraining.
• Towards Learning (Dis)-Similarity of Source Code from Program Contrasts
  • Yangruibo Ding, Luca Buratti, Saurabh Pujar, Alessandro Morari, Baishakhi Ray, Saikat Chakraborty
  • TLDR: We present DISCO (DIS-similarity of COde), a novel self-supervised model focusing on identifying (dis)similar functionalities of source code.
• Guided Attention Multimodal Multitask Financial Forecasting with Inter-Company Relationships and Global and Local News
  • Gary Ang, Ee-Peng Lim
  • TLDR: We propose a model that captures both global and local multimodal information for investment and risk management-related forecasting tasks.
• On Vision Features in Multimodal Machine Translation
  • Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, JingBo Zhu
  • TLDR: We investigate the impact of vision models on multimodal machine translation and show that stronger vision models are helpful for learning translation from the visual modality.
• CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning
  • Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca Passonneau, Rui Zhang
  • TLDR: We present CONTaiNER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few-Shot NER.
• Cree Corpus: A Collection of nêhiyawêwin Resources
  • Daniela Teodorescu, Josie Matalski, Delaney Lothian, Denilson Barbosa, Carrie Demmans Epp
  • TLDR: We develop a corpus of nêhiyawêwin language data for use in language technologies for nêhiyawêw.
• Learning to Rank Visual Stories From Human Ranking Data
  • Chi-Yang Hsu, Yun-Wei Chu, Vincent Chen, Kuan-Chieh Lo, Chacha Chen, Ting-Hao Huang, Lun-Wei Ku
  • TLDR: We present a novel reference-free VIST metric for story evaluation that is significantly more aligned to human evaluation than other metrics.
• Universal Conditional Masked Language Pre-training for Neural Machine Translation
  • Pengfei Li, Liangyou Li, Meng Zhang, Minghao Wu, Qun Liu
  • TLDR: We propose a novel language model for neural machine translation that can improve both Autoregressive and non-autoregressive NMT.
• CARETS: A Consistency And Robustness Evaluative Test Suite for VQA
  • Carlos E. Jimenez, Olga Russakovsky, Karthik Narasimhan
  • TLDR: We present CARETS, a systematic test suite to measure consistency and robustness of modern VQA models through a series of six fine-grained capability tests.
• Phrase-aware Unsupervised Constituency Parsing
  • Xiaotao Gu, Yikang Shen, Jiaming Shen, Jingbo Shang, Jiawei Han
  • TLDR: We propose phrase-guided masking for constituency parsing and phrase regularization for unsupervised grammar induction.
• Achieving Reliable Human Assessment of Open-Domain Dialogue Systems
  • Tianbo Ji, Yvette Graham, Gareth Jones, Chenyang Lyu, Qun Liu
  • TLDR: We present a new method for evaluating open-domain dialogue systems that is highly reliable and feasible while still remaining feasible and low cost.
• Updated Headline Generation: Creating Updated Summaries for Evolving News Stories
  • Sheena Panthaplackel, Adrian Benton, Mark Dredze
  • TLDR: We propose the task of updated headline generation, in which a system generates a headline for an updated article, considering both the previous article and headline.
• SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures
  • Megan Ung, Jing Xu, Y-Lan Boureau
  • TLDR: We propose SaFeRDialogues, a task and dataset of graceful responses to conversational feedback about safety failures.
• Compositional Generalization in Dependency Parsing
  • Emily Goodwin, Siva Reddy, Timothy O’Donnell, Dzmitry Bahdanau
  • TLDR: We show that increasing compound divergence degrades dependency parsing performance, although not as dramatically as semantic parsing performance.
• ASPECTNEWS: Aspect-Oriented Summarization of News Documents
  • Ojas Ahuja, Jiacheng Xu, Akshay Gupta, Kevin Horecka, Greg Durrett
  • TLDR: We present a dataset of realistic aspect-oriented summaries for earthquakes and fraud investigations, which are better than generic summaries.
• MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes
  • Nianlong Gu, Elliott Ash, Richard Hahnloser
  • TLDR: We introduce MemSum (Multi-step Episodic Markov decision process extractive SUMmarizer), a reinforcement-learning-based extractive summarizer enriched at each step with information on the current extraction history.
• CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations
  • Rakesh R. Menon, Sayan Ghosh, Shashank Srivastava
  • TLDR: We present a benchmark for classifier learning using natural language explanations and a entailment-based model for describing the influence of explanations on classifier classification.
• Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing
  • Freda Shi, Kevin Gimpel, Karen Livescu
  • TLDR: We present substructure distribution projection (SubDP), a technique that projects a distribution over structures in one domain to another, by projecting substructure distributions separately.
• Multilingual Detection of Personal Employment Status on Twitter
  • Manuel Tonneau, Dhaval Adjodah, Joao Palotti, Nir Grinberg, Samuel Fraiberger
  • TLDR: We identify five types of disclosures about individuals’ employment status on social media using BERT-based classification models and show that a small number of AL iterations is sufficient to obtain large and significant gains in precision, recall, and diversity of results compared to a supervised baseline.
• MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data
  • Yilun Zhao, Yunxiang Li, Chenying Li, Rui Zhang
  • TLDR: We present a new large-scale benchmark for numerical reasoning over hybrid data with multi-step reasoning processes and support facts.
• Transformers in the loop: Polarity in neural models of language
  • Lisa Bylinina, Alexey Tikhonov
  • TLDR: We show that metrics derived from language models are more consistent with data from psycholinguistic experiments than linguistic theory predictions.
• Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation
  • Zhiwei He, Xing Wang, Rui Wang, Shuming Shi, Zhaopeng Tu
  • TLDR: Back-translation is a critical component of Unsupervised Neural Machine Translation (UNMT), which generates pseudo parallel data from target monolingual data.
• SDR: Efficient Neural Re-ranking using Succinct Document Representation
  • Nachshon Cohen, Amit Portnoy, Besnik Fetahu, Amir Ingber
  • TLDR: We propose a novel algorithm for efficient ranking models that uses a novel document representation for ranking models.
• The AI Doctor Is In: A Survey of Task-Oriented Dialogue Systems for Healthcare Applications
  • Mina Valizadeh, Natalie Parde
  • TLDR: We identify 70 papers discussing the system-level implementation of task-oriented dialogue systems for healthcare applications.
• SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher
  • Thai Le, Noseong Park, Dongwon Lee
  • TLDR: We propose a novel algorithm, SHIELD, which modifies and re-trains only the last layer of a textual NN, and thus it “patches” and “transforms” the NN into a stochastic weighted ensemble of multi-expert prediction heads.
• Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding
  • Soumya Chatterjee, Sunita Sarawagi, Preethi Jyothi
  • TLDR: We propose a novel posterior alignment technique that is truly online in its execution and superior in terms of alignment error rates compared to existing methods.
• Leveraging Task Transferability to Meta-learning for Clinical Section Classification with Limited Data
  • Zhuohao Chen, Jangwon Kim, Ram Bhakta, Mustafa Sir
  • TLDR: We propose a novel algorithm for improving cross-domain meta-learning based on the NNCE measure and propose a new task transferability measure for cross-dimensional clinical note data.
• Reinforcement Guided Multi-Task Learning Framework for Low-Resource Stereotype Detection
  • Rajkumar Pujari, Erik Oveson, Priyanka Kulkarni, Elnaz Nouri
  • TLDR: We present a novel evaluation set for large Pre-trained Language Models that addresses the reliability issues of existing benchmark datasets by de-constructing various ways in which stereotypes manifest in text.
• Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings
  • Sidsel Boldsen, Patrizia Paggio
  • TLDR: We propose a new method for detecting sound change through historical spelling and show that it can be used to trace the change of lenition of plosives in Danish historical sources.
• A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
  • Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan
  • TLDR: We propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDeS (HAllucination DEtection dataSet).
• Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice
  • Andreas Grivas, Nikolay Bogoychev, Adam Lopez
  • TLDR: We show that large language models and translation models can have unargmaxable tokens, but they are very rare and unlikely to impact model quality.
• Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction
  • Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao
  • TLDR: We propose a novel model for sentence-level and document-level event argument extraction that captures argument interactions via multi-role prompts and generalizes well when there is a lack of training data.
• Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework
  • Shaolei Zhang, Yang Feng
  • TLDR: We propose a Length-Aware Framework to reduce the position bias in SiMT by bridging the structural gap between SiMT and full-sentence MT.
• A Statutory Article Retrieval Dataset in French
  • Antoine Louis, Gerasimos Spanakis
  • TLDR: We present a novel dataset for statutory article retrieval and benchmark several state-of-the-art retrieval architectures, including lexical and dense architectures, both in zero-shot and supervised setups.
• ParaDetox: Detoxification with Parallel Data
  • Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, Alexander Panchenko
  • TLDR: We present a novel pipeline for the collection of parallel data for the detoxification task.
• Interpreting Character Embeddings With Perceptual Representations: The Case of Shape, Sound, and Color
  • Sidsel Boldsen, Manex Agirrezabal, Nora Hollenstein
  • TLDR: We propose a new set of probing classifiers to intrinsically evaluate what phonological information is encoded in character embeddings.
• Fine-Grained Controllable Text Generation Using Non-Residual Prompting
  • Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, Magnus Sahlgren
  • TLDR: We propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps, and demonstrate its potential on various experiments, including the novel task of contextualized word inclusion.
• Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features
  • Florian Lux, Thang Vu
  • TLDR: We use embeddings derived from articulatory vectors rather than phoneme identities to learn phoneme representations that hold across languages.
• TwittIrish: A Universal Dependencies Treebank of Tweets in Modern Irish
  • Lauren Cassidy, Teresa Lynn, James Barry, Jennifer Foster
  • TLDR: We present a treebank of dependencies for natural language processing of user-generated content in Irish.
• Length Control in Abstractive Summarization by Pretraining Information Selection
  • Yizhu Liu, Qi Jia, Kenny Zhu
  • TLDR: We propose a length-aware attention mechanism for summarization that adapts the encoding of the source based on the desired length.
• CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation
  • Zichu Fei, Qi Zhang, Tao Gui, Di Liang, Sirui Wang, Wei Wu, Xuanjing Huang
  • TLDR: We propose the CQG, a novel multi-hop question generation framework that improves multi-hob question generation and improves performance on HotpotQA.
• Word Order Does Matter and Shuffled Language Models Know It
  • Mostafa Abdou, Vinit Ravishankar, Artur Kulmizev, Anders Søgaard
  • TLDR: Language models trained on text shuffled after subword segmentation retain some semblance of information about word order because of the statistical dependencies between sentence length and unigram probabilities.
• An Empirical Study on Explanations in Out-of-Domain Settings
  • George Chrysostomou, Nikolaos Aletras
  • TLDR: We show that in many cases out-of-domain post-hoc explanation faithfulness is higher than in-domain.
• MILIE: Modular & Iterative Multilingual Open Information Extraction
  • Bhushan Kotnis, Kiril Gashteovski, Daniel Rubio, Ammar Shaker, Vanesa Rodriguez-Tembras, Makoto Takamoto, Mathias Niepert, Carolin Lawrence
  • TLDR: We propose a neural OpenIE system that operates in an iterative fashion, and show that it outperforms SOTA systems on multiple languages ranging from Chinese to Arabic.
• What Makes Reading Comprehension Questions Difficult?
  • Saku Sugawara, Nikita Nangia, Alex Warstadt, Samuel Bowman
  • TLDR: We crowdsource multiple-choice reading comprehension questions for natural language understanding benchmarks and show that passage source, length, and readability measures do not significantly affect question difficulty.
• From Simultaneous to Streaming Machine Translation by Leveraging Streaming History
  • Javier Iranzo Sanchez, Jorge Civera, Alfons Juan-Císcar
  • TLDR: We propose a stream-level adaptation of the current sentence-level latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task.
• A Rationale-Centric Framework for Human-in-the-loop Machine Learning
  • Jinghui Lu, Linyi Yang, Brian Namee, Yue Zhang
  • TLDR: We present a novel rational-centric framework with human-in-the-loop for improving model out-of-distribution performance in few-shot learning scenarios.
• Challenges and Strategies in Cross-Cultural NLP
  • Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, Anders Søgaard
  • TLDR: We propose a principled framework for cross-cultural and multicultural NLP, and survey existing and potential strategies.
• Prototypical Verbalizer for Prompt-based Few-shot Tuning
  • Ganqu Cui, Shengding Hu, Ning Ding, Longtao Huang, Zhiyuan Liu
  • TLDR: We propose a novel verbalizer for pre-trained language models that learns prototype vectors as verbalizers by contrastive learning.
• Clickbait Spoiling via Question Answering and Passage Retrieval
  • Matthias Hagen, Maik Fröbe, Artur Jurk, Martin Potthast
  • TLDR: We introduce and study the task of clickbait spoiling: generating a short text that satisfies the curiosity induced by a clickbail post.
• BERT Learns to Teach: Knowledge Distillation with Meta Learning
  • Wangchunshu Zhou, Canwen Xu, Julian McAuley
  • TLDR: We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training.
• STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation
  • Qingkai Fang, Rong Ye, Lei Li, Yang Feng, Mingxuan Wang
  • TLDR: We propose a new method to learn a better speech representation for end-to-end speech-to text translation with limited labeled data.
• Integrating Vectorized Lexical Constraints for Neural Machine Translation
  • Shuo Wang, Zhixing Tan, Yang Liu
  • TLDR: We propose to open up the black box of neural machine translation by directly integrating lexical constraints into NMT models.
• MPII: Multi-Level Mutual Promotion for Inference and Interpretation
  • Yan Liu, Sanyuan Chen, Yazheng Yang, Qi Dai
  • TLDR: We propose a multi-level Mutual Promotion mechanism for self-evolved Inference and sentence-level Interpretation (MPII) that can significantly improve the inference performance and the interpretation quality.
• StableMoE: Stable Routing Strategy for Mixture of Experts
  • Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei
  • TLDR: We propose StableMoE, a novel method for learning to route Mixture-of-Experts using lightweight router and stable routing strategy.
• Boundary Smoothing for Named Entity Recognition
  • Enwei Zhu, Jinpeng Li
  • TLDR: We propose boundary smoothing as a regularization technique for span-based neural NER models.
• Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification
  • Zihan Wang, Peiyi Wang, Lianzhe Huang, Xin Sun, Houfeng Wang
  • TLDR: We propose Hierarchy-guided Contrastive Learning to embed the label hierarchy into a text encoder and use it to generate the hierarchy-aware text representation independently.
• Signal in Noise: Exploring Meaning Encoded in Random Character Sequences with Character-Aware Language Models
  • Mark Chu, Bhargav Srinivasa Desikan, Ethan Nadler, Donald Ruggiero Lo Sardo, Elise Darragh-Ford, Douglas Guilbeault
  • TLDR: We show that randomly generated character n-grams lack meaning but contain primitive information based on the distribution of characters they contain.
• Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering
  • Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen
  • TLDR: We propose HyperLink-induced Pre-training, a method to pre-train the dense retriever with the text relevance induced by hyperlink-based topology within Web documents.
• AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension
  • Xiao Li, Gong Cheng, Ziheng Chen, Yawei Sun, Yuzhong Qu
  • TLDR: We present a novel neural-symbolic approach for logical reasoning over text.
• CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
  • Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao
  • TLDR: We propose a new ensemble learning approach based on perturbed models that improves generalization performance and memory efficiency.
• Interpretability for Language Learners Using Example-Based Grammatical Error Correction
  • Masahiro Kaneko, Sho Takase, Ayana Niwa, Naoaki Okazaki
  • TLDR: We present an example-based GEC model that presents examples to language learners as a basis for a correction result.
• Rethinking Negative Sampling for Handling Missing Entity Annotations
  • Yangming Li, Lemao Liu, Shuming Shi
  • TLDR: We propose a new negative sampling distribution that improves negative sampling and a new adaptive and weighted sampling distribution.
• Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning
  • Kang Zhou, Yuepei Li, Qi Li
  • TLDR: We propose a novel Confidence-based MPU based approach for the named entity recognition task under distant supervision.
• UniXcoder: Unified Cross-Modal Pre-training for Code Representation
  • Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, Jian Yin
  • TLDR: Unified cross-modal pre-trained model for programming language that learns representation of code fragment with contrastive learning and optimizes code completion.
• One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
  • Alham Fikri Aji, Genta Indra Winata, Fajri Koto, Samuel Cahyawijaya, Ade Romadhony, Rahmad Mahendra, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Timothy Baldwin, Jey Han Lau, Sebastian Ruder
  • TLDR: We provide an overview of the current state of NLP research for Indonesia’s 700+ languages and provide general recommendations to help develop NLP technology not only for languages of Indonesia but also other underrepresented languages.
• Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
  • Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, Yejin Choi
  • TLDR: We propose a new framework called Scarecrow for scrutinizing machine text via crowd annotation.
• Transkimmer: Transformer Learns to Layer-wise Skim
  • Yue Guan, Zhengyi Li, Jingwen Leng, Zhouhan Lin, Minyi Guo
  • TLDR: We propose Transkimmer architecture, which learns to identify hidden state tokens that are not required by each layer.
• SkipBERT: Efficient Inference with Shallow Layer Skipping
  • Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley
  • TLDR: We propose SkipBERT to accelerate BERT inference by skipping the computation of shallow layers.
• Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models
  • Ryokan Ri, Yoshimasa Tsuruoka
  • TLDR: We investigate what kind of structural knowledge learned in neural network encoders is transferable to processing natural language.
• mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
  • Ryokan Ri, Ikuya Yamada, Yoshimasa Tsuruoka
  • TLDR: We train a multilingual language model with 24 languages with entity representations and showthe model consistently outperforms word-based pretrained models in various cross-lingual transfer tasks.
• Evaluating Factuality in Text Simplification
  • Ashwin Devaraj, William Sheffield, Byron Wallace, Junyi Jessy Li
  • TLDR: We present a taxonomy of errors in automated simplification models and show that errors often appear in both datasets and outputs.
• Requirements and Motivations of Low-Resource Speech Synthesis for Language Revitalization
  • Aidan Pine, Dan Wells, Nathan Brinklow, Patrick Littell, Korin Richmond
  • TLDR: We propose a new approach to speech synthesis for language revitalization by building speech synthesis systems for three Indigenous languages spoken in Canada.
• Sharpness-Aware Minimization Improves Language Model Generalization
  • Dara Bahri, Hossein Mobahi, Yi Tay
  • TLDR: We show that Sharpness-aware minimization can substantially improve the generalization of language models without much computational overhead.
• Adversarial Authorship Attribution for Deobfuscation
  • Wanyue Zhai, Jonathan Rusert, Zubair Shafiq, Padmini Srinivasan
  • TLDR: We investigate the problem of adversarial authorship attribution for deobfuscation and show that adversarially trained authorship attributors are able to degrade the effectiveness of existing obfuscators from 20-30% to 5-10%.
• Weakly Supervised Word Segmentation for Computational Language Documentation
  • Shu Okabe, Laurent Besacier, François Yvon
  • TLDR: Weak supervision can be used to improve word and morpheme segmentation in documentary linguistics.
• SciNLI: A Corpus for Natural Language Inference on Scientific Text
  • Mobashir Sadat, Cornelia Caragea
  • TLDR: We present SciNLINLI, a large dataset for NLI that captures the formality in scientific text and contains 107,412 sentence pairs extracted from scholarly papers on NLP and computational linguistics.
• Neural reality of argument structure constructions
  • Bai Li, Zining Zhu, Guillaume Thomas, Frank Rudzicz, Yang Xu
  • TLDR: We propose a new method for probing for argument structure constructions in Transformer-based language models and show that LMs associate ASCs with meaning, even in semantically nonsensical sentences.
• On the Robustness of Offensive Language Classifiers
  • Jonathan Rusert, Zubair Shafiq, Padmini Srinivasan
  • TLDR: We systematically analyze the robustness of offensive language classifiers against more crafty adversarial attacks that leverage greedy- and attention-based word selection and context-aware embeddings for word replacement.
• Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
  • Kalpesh Krishna, Deepak Nathani, Xavier Garcia, Bidisha Samanta, Partha Talukdar
  • TLDR: We propose a new method for style transfer in languages where style-labelled corpora are not available.
• ABC: Attention with Bounded-memory Control
  • Hao Peng, Jungo Kasai, Nikolaos Pappas, Dani Yogatama, Zhaofeng Wu, Lingpeng Kong, Roy Schwartz, Noah A. Smith
  • TLDR: We present a new approach to causal attention that improves the inference time and space efficiency with no or negligible accuracy loss.
• The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail
  • Samuel Bowman
  • TLDR: We present a new paper that addresses the problem of misleading or false claims about the limits of our best technology.
• RELiC: Retrieving Evidence for Literary Claims
  • Katherine Thai, Yapei Chang, Kalpesh Krishna, Mohit Iyyer
  • TLDR: We present a novel task for literary evidence retrieval in which uses a novel method of dense passage retrieval and propose a novel algorithm for its retrieval.
• Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas
  • Raphael Schumann, Stefan Riezler
  • TLDR: Visual vision and language navigation on unseen data.
• Adapting Coreference Resolution Models through Active Learning
  • Michelle Yuan, Patrick Xia, Chandler May, Benjamin Van Durme, Jordan Boyd-Graber
  • TLDR: We explore how to actively label coreference, examining sources of model uncertainty and document reading costs.
• An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models
  • Sweta Agrawal, Marine Carpuat
  • TLDR: We propose a framework for training sequence-to-sequence models for editing tasks, where the original input sequence is iteratively edited to produce the output.
• Memorisation versus Generalisation in Pre-trained Language Models
  • Michael Tänzer, Sebastian Ruder, Marek Rei
  • TLDR: We study how language models learn in noisy and low-resource scenarios and propose a novel extension based on prototypical networks that improves performance in low-frequency named entity recognition tasks.
• ChatMatch: Evaluating Chatbots by Autonomous Chat Tournaments
  • Ruolan Yang, Zitong Li, Haifeng Tang, Kenny Zhu
  • TLDR: We propose an interactive chatbot evaluation framework in which chatbots compete with each other like in a sports tournament, using flexible scoring metrics.
• Do self-supervised speech models develop human-like perception biases?
  • Juliette Millet, Ewan Dunbar
  • TLDR: We show that the CPC model shows a small native language effect, but that wav2vec and HuBERT seem to develop a universal speech perception space which is not language specific.
• Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
  • Jing Gu, Eliana Stefani, Qi Wu, Jesse Thomason, Xin Wang
  • TLDR: We review contemporary studies in the emerging field of vision-and-language navigation and provide a thorough reference for the VLN research community.
• Learning to Generate Programs for Table Fact Verification via Structure-Aware Semantic Parsing
  • Suixin Ou, Yongmei Liu
  • TLDR: We propose Structure-Aware Semantic Parsing to generate latent programs from statements and show that it is more accurate than existing semantic parsers.
• Cluster & Tune: Boost Cold Start Performance in Text Classification
  • Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, Noam Slonim
  • TLDR: We propose an intermediate unsupervised classification task for text classification tasks, which can significantly improve performance.
• Overcoming a Theoretical Limitation of Self-Attention
  • David Chiang, Peter Cholak
  • TLDR: We show that transformers can fail to generalize to longer strings when they need to focus on a single position.
• Prediction Difference Regularization against Perturbation for Neural Machine Translation
  • Dengji Guo, Zhengrui Ma, Min Zhang, Yang Feng
  • TLDR: We propose prediction difference regularization for ground-truth tokens to analyze the fitting of token-level samples and find that under-fitting is almost as common as over-fitting.
• Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages
  • Wietse de Vries, Martijn Wieling, Malvina Nissim
  • TLDR: We show that pre-training of both source and target languages, as well as matching language families, writing systems, word order systems, and lexical-phonetic distance significantly impact cross-lingual performance.
• Should a Chatbot be Sarcastic? Understanding User Preferences Towards Sarcasm Generation
  • Silviu Vlad Oprea, Steven Wilson, Walid Magdy
  • TLDR: We propose a theory-driven framework for generating sarcastic responses that are recognisable to humans.
• How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation?
  • Xunjian Yin, Xiaojun Wan
  • TLDR: We evaluate the ability of Seq2Seq models on end-to-end data-to text generation and find that the quality of the output is not as good as the output of human authors.
• Probing for Labeled Dependency Trees
  • Max Müller-Eberstein, Rob van der Goot, Barbara Plank
  • TLDR: We propose a new method for extracting labeled and directed dependency parse trees from embeddings while using fewer parameters and compute than prior methods.
• DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation
  • Nitay Calderon, Eyal Ben-David, Amir Feder, Roi Reichart
  • TLDR: We propose a controllable generation approach for domain adaptation in NLP that generates coherent counterfactuals consisting of multiple sentences.
• LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
  • Jiapeng Wang, Lianwen Jin, Kai Ding
  • TLDR: Language-independent Layout Transformer for structured document understanding.
• Dependency-based Mixture Language Models
  • Zhixian Yang, Xiaojun Wan
  • TLDR: We present a novel dependency modeling objective for neural language models and show that it can be easily and effectively applied to different neural language model while improving neural text generation on various tasks.
• Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining?
  • Subhabrata Dutta, Jeevesh Juneja, Dipankar Das, Tanmoy Chakraborty
  • TLDR: We propose a novel transfer learning strategy to overcome the constraints of argument mining.
• Entity-based Neural Local Coherence Modeling
  • Sungho Jeon, Michael Strube
  • TLDR: We propose an entity-based neural local coherence model which is linguistically more sound than previously proposed neural coherence models.
• “That Is a Suspicious Reaction!”: Interpreting Logits Variation to Detect NLP Adversarial Attacks
  • Edoardo Mosca, Shreyash Agarwal, Javier Rando Ramírez, Georg Groh
  • TLDR: We present a model-agnostic detector for adversarial text examples in natural language processing.
• Local Languages, Third Spaces, and other High-Resource Scenarios
  • Steven Bird
  • TLDR: Language technology can address the diverse situations of the world’s languages.
• That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory
  • Xuemei Tang, Qi Su
  • TLDR: We propose a cross-era learning framework for Chinese word segmentation and machine translation, CROSSWISE, which uses the Switch-memory module to incorporate era-specific linguistic knowledge.
• Fair and Argumentative Language Modeling for Computational Argumentation
  • Carolin Holtermann, Anne Lauscher, Simone Ponzetto
  • TLDR: We propose a novel method for quantifying and quantifying the intrinsic bias in argumentative language models and show that it can be removed sustainably and sustainably.
• Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation
  • Ruiqing Zhang, Zhongjun He, Hua Wu, Haifeng Wang
  • TLDR: We propose an adaptive segmentation policy for end-to-end simultaneous speech-to text translation that learns to segment the source streaming speech into meaningful units by considering both acoustic features and translation history.
• Can Pre-trained Language Models Interpret Similes as Smart as Human?
  • Qianyu He, Sijie Cheng, Zhixu Li, Rui Xie, Yanghua Xiao
  • TLDR: We propose a novel task to let PLMs infer simile properties from textual corpora and human-designed questions.
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
  • Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
  • TLDR: We present the first Chinese biomedical language understanding evaluation benchmark and a platform for evaluation and comparison of neural models.
• Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
  • Puyuan Liu, Chenyang Huang, Lili Mou
  • TLDR: We propose a novel algorithm for text summarization that uses edit-based search and non-autoregressive Transformer to generate pseudo-groundtruths for text summary.
• Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation
  • Xiangpeng Wei, Heng Yu, Yue Hu, Rongxiang Weng, Weihua Luo, Rong Jin
  • TLDR: We present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT), which augments each training instance with an adjacency semantic region that could cover adequate variants of literal expression under the same meaning.
• Lexical Knowledge Internalization for Neural Dialog Generation
  • Zhiyong Wu, Wei Bi, Xiang Li, Lingpeng Kong, Ben Kao
  • TLDR: We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models.
• Modeling Syntactic-Semantic Dependency Correlations in Semantic Role Labeling Using Mixture Models
  • Junjie Chen, Xiangheng He, Yusuke Miyao
  • TLDR: We propose a mixture model-based end-to-end method to model the syntactic-semantic dependency correlation in Semantic Role Labeling.
• Learning the Beauty in Songs: Neural Singing Voice Beautifier
  • Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao
  • TLDR: We propose a novel generative model for singing voice beautification and a novel time-warping approach for pitch correction.
• A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation
  • Yu Cao, Wei Bi, Meng Fang, Shuming Shi, Dacheng Tao
  • TLDR: We propose a data manipulation method for persona-based dialogue generation that is model-agnostic to be packed with any persona- based dialogue generation model to improve their performance.
• LinkBERT: Pretraining Language Models with Document Links
  • Michihiro Yasunaga, Jure Leskovec, Percy Liang
  • TLDR: We propose LinkBERT, an LM pretraining method that leverages links between documents, e.g., hyperlinks, to capture dependencies and knowledge that span across documents.
• Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphs
  • Chao Shang, Guangtao Wang, Peng Qi, Jing Huang
  • TLDR: We propose a time-sensitive question answering framework for temporal knowledge graphs that can answer complex questions that require multiple steps of reasoning over facts in the temporal KG.
• Self-supervised Semantic-driven Phoneme Discovery for Zero-resource Speech Recognition
  • Liming Wang, Siyuan Feng, Mark Hasegawa-Johnson, Chang Yoo
  • TLDR: We propose a novel neural discrete representation learning model for phoneme inventory with raw speech and word labels.
• Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions
  • Haw-Shiuan Chang, Andrew McCallum
  • TLDR: We show that the single hidden state of neural language models cannot produce all probability distributions regardless of the LM size or training data size because the single word embedding cannot be close to the embeddings of all the possible next words simultaneously when there are other interfering word embeddINGS between them.
• Ditch the Gold Standard: Re-evaluating Conversational Question Answering
  • Huihan Li, Tianyu Gao, Manan Goenka, Danqi Chen
  • TLDR: We present a large-scale human evaluation of state-of-the-art conversational question answering systems, where human evaluators converse with models and judge the correctness of their answers.
• Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
  • Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp
  • TLDR: We show that the order in which the samples are provided can make the difference between near state-of-the-art and random guess performance: essentially some permutations are “fantastic” and some not.
• Situated Dialogue Learning through Procedural Environment Generation
  • Prithviraj Ammanabrolu, Renee Jia, Mark Riedl
  • TLDR: We teach goal-driven agents to interactively act and speak in situated environments by training on generated curriculums.
• UniTE: Unified Translation Evaluation
  • Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Derek Wong, Lidia Chao
  • TLDR: Translation quality evaluation plays a crucial role in machine translation.
• Program Transfer for Answering Complex Questions over Knowledge Bases
  • Shulin Cao, Jiaxin Shi, Zijun Yao, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Zhiyuan Liu, Jinghui Xiao
  • TLDR: We propose a novel two-stage parsing framework for program induction over knowledge bases and a novel ontology-guided pruning strategy for program transfer.
• EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation
  • Yulin Xu, Zhen Yang, Fandong Meng, Jie Zhou
  • TLDR: Extract and Generate multi-way aligned bilingual corpus from bilingual data.
• Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings
  • Jiangbin Zheng, Yile Wang, Ge Wang, Jun Xia, Yufei Huang, Guojiang Zhao, Yue Zhang, Stan Li
  • TLDR: We propose a new method for static embeddings that can be used to improve word embeddling in low-resource and lightweight settings.
• Multimodal Sarcasm Target Identification in Tweets
  • Jiquan Wang, Lin Sun, Yi Liu, Meizhi Shao, Zengwei Zheng
  • TLDR: We propose a novel multi-scale cross-modality model that can simultaneously perform textual target labeling and visual target detection.
• Flexible Generation from Fragmentary Linguistic Input
  • Peng Qian, Roger Levy
  • TLDR: We propose a new model for novel language tasks that outperforms all existing direct-specialization models in terms of completion quality, satisfaction of syntactic constraints imposed by the input fragment, and similarity to human behavior in the structural statistics of the completions.
• Revisiting Over-Smoothness in Text to Speech
  • Yi Ren, Xu Tan, Tao Qin, Zhou Zhao, Tie-Yan Liu
  • TLDR: We study methods reducing the complexity of data distributions and improving the ability of modeling NAR-TTS models.
• Coherence boosting: When your pretrained language model is not paying enough attention
  • Nikolay Malkin, Zhen Wang, Nebojsa Jojic
  • TLDR: We present coherence boosting, an inference procedure that increases a LM’s focus on a long context.
• Uncertainty Estimation of Transformer Predictions for Misclassification Detection
  • Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgenii Tsymbalov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, Leonid Zhukov
  • TLDR: We perform a vast empirical investigation of state-of-the-art UE methods for Transformer models on misclassification detection in named entity recognition and text classification tasks and propose two computationally efficient modifications, one of which approaches or even outperforms computationally intensive methods.
• VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
  • Letitia Parcalabescu, Michele Cafagna, Lilitta Muradjan, Anette Frank, Iacer Calixto, Albert Gatt
  • TLDR: We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena.
• The Grammar-Learning Trajectories of Neural Language Models
  • Leshem Choshen, Guy Hacohen, Daphna Weinshall, Omri Abend
  • TLDR: We show that neural language models acquire linguistic phenomena in a similar order, despite their different initialization, architecture, and training data.
• Generating Scientific Definitions with Controllable Complexity
  • Tal August, Katharina Reinecke, Noah A. Smith
  • TLDR: We present a new task and dataset for defining scientific terms and controlling the complexity of generated definitions as a way of adapting to a specific reader’s background knowledge.
• Label Semantic Aware Pre-training for Few-shot Text Classification
  • Aaron Mueller, Jason Krone, Salvatore Romeo, Saab Mansour, Elman Mansimov, Yi Zhang, Dan Roth
  • TLDR: We propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems.
• ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation
  • Bei Li, Quan Du, Tao Zhou, Yi Jing, Shuhan Zhou, Xin Zeng, Tong Xiao, JingBo Zhu, Xuebo Liu, Min Zhang
  • TLDR: We show that Transformer is a residual network that can be described as a higher-order solution to ODE.
• A Comparison of Strategies for Source-Free Domain Adaptation
  • Xin Su, Yiyun Zhao, Steven Bethard
  • TLDR: We take algorithms that traditionally assume access to the source-domain training data and adapt them for source free domain adaptation.
• Ethics Sheets for AI Tasks
  • Saif Mohammad
  • TLDR: Ethics sheets for AI tasks.
• Learning Disentangled Representations of Negation and Uncertainty
  • Jake Vasilakes, Chrysoula Zerva, Makoto Miwa, Sophia Ananiadou
  • TLDR: We propose a Variational Autoencoder for representation learning that disentangles negation, uncertainty, and content.
• latent-GLAT: Glancing at Latent Variables for Parallel Text Generation
  • Yu Bao, Hao Zhou, Shujian Huang, Dongqi Wang, Lihua Qian, Xinyu Dai, Jiajun Chen, Lei Li
  • TLDR: We propose GLAT, a novel method for parallel text generation without the help of an autoregressive model.
• PPT: Pre-trained Prompt Tuning for Few-shot Learning
  • Yuxian Gu, Xu Han, Zhiyuan Liu, Minlie Huang
  • TLDR: We propose to pre-train soft prompts for language models and tune them for downstream tasks.
• Deduplicating Training Data Makes Language Models Better
  • Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini
  • TLDR: We find that existing language modeling datasets contain many near-duplicate examples and long repetitive substrings.
• Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires
  • Thong Nguyen, Andrew Yates, Ayah Zirikly, Bart Desmet, Arman Cohan
  • TLDR: We propose approaches for depression detection that are constrained to different degrees by the presence of symptoms described in PHQ9, a questionnaire used by clinicians in the depression screening process.
• Internet-Augmented Dialogue Generation
  • Mojtaba Komeili, Kurt Shuster, Jason Weston
  • TLDR: We propose a new approach to search-query based access of the internet in conversation that can generate more relevant information for conversational agents.
• SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities
  • Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy Liu, Cheng-I Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
  • TLDR: We present SUPERB-SG, a new benchmark for evaluating the robustness of pre-trained models under shifts in data domain and quality across different types of tasks.
• Knowledge Neurons in Pretrained Transformers
  • Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, Furu Wei
  • TLDR: We propose a knowledge attribution method to identify the neurons that express relational facts in pretrained Transformers.
• Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing
  • Anna Langedijk, Verna Dankers, Phillip Lippe, Sander Bos, Bryan Cardenas Guevara, Helen Yannakoudakis, Ekaterina Shutova
  • TLDR: Meta-learning with pre-training for cross-lingual dependency parsing.
• French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English
  • Aurélie Névéol, Yoann Dupont, Julien Bezançon, Karën Fort
  • TLDR: We present a multilingual dataset for social bias in language models and show that language models favor sentences that express stereotypes in most bias categories.
• Few-Shot Learning with Siamese Networks and Label Tuning
  • Thomas Müller, Guillermo Pérez-Torró, Marc Franco-Salvador
  • TLDR: We show that with proper pre-training, Siamese networks that embed texts and labels offer a competitive alternative to neural textual entailment models for zero and few-shot text classification.
• Inferring Rewards from Language in Context
  • Jessy Lin, Daniel Fried, Dan Klein, Anca Dragan
  • TLDR: We present a model that infers rewards from language pragmatically: reasoning about how speakers choose utterances not only to elicit desired actions, but also to reveal information about their preferences.
• Generating Biographies on Wikipedia: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies
  • Angela Fan, Claire Gardent
  • TLDR: We present a model for generating long-form biographies that uses a retrieval mechanism to identify relevant supporting information on the web and a cache-based pre-trained encoder-decoder to generate long-format biographies section by section, including citation information.
• Your Answer is Incorrect… Would you like to know why? Introducing a Bilingual Short Answer Feedback Dataset
  • Anna Filighera, Siddharth Parihar, Tim Steuer, Tobias Meuser, Sebastian Ochs
  • TLDR: We present a new dataset for explainable and understandable feedback systems for learners and teachers.
• Towards Better Characterization of Paraphrases
  • Timothy Liu, De Wen Soh
  • TLDR: We propose two new metrics to characterize the nature of paraphrase pairs without expert human annotation and propose improvements to the MRPC dataset.
• SummScreen: A Dataset for Abstractive Screenplay Summarization
  • Mingda Chen, Zewei Chu, Sam Wiseman, Kevin Gimpel
  • TLDR: We present a novel summarization dataset for TV series that combines TV series transcripts and human written recaps.
• Sparsifying Transformer Models with Trainable Representation Pooling
  • Michał Pietruszka, Łukasz Borchmann, Łukasz Garncarek
  • TLDR: We propose a novel method to sparsify attention in the Transformer model by learning to select the most-informative token representations during the training process, thus focusing on the task-specific parts of an input.
• Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models
  • Felix Stahlberg, Ilia Kulikov, Shankar Kumar
  • TLDR: We show that sentence-level uncertainty in neural sequence models is a major issue in NLP tasks and propose a novel exact n-best search algorithm for neural sequence model.
• FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning
  • Jing Zhou, Yanan Zheng, Jie Tang, Li Jian, Zhilin Yang
  • TLDR: We propose a novel data augmentation method that combines a generative model and a classifier to generate label-flipped data.
• Text-Free Prosody-Aware Generative Spoken Language Modeling
  • Eugene Kharitonov, Ann Lee, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu Anh Nguyen, Morgane Riviere, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu
  • TLDR: We present a new speech pre-training model that can utilize prosody to improve both prosody and content modeling, and also generate natural, meaningful, and coherent speech given a spoken prompt.
• Lite Unified Modeling for Discriminative Reading Comprehension
  • Yilin Zhao, Hai Zhao, Libin Shen, Yinggong Zhao
  • TLDR: We propose a lightweight POS-Enhanced Iterative Co-Attention Network (POI-Net) as the first attempt of unified modeling with pertinence, to handle diverse discriminative MRC tasks synchronously.
• Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining
  • Chih-chan Tien, Shane Steinert-Threlkeld
  • TLDR: We propose methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts.
• End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding
  • Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, Peng Wang, Shiliang Pu, Fei Wu
  • TLDR: We propose a novel end-to-end one-shot video grounding model for natural language spatial video grounding and learn to ground natural language in all video frames with solely one frame labeled, in an end- to-end manner.
• RNSum: A Large-Scale Dataset for Automatic Release Note Generation via Commit Logs Summarization
  • Hisashi Kamezawa, Noriki Nishida, Nobuyuki Shimizu, Takashi Miyazaki, Hideki Nakayama
  • TLDR: We present a new dataset for generating release notes and associated commit messages derived from GitHub.
• Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
  • Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Claire Cardie
  • TLDR: We propose to represent relations implicitly by situating such an argument pair in a context and use it to improve MRC.
• Modeling Persuasive Discourse to Adaptively Support Students’ Argumentative Writing
  • Thiemo Wambsganss, Christina Niklaus
  • TLDR: We introduce an argumentation annotation approach to model the structure of argumentative discourse in student-written business model pitches.
• Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons
  • Akash Kumar Mohankumar, Mitesh Khapra
  • TLDR: Recent studies have shown the advantages of evaluating NLG systems using pairwise comparisons as opposed to direct assessment.
• The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments
  • Milad Alshomary, Roxanne El Baff, Timon Gurcke, Henning Wachsmuth
  • TLDR: We propose a system that automatically generates morally framed arguments and show how their impact on different audiences is different.
• Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection
  • Xin Huang, Ashish Khetan, Rene Bidart, Zohar Karnin
  • TLDR: We present a novel solution to this problem, called Pyramid-BERT where we replace previously used heuristics with a new algorithm that can efficiently transform tokens through encoders.
• Probing for the Usage of Grammatical Number
  • Karim Lasri, Tiago Pimentel, Alessandro Lenci, Thierry Poibeau, Ryan Cotterell
  • TLDR: We propose a usage-based probing setup to find out how pre-trained models encode grammatical number, and show how it uses this encoding to solve the number agreement task.