Academic News----Natural Language Processing Group

NLP Group has 1 paper accepted by AAAI 2024

In December 2023, NLP group has one paper accepted by AAAI 2024.In December 2023, NLP group has one paper accepted by AAAI 2024. The full name of AAAI 2024 is the Thirty-Eighth AAAI Conference on Artificial Intelligence, which is one of the top conferences in artificial intelligence. It is supported annually by the AAAI, the Association for the Advancement of Artificial Intelligence. AAAI 2024 will be held in Vancouver, Canada from February 20 to February 27, 2024. The accepted paper is summarized as follows: - TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling (Longxiang Liu, Xiuxing Li, Yang Feng)- AAAI Main Conference, long paperAbstract: Task-oriented dialog systems have witnessed substantial progress due to conversational pre-training techniques. Yet, two significant challenges persist. First, most systems primarily utilize the latest turn's state label for the generator. This practice overlooks the comprehensive value of state labels in boosting the model's understanding for future generations. Second, an overreliance on generated policy often leads to error accumulation, resulting in suboptimal responses when adhering to incorrect actions. To combat these challenges, we propose turn-level multi-task objectives for the encoder. With the guidance of essential information from labeled intermediate states, we establish a more robust representation for both understanding and generation. For the decoder, we introduce an action tree-based scheduled sampling technique. Specifically, inspired by SPACE, we model the hierarchical policy as trees and utilize the similarity between trees to sample negative policy based on scheduled sampling, hoping the model to generate invariant responses under perturbations. This method simulates potential pitfalls by sampling similar negative policy, bridging the gap between task-oriented dialog training and inference. Among methods without continual pre-training, our approach achieved state-of-the-art (SOTA) performance on the MultiWOZ dataset series and was also competitive with pre-trained SOTA methods.

2023-12-20

NLP Group Has 6 Papers Accepted by EMNLP 2023

Congratulations to NLP group for having six papers accepted by EMNLP 2023, including 2 papers by the main conference and 4 paper by Findings of EMNLP!Congratulations to NLP group for having six papers accepted by EMNLP 2023, including 2 papers by the main conference and 4 paper by Findings of EMNLP! The full name of EMNLP 2023 is the 2023 Conference on Empirical Methods in Natural Language Processing. EMNLP is organized by the ACL special interest group on linguistic data (SIGDAT). It is held once a year and is one of the most influential international conferences in the field of natural language processing. EMNLP 2023 will be held in Singapore from Dec 6 to Dec 10, 2023. The accepted papers are summarized as follows: - Non-autoregressive Streaming Transformer for Simultaneous Translation (Zhengrui Ma, Shaolei Zhang, Shoutao Guo, Chenze Shao, Min Zhang, Yang Feng). - Accepted by Main Conference.Abstract: Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while maintaining low latency often leads to a tendency for aggressive anticipation. We argue that such issue stems from the autoregressive architecture upon which most existing SiMT models are built. To address those issues, we propose non-autoregressive streaming Transformer (NAST) which comprises a unidirectional encoder and a non-autoregressive decoder with intra-chunk parallelism. We enable NAST to generate blank token or repetitive tokens to adjust its READ/WRITE strategy flexibly, and train it to maximize the non-monotonic latent alignment with an alignment-based latency loss. Experiments on various SiMT benchmarks demonstrate that NAST outperforms previous strong autoregressive SiMT baselines. - Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation (Wenyu Guo, Qingkai Fang, Dong Yu, Yang Feng).- Accepted by Main Conference.Abstract: Multimodal machine translation (MMT) simultaneously takes the source sentence and a relevant image as input for translation. Since there is no paired image available for the input sentence in most cases, recent studies suggest utilizing powerful text-to-image generation models to provide image inputs. Nevertheless, synthetic images generated by these models often follow different distributions compared to authentic images. Consequently, using authentic images for training and synthetic images for inference can introduce a distribution shift, resulting in performance degradation during inference. To tackle this challenge, in this paper, we feed synthetic and authentic images to the MMT model, respectively. Then we minimize the gap between the synthetic and authentic images by drawing close the input image representations of the Transformer Encoder and the output distributions of the Transformer Decoder. Therefore, we mitigate the distribution disparity introduced by the synthetic images during inference, thereby freeing the authentic images from the inference process. Experimental results show that our approach achieves state-of-the-art performance on the Multi30K En-De and En-Fr datasets, while remaining independent of authentic images during inference. - Scaling Law for Document-Level Neural Machine Translation (Zhang Zhuocheng, Shuhao Gu, Min zhang, Yang Feng).- Accepted by Findings of EMNLP. Abstract: The scaling laws of language models have played a significant role in advancing large language models. In order to promote the development of document translation, we systematically examine the scaling laws in this field. In this paper, we carry out an in-depth analysis of the influence of three factors on translation quality: model scale, data scale, and sequence length. Our findings reveal that increasing sequence length effectively enhances model performance when model size is limited. However, sequence length cannot be infinitely extended; it must be suitably aligned with the model scale and corpus volume. Further research shows that providing adequate context can effectively enhance the translation quality of a document's initial portion. Nonetheless, exposure bias remains the primary factor hindering further improvement in translation quality for the latter half of the document. - Addressing the Length Bias Challenge in Document-Level Neural Machine Translation (Zhang Zhuocheng, Shuhao Gu, Min zhang, Yang Feng).- Accepted by Findings of EMNLP. Abstract: Document-level neural machine translation (DNMT) has shown promising results by incorporating context information through increased maximum lengths of source and target sentences. However, this approach also introduces a length bias problem, whereby DNMT suffers from significant translation quality degradation when decoding sentences that are much shorter or longer than the maximum sentence length during training, i.e., the length bias problem. To prevent the model from neglecting shorter sentences, we sample the training data to ensure a more uniform distribution across different sentence lengths while progressively increasing the maximum sentence length during training. Additionally, we introduce a length-normalized attention mechanism to aid the model in focusing on target information, mitigating the issue of attention divergence when processing longer sentences. Furthermore, during the decoding stage of DNMT, we propose a sliding decoding strategy that limits the length of target sentences to not exceed the maximum length encountered during training. The experimental results indicate that our method can achieve state-of-the-art results on several open datasets, and further analysis shows that our method can significantly alleviate the length bias problem. - Enhancing Neural Machine Translation with Semantic Units (Langlin Huang, Shuhao Gu, Zhuocheng Zhang, Yang Feng*).- Accepted by Findings of EMNLP. Abstract: Conventional neural machine translation (NMT) models typically use subwords and words as the basic units for model input and comprehension. However, complete words and phrases composed of several tokens are often the fundamental units for expressing semantics, referred to as semantic units. To address this issue, we propose to recover the integral meanings of semantic units within a sentence, which are then leveraged to provide a new perspective for understanding the sentence. Specifically, we first propose Word Pair Encoding (WPE), a phrase extraction method to help identify the boundaries of semantic units. Next, we design an Attentive Semantic Fusion (ASF) layer to integrate the semantics of multiple subwords into a single vector: the semantic unit representation. Lastly, the semantic-unit-level sentence representation is concatenated to the token-level one, and they are combined together as the input to the encoder. Experimental results demonstrate that our method effectively models and leverages semantic-unit-level information and outperforms the strong baselines. - Simultaneous Machine Translation with Tailored Reference (Shoutao Guo, Shaolei Zhang, Yang Feng).- Accepted by Findings of EMNLP. Abstract: Simultaneous machine translation (SiMT) generates translation while reading the whole source sentence. However, existing SiMT models are typically trained using the same reference disregarding the varying amounts of available source information at different latency. Training the model with ground-truth at low latency introduces forced anticipations, whereas utilizing reference consistent with the source word order at high latency results in performance degradation. Consequently, it is crucial to train the SiMT model with the reference that avoids forced anticipations during training while maintaining high quality. In this paper, we propose a novel method that provides tailored reference for the SiMT models trained at different latency by rephrasing the ground-truth. Specifically, we introduce the tailor, induced by reinforcement learning, to modify ground-truth to the tailored reference. The SiMT model is trained with the tailored reference and jointly optimized with the tailor to enhance performance. Importantly, our method is applicable to a wide range of current SiMT approaches. Experiments on three tasks demonstrate that our method achieves state-of-the-art performance in fixed and adaptive policies.

2023-10-24

NLP Group Has 4 Papers Accepted by NeurIPS 2023

Congratulations to NLP group for having four papers accepted by NeurIPS 2023!Congratulations to NLP group for having four papers accepted by NeurIPS 2023! The full name of NeurIPS 2023 is the Thirty-seventh Conference on Neural Information Processing Systems, which is one of the top conferences in the field of artificial intelligence. NeurIPS is currently ranked 10th in Google Scholar's ranking of academic conferences/journals. NeurIPS 2023 will be held in New Orleans, LA from Dec 10 to Dec 16, 2023. The accepted papers are summarized as follows: - Unified Segment-to-Segment Framework for Simultaneous Sequence Generation (Shaolei Zhang, Yang Feng) .- Accepted by NeurIPS 2023. Long Paper. Abstract: Simultaneous sequence generation is a pivotal task for real-time scenarios, where the target sequence is generated while receiving the source sequence. The crux of achieving high-quality generation with low latency lies in identifying the optimal moments for generating, accomplished by learning a mapping between the source and target sequences. However, existing methods often rely on task-specific heuristics for different sequence types, limiting the model’s capacity to adaptively learn the source-target mapping and hindering the exploration of multi-task learning for various simultaneous tasks. In this paper, we propose a unified segment-to-segment framework (Seg2Seg) for simultaneous sequence generation, which learns the mapping in an adaptive and unified manner. During the process of simultaneous generation, the model alternates between waiting for a source segment and generating a target segment, making the segment serve as the natural bridge between the source and target. To accomplish this, Seg2Seg introduces a latent segment as the pivot between source to target and explore all potential source-target mappings via the proposed expectation training, thereby learning the optimal moments for generating. Experiments on multiple simultaneous generation tasks, including streaming speech recognition, simultaneous machine translation and simultaneous speech translation, demonstrate that Seg2Seg achieves state-of-the-art performance and exhibits better generality across various simultaneous generation tasks. - DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation（Qingkai Fang, Yan Zhou, Yang Feng) . - Accepted by NeurIPS 2023. Long Paper. Abstract: Direct speech-to-speech translation (S2ST) directly translates speech of the source language into the target language, which can break the communication barriers between different language groups and has broad application prospects. However, due to the presence of linguistic and acoustic diversity, the target speech follows a complex multimodal distribution, posing challenges to achieving both high-quality translations and fast decoding speeds for S2ST models. In this paper, we propose DASpeech, a non-autoregressive direct S2ST model which realizes both fast and high-quality S2ST. DASpeech adopts the two-pass architecture to decompose the generation process into two steps, where a DA-Transformer model first generates the target text, and a FastSpeech 2 model then generates the target speech based on the hidden states of the DA-Transformer. To consider all potential paths in the DAG during training, we calculate the expected hidden states for each target token via dynamic programming. Experiments on the CVSS benchmark demonstrate that DASpeech can achieve comparable or even better performance than the state-of-the-art S2ST model Translatotron 2, while preserving up to 18.53x speedup compared to the autoregressive baseline model. Compared with the previous non-autoregressive S2ST model, DASpeech does not rely on knowledge distillation and iterative decoding, achieving significant improvements in both translation quality and decoding speed. Furthermore, DASpeech shows the ability to preserve the speaker’s voice of the source speech during translation. - Beyond MLE: Convex Loss for Text Generation（Chenze Shao*, Zhengrui Ma*, Min Zhang, Yang Feng）. - Accepted by NeurIPS 2023. Long Paper. Abstract: Maximum likelihood estimation (MLE) is a statistical method used to estimate the parameters of a probability distribution that best explain the observed data. In the context of text generation, MLE is often used to train generative language models, which can then be used to generate new text. However, we argue that MLE is not always necessary and optimal, especially for closed-ended text generation tasks like machine translation. In these tasks, the goal of model is to generate the most appropriate response, which does not necessarily require it to estimate the entire data distribution with MLE. To this end, we propose a novel class of training objectives based on convex functions, which enables text generation models to focus on highly probable outputs without having to estimate the entire data distribution. We investigate the theoretical properties of the optimal predicted distribution when applying convex functions to the loss, demonstrating that convex functions can sharpen the optimal distribution, thereby enabling the model to better capture outputs with high probabilities. Experiments on various text generation tasks and models show the effectiveness of our approach. Specifically, it enables autoregressive models to bridge the gap between greedy and beam search, and facilitates the learning of non-autoregressive models with a maximum improvement of 9+ BLEU points. - Non-autoregressive Machine Translation with Probabilistic Context-free Grammar（Shangtong Gui, Chenze Shao, Zhengrui Ma, Xishan Zhang, Yunji Chen, Yang Feng）.- Accepted by NeurIPS 2023. Long Paper. Abstract: Non-autoregressive Transformer(NAT) significantly accelerates the inference of neural machine translation. However, conventional NAT models suffer from limited-expression power and performance degradation compared to autoregressive (AT) models due to the assumption of conditional independence among target tokens. To address these limitations, we propose a novel approach called PCFG-NAT, which leverages a specially designed Probabilistic Context-Free Grammar (PCFG) to enhance the ability of NAT models to capture complex dependencies among output tokens. Experimental results on major machine translation benchmarks demonstrate that PCFG-NAT further narrows the gap in translation quality between NAT and AT models. Moreover, PCFG-NAT facilitates a deeper understanding of the generated sentences, addressing the lack of satisfactory explainability in neural machine translation.

2023-09-30

NLP Group Has 5 Papers Accepted by ACL 2023

Congratulations to NLP group for having five papers accepted by ACL 2023, including 4 papers by the main conference and 1 paper by Findings of ACL!Congratulations to NLP group for having five papers accepted by ACL 2023, including 4 papers by the main conference and 1 paper by Findings of ACL! ACL stands for The Association for Computational Linguistics, which is the most influential academic organization in the international computational linguistics community. The annual ACL conference is also the most important international conference in the field of computational linguistics and the only CCF-A conference in the field of computational linguistics. The main conference papers of ACL will be published in Proceedings of ACL, while Findings of ACL is an online affiliated publication introduced since ACL 2021. ACL 2023 will be held in Toronto, Canada from July 9 to July 14, 2023. The accepted papers are summarized as follows: - Understanding and Bridging the Modality Gap for Speech Translation (Qingkai Fang, Yang Feng) . - Accepted by ACL 2023 Main Conference. Long Paper. Abstract: Speech translation (ST) translates input speech directly into text in another language, effectively reducing the communication barrier between people of different languages. Due to the scarcity of ST corpus and the difficulty of learning the mapping from source speech to target text, researchers usually introduce the machine translation (MT) task to assist ST training. However, due to the modality gap between speech and text, the performance of ST usually lags behind MT. In this paper, we first measure the significance of the modality gap based on the decoder representation differences, and we find that (1) the exposure bias problem causes the modality gap to increase gradually with time steps during inference, and (2) the modality gap has a long-tail problem, i.e., there are a few cases where the modality gap is very large. To solve these problems, we propose Cross-modal Regularization with Scheduled Sampling (CRESS), which simulates the decoding situation through scheduled sampling during training, and introduces cross-modal regularization loss to reduce the prediction difference between ST and MT, so that the predictions of the two tasks in the decoding stage are more consistent. Experimental results show that our method achieves significant improvement in all eight directions of the ST benchmark dataset MuST-C, reaching the current state-of-the-art performance. - Back Translation for Speech-to-text Translation Without Transcripts（Qingkai Fang, Yang Feng). - Accepted by ACL 2023 Main Conference. Long Paper. Abstract: Speech translation (ST) usually faces the challenge of data scarcity, and most of the existing approaches utilize additional automatic speech recognition (ASR) data or machine translation (MT) data during training to enhance the ST model. However, it is estimated that there are about 3000 languages in the world without corresponding transcribed text. For ST tasks from these languages to other languages, there is no ASR or MT data to exploit. In this paper, we aim to utilize the large-scale monolingual corpus of the target language to enhance ST in the absence of source transcripts. Inspired by the back translation (BT) approach in MT, we aim to design a back translation approach for ST that can synthesize a pseudo-ST corpus from the target monolingual corpus without relying on source transcripts. The generation from target text to source speech is challenging due to the generation from short sequences to long sequences and the existence of a one-to-many mapping between them. For this reason, we introduce discrete units as an intermediate representation. We first translate the target text to a sequence of discrete units corresponding to the source speech by a sequence-to-sequence model, and then convert it to the corresponding waveform by a vocoder. Experiments on the MuST-C dataset show that our method is able to synthesize high-quality pseudo-ST data from target monolingual data, significantly improving the performance of the baseline model. - Learning Optimal Policy for Simultaneous Machine Translationvia Binary Search（Shoutao Guo, Shaolei Zhang, Yang Feng）. - Accepted by ACL 2023 Main Conference. Long Paper. Abstract: Simultaneous machine translation (SiMT) starts to output the generated translation before reading the whole source sentence and needs a precise policy to decide when to output the generated translation. However, there is no golden policy corresponding to parallel sentences as explicit supervision, which often leads to a lack of source information or the introduction of additional delays. In this paper, we propose the SiMT method based on binary search (BS-SiMT), which utilizes binary search to construct the optimal translation policy online and trains the SiMT model accordingly. Then the SiMT model can learn the optimal policy with explicit supervision and complete translation according to the learned policy during inference. Experiments on multiple translation tasks have shown that our method can exceed strong baselines under all latency and the learned policy is more accurate. - CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation（Yan Zhou, Qingkai Fang, Yang Feng）.- Accepted by ACL 2023 Main Conference. Long Paper. Abstract: End-to-end speech translation (ST) is the task of translating speech signals in the source language into text in the target language. As a cross-modal task, end-to-end ST is difficult to train with limited data. Existing methods often try to transfer knowledge from machine translation (MT), but their performances are restricted by the modality gap between speech and text. In this paper, we propose Cross-modal Mixup via Optimal Transport (CMOT) to overcome the modality gap. We find the alignment between speech and text sequences via optimal transport, and then mix up the sequences from different modalities at a token level using the alignment. Experiments on MuST-C ST benchmark demonstrate that CMOT achieves an average BLEU of 30.0 in 8 translation directions, outperforming previous methods. Further analysis shows CMOT can adaptively find the alignment between modalities, which helps alleviate the modality gap between speech and text. - End-to-End Simultaneous Speech Translation with Differentiable Segmentation（Shaolei Zhang, Yang Feng）. - Accepted by Findings of ACL 2023. Long Paper. Abstract: End-to-end simultaneous translation (ST) outputs translation while receiving the streaming speech inputs, and hence needs to segment the speech inputs and then translate based on the current received speech. Segmenting at those unfavorable moments will break the acoustic integrity and further drop the performance of the translation model. Therefore, learning to segment the speech inputs at those moments that are beneficial for the translation model to produce high-quality translation is the key to ST. Existing ST methods, either using the fixed-length segmentation or external segmentation model, always separate segmentation from the underlying translation model, where the gap makes the segmentation results not necessarily beneficial to the translation. In this paper, we propose Differentiable Segmentation (DiSeg) for ST to directly learn segmentation from the underlying translation model. DiSeg turns hard segmentation into differentiable through the proposed expectation training, enabling it to be jointly trained with the translation model and thereby learn the translation-beneficial segmentation. Experiments on two ST benchmarks show that DiSeg outperforms strong baselines and achieves state-of-the-art performance.

2023-06-03

NLP's 2 papers are accepted by ICLR 2023

Congratulations! NLP's 2 papers are accepted by ICLR 2023!Congratulations! NLP's 2 papers are accepted by ICLR 2023! The full name of ICLR 2023 is the 11th International Conference on Learning Representations. ICLR was founded by Yoshua Bengio and Yann LeCun, two of the three giants of deep learning, and has been held annually since 2013. It is one of the top conferences in the field of deep learning. ICLR is currently ranked 9th in Google Scholar's ranking of academic conferences/journals. ICLR 2023 will be held in Kigali, Rwanda from May 1 to May 5, 2023.The accepted paper are summarized as follows: 1. Title: Hidden Markov Transformer for Simultaneous Machine Translation Authors: Shaolei Zhang, Yang FengAbstract: Simultaneous machine translation (SiMT) outputs the target sequence while receiving the source sequence, and hence learning when to start translating each target token is the core challenge for SiMT. However, it is non-trivial to learn the optimal moment among many possible moments of starting translating, as the moments of starting translating always hide inside the model and we can only supervise the SiMT model with the observed target sequence. In this paper, we propose Hidden Markov Transformer (HMT), which treats the moments of starting translating as hidden events and the target sequence as the corresponding observed events, thereby organizing them as a hidden Markov model. HMT explicitly models multiple moments of starting translating, used as the candidate hidden events, and then selects one to generate the target token. During training, by maximizing the marginal likelihood of the target sequence over multiple moments of starting translating, HMT learns to start translating at the moments that target tokens can be generated more accurately. Experiments on multiple SiMT benchmarks show that HMT outperforms strong baselines and achieves state-of-the-art performance. 2. Title: Fuzzy Alignments in Directed Acyclic Graph for Non-autoregressive Machine Translation Authors: Zhengrui Ma, Chenze Shao, Shangtong Gui, Min Zhang and Yang FengAbstract: Non-autoregressive translation (NAT) reduces the decoding latency but suffers from performance degradation due to the multi-modality problem. Recently, the structure of Directed Acyclic Graph has achieved great success in NAT, which tackles the multi-modality problem by introducing dependency between vertices. However, training it with Negative Log Likelihood loss implicitly requires a strict alignment between reference tokens and vertices, weakening its ability to handle multiple translation modalities. In this paper, we hold the view that all paths in the graph are fuzzily aligned with the reference sentence. We do not require the exact alignment but train the model to maximize a fuzzy alignment score between the graph and reference, which takes captured translations in all modalities into account. Extensive experiments on major WMT benchmarks show that our method substantially improves translation performance and increases prediction confidence, setting a new state of the art for NAT on the raw training data.

2023-03-06

NLP's NAT turtorial is accepted by EMNLP 2022

In December 2022, the lab's tutorial workshop entitled"Non-Autoregressive Models for Fast Sequence Generation"was accepted at the EMNLP 2022 conference.And looks forward to the future development direction of non-autoregressive generation.In December 2022, the lab's tutorial workshop entitled "Non-Autoregressive Models for Fast Sequence Generation" was accepted at the EMNLP 2022 conference.This three-hour workshop was conducted by researcher Yang Feng and Chenze Shao to talk about the latest research progress of non-autoregressive sequence generation models. Non-autoregressive sequence generation model refers to the model of parallel decoding to generate the entire sequence, which can significantly accelerate the speed of sequence generation, and has attracted widespread attention in the fields of machine translation, speech recognition, speech synthesis and so on.This workshop comprehensively expounds the multi-modal challenges faced by non-autoregressive models in sequence generation and the current mainstream solutions, such as knowledge distillation, enhancing expression ability, modeling hidden variables, improving training objectives, improving decoding strategies, etc., and introduces in detail the progress of non-autoregressive models in various sequence generation tasks and the commonalities and differences of its application in different tasks, and looks forward to the future development direction of non-autoregressive generation.

2022-12-17

NLP's 8 papers are accepted by EMNLP 2022

Congratulations! In October 2022, NLP's eight papers are accepted by EMNLP 2022, including 4 main conference papers and 4 findings papers! The full name of EMNLP 2022 is the Conference on Empirical Methods in Natural Language Processing, organized by SIGDAT, a division of the International Association for Computational Linguistics (ACL), and held annually as one of the most influential international conferences in the field of natural language processing.Congratulations! In October 2022, NLP's eight papers are accepted by EMNLP 2022, including 4 main conference papers and 4 findings papers! The full name of EMNLP 2022 is the Conference on Empirical Methods in Natural Language Processing, organized by SIGDAT, a division of the International Association for Computational Linguistics (ACL), and held annually as one of the most influential international conferences in the field of natural language processing. The accepted papers are summarized as follows:1. Counterfactual Data Augmentation via Perspective Transition for Open-Domain Dialogues (Jiao Ou, Jinchao Zhang, Yang Feng, Jie Zhou).Accepted by Main Conference.Abstract: The construction of open-domain dialogue systems requires high-quality dialogue datasets. The dialogue data admits a wide variety of responses for a given dialogue history, especially responses with different semantics. However, collecting high-quality such a dataset in most scenarios is labor-intensive and time- consuming. In this paper, we propose a data augmentation method to automatically augment high-quality responses with different semantics by counterfactual inference. Specifically, given an observed dialogue, our counterfactual generation model first infers semantically different responses by replacing the observed reply perspective with substituted ones. Furthermore, our data selection method filters out detrimental augmented responses. Experimental results show that our data augmentation method can augment high-quality responses with different semantics for a given dialogue history, and can outperform competitive baselines on multiple downstream tasks.2. Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions (Shuhao Gu, Bojie Hu, Yang Feng).Accepted by Main Conference.Abstract: This paper considers continual learning of large-scale pretrained neural machine translation model without accessing the previous training data or introducing model separation. We argue that the widely used regularization-based methods, which perform multi-objective learning with an auxiliary loss, suffer from the misestimate problem and cannot always achieve a good balance between the previous and new tasks. To solve the problem, we propose a two-stage training method based on the local features of the real loss. We first search low forgetting risk regions, where the model can retain the performance on the previous task as the parameters are updated, to avoid the catastrophic forgetting problem. Then we can continually train the model within this region only with the new training data to fit the new task. Specifically, we propose two methods to search the low forgetting risk regions, which are based on the curvature of loss and the impacts of the parameters on the model output, respectively. We conduct experiments on domain adaptation and more challenging language adaptation tasks, and the experimental results show that our method can achieve significant improvements compared with several strong baselines. 3. Information-Transport-based Policy for Simultaneous Translation (Shaolei Zhang, Yang Feng).Accepted by Main Conference.Abstract: Simultaneous translation (ST) outputs translation while receiving the source inputs, and hence requires a policy to determine whether to translate a target token or wait for the next source token. The major challenge of ST is that each target token can only be translated based on the current received source tokens, where the received source information will directly affect the translation quality. So naturally, how much source information is received for the translation of the current target token is supposed to be the pivotal evidence for the ST policy to decide between translating and waiting. In this paper, we treat the translation as information transport from source to target and accordingly propose an Information-Transport-based Simultaneous Translation (ITST). ITST quantifies the transported information weight from each source token to the current target token, and then decides whether to translate the target token according to its accumulated received information. Experiments on both text-to-text ST and speech-to-text ST (a.k.a., streaming speech translation) tasks show that ITST outperforms strong baselines and achieves state-of-the-art performance. 4. Low-resource Neural Machine Translation with Cross-modal Alignment (Zhe Yang, Qingkai Fang, Yang Feng).Accepted by Main Conference.Abstract: For low-resource neural machine translation, existing techniques often rely on large-scale monolingual corpora, which is impractical for some low-resource languages. In this paper, we turn to connect several low-resource languages to a particular high-resource one by additional visual modality. Specifically, we propose a cross-modal contrastive learning method to learn a shared space for all languages, where both a coarsegrained sentence-level objective and a finegrained token-level one are introduced. Experimental results and further analysis show that our method can effectively learn the crossmodal and cross-lingual alignment with a small amount of image-text pairs and achieves significant improvements over the text-only baseline under both zero-shot and few-shot scenarios. 5. Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappings (Shuhao Gu, Yang Feng).Accepted by Findings of EMNLP.Abstract: The many-to-many multilingual neural machine translation can translate between language pairs unseen during training, i.e., zero-shot translation. Improving zero-shot translation requires the model to learn universal representations and cross-mapping relationships to transfer the knowledge learned on the supervised directions to the zero-shot directions. In this work, we propose the state mover’s distance based on the optimal theory to model the difference of the representations output by the encoder. Then, we bridge the gap between the semantic-equivalent representations of different languages at the token level by minimizing the proposed distance to learn universal representations. Besides, we propose an agreement-based training scheme, which can help the model make consistent predictions based on the semantic-equivalent sentences to learn universal cross-mapping relationships for all translation directions. The experimental results on diverse multilingual datasets show that our method can improve consistently com- pared with the baseline system and other contrast methods. The analysis proves that our method can better align the semantic space and improve the prediction consistency. 6. Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation (Chenze Shao, Zhengrui Ma, Yang Feng).Accepted by Findings of EMNLP.Abstract: Non-autoregressive models achieve significant decoding speedup in neural machine translation but lack the ability to capture sequential dependency. Directed Acyclic Transformer (DA-Transformer) was recently proposed to model sequential dependency with a directed acyclic graph. Consequently, it has to apply a sequential decision process at inference time, which harms the global translation accuracy. In this paper, we present a Viterbi decoding framework for DA-Transformer, which guarantees to find the joint optimal solution for the translation and decoding path under any length constraint. Experimental results demonstrate that our approach consistently improves the performance of DA-Transformer while maintaining a similar decoding speedup.7. Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation (Shaolei Zhang, Shoutao Guo, Yang Feng).Accepted by Findings of EMNLP.Abstract: Simultaneous machine translation (SiMT) outputs the translation while receiving the source inputs, and hence needs to balance the received source information and translated target information to make a reasonable decision between waiting for inputs or outputting translation. Previous methods always balance source and target information at the token level, either directly waiting for a fixed number of tokens or adjusting the waiting based on the current token. In this paper, we propose a Wait-info Policy to balance source and target at the information level. We first quantify the amount of information contained in each token, named info. Then during simultaneous translation, the decision of waiting or outputting is made based on the comparison results between the total info of previous target outputs and received source inputs. Experiments show that our method outperforms strong baselines under and achieves better balance via the proposed info. 8. Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation (Shoutao Guo, Shaolei Zhang, Yang Feng).Accepted by Findings of EMNLP.Abstract: Simultaneous machine translation starts translation before the whole source sentence is read, which adopts fixed policy or adaptive policy to obtain better trade-offs between latency and translation quality. The previous methods rely too much on the decision module of the READ/WRITE action, which writes the generated token immediately after deciding on WRITE action, but this will inevitably lead to incorrect actions. This paper introduces rationality evaluation into the READ/WRITE policy, which uses the change of source information to evaluate the rationality of the READ/WRITE action before executing the decided action, and then executes the corresponding action accordingly. Therefore, it can reduce unreasonable operations and obtaining a better balance between translation quality and latency.

2022-12-12

NLP's 1 paper is accepted by AAAI 2023

As we train NAT based on the rephraser output rather than the reference sentence, the rephraser output should fit well with the NAT output and not deviate too far from the reference.
　　Congratulations! NLP's one paper is accepted by AAAI 2023! The full name of AAAI 2023 is the Thirty-Seventh AAAI Conference on Artificial Intelligence, which is one of the top conferences in artificial intelligence. It is supported annually by the AAAI, the Association for the Advancement of Artificial Intelligence. AAAI 2023 will be held in Washington, D.C., USA from February 7 to February 14, 2023.
　　The accepted paper is summarized as follows:
　　- Rephrasing the Reference for Non-Autoregressive Machine Translation (Chenze Shao, Jinchao Zhang, Jie Zhou, Yang Feng)
　　- AAAI Main Conference, long paper
　　Abstract: Non-autoregressive neural machine translation (NAT) significantly improves the decoding speed by generating all tokens in parallel. The major challenge that limits the NAT performance is the multi-modality problem that there may exist multiple possible translations of a source sentence. Therefore, the reference sentence may be inappropriate for the training when the NAT output is closer to other translations. In response to this problem, we introduce a rephraser to provide a better training target for NAT by rephrasing the reference sentence according to the NAT output. As we train NAT based on the rephraser output rather than the reference sentence, the rephraser output should fit well with the NAT output and not deviate too far from the reference, which can be quantified as reward functions and optimized by reinforcement learning. Experiments on major WMT benchmarks and NAT baselines show that our approach consistently improves the translation quality of NAT. Specifically, our best variant achieves comparable performance to the autoregressive Transformer, while being 14.7 times more efficient in inference.
　　

2022-12-12

NLP's 1 paper is accepted by NeurIPS 2022

Congratulations! NLP's one paper is accepted by NeurIPS 2022! The full name of NeurIPS 2022 is the Thirty-sixth Conference on Neural Information Processing Systems, which is one of the top conferences in artificial intelligence.
　　Congratulations! NLP's one paper is accepted by NeurIPS 2022!
　　The full name of NeurIPS 2022 is the Thirty-sixth Conference on Neural Information Processing Systems, which is one of the top conferences in artificial intelligence. NeurIPS 2022 will be held in New Orleans, USA, from November 28 to December 9, 2022.
　　The accepted paper is summarized as follows:
　　 - Exploring Non-Monotonic Latent Alignments for Non-Autoregressive Machine Translation (Chenze Shao, Yang Feng)
　　 - NeurIPS Main Conference, long paper
　　

2022-12-02

NLP's 1 paper is accepted by NAACL 2022

Congratulations! In April 2022, NLP's one paper is accepted by NAACL 2022! The full name of NAACL 2022 is the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL-HLT 2022), which is the North American Chapter of ACL, one of the top conferences in the field of natural language processing. NAACL 2022 will be held in Seattle, USA, July 10-15, 2022.Congratulations! In April 2022, NLP's one paper is accepted by NAACL 2022! The full name of NAACL 2022 is the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL-HLT 2022), which is the North American Chapter of ACL, one of the top conferences in the field of natural language processing. NAACL 2022 will be held in Seattle, USA, July 10-15, 2022. The accepted paper is summarized as follows: - One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation (Chenze Shao, Xuanfu Wu, Yang Feng)- NAACL Main Conference, long paperAbstract: Non-autoregressive neural machine translation (NAT) suffers from the multi-modality problem: the source sentence may have multiple correct translations, but the loss function is calculated only according to the reference sentence. Sequence-level knowledge distillation makes the target more deterministic by replacing the target with the output from an autoregressive model. However, the multi-modality problem in the distilled dataset is still nonnegligible. Furthermore, learning from a specific teacher limits the upper bound of the model capability, restricting the potential of NAT models. In this paper, we argue that one reference is not enough and propose diverse distillation with reference selection (DDRS) for NAT. Specifically, we first propose a method called SeedDiv for diverse machine translation, which enables us to generate a dataset containing multiple high-quality reference translations for each source sentence. During the training, we compare the NAT output with all references and select the one that best fits the NAT output to train the model. Experiments on widely-used machine translation benchmarks demonstrate the effectiveness of DDRS, which achieves 29.82 BLEU with only one decoding pass on WMT14 En-De, improving the state-of-the-art performance for NAT by over 1 BLEU.

2022-11-29