Attribute Controlled Dialogue Prompting

Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks.However, both discrete prompting and continuous prompting assume fixed prompts for all data samples within a task, neglecting the fact that inputs vary greatly in some tasks such as open-domain dialogue generation.In this paper, we present a novel, instancespecific prompt-tuning algorithm for dialogue generation.Specifically, we generate prompts based on instance-level control code, rather than the conversation history, to explore their impact on controlled dialogue generation.Experiments on popular open-domain dialogue datasets, evaluated on both automated metrics and human evaluation, demonstrate that our method is superior to prompting baselines and comparable to fine-tuning with only 5%-6% of total parameters. * Work done during an internship at Huawei.overhead.We present results on both intent and persona controlled dialogue.2 Related Work GPT-3 (Brown et al., 2020) introduces prompting, a method to steer a frozen PLM by transforming inputs into cloze-style phrases with task description and some task examples.Though it is memoryefficient since one single copy of the PLM can be shared across different tasks, the model's performance is largely restricted by the maximum conditional input length, the model size and manual guesswork for prompts (Zhao et al., 2021; Schick and Schütze, 2021a,b;Jiang et al., 2020).Other works focus on automatically searching for better discrete prompts (Jiang et al., 2020;Shin et al., 2020;Gao et al., 2021;Ben-David et al., 2021).Recently, there has been an increased interest in continuous prompts / prompt-tuning, which bridges the gap between prompting and fine-tuning, while remaining efficient during training (Lester et al., 2021;Li and Liang, 2021;Liu et al., 2021Liu et al., , 2022)).Continuous prompts extend prompt selection to the entire space of embeddings, including vector embeddings that do not correspond to any humaninterpretable natural language tokens.Hence, soft prompts are more expressive than discrete prompts.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.