nharrel commited on
Commit
aaabbbe
·
verified ·
1 Parent(s): ddc7180

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -5,3 +5,21 @@ In this study, we employ the Microsoft DeBERTa v3 model, which introduces an add
5
  To model this threshold effectively, we opted for the tanh activation function, which provides a more appropriate representation. Consequently, we implemented an MSE loss function with tanh activation, followed by rounding to the nearest integer for evaluation purposes.
6
 
7
  Utilizing this approach, we demonstrated improvements in regression tasks for evaluating stances on each test scenario. While the overall MSE did not show significant improvement, we observed higher accuracy, recall, and precision for the regression tasks. It is important to note that the classification task specified in Qiu et al. (2022) solely determines the presence or absence of the value in question, without considering the specific stance presented in the text. Therefore, our regression task, which assesses the particular stance, should not be directly compared with the classification task from Qiu et al. (2022).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  To model this threshold effectively, we opted for the tanh activation function, which provides a more appropriate representation. Consequently, we implemented an MSE loss function with tanh activation, followed by rounding to the nearest integer for evaluation purposes.
6
 
7
  Utilizing this approach, we demonstrated improvements in regression tasks for evaluating stances on each test scenario. While the overall MSE did not show significant improvement, we observed higher accuracy, recall, and precision for the regression tasks. It is important to note that the classification task specified in Qiu et al. (2022) solely determines the presence or absence of the value in question, without considering the specific stance presented in the text. Therefore, our regression task, which assesses the particular stance, should not be directly compared with the classification task from Qiu et al. (2022).
8
+
9
+ ## Acknowledgements
10
+
11
+ We would like to acknowledge the authors of the ValueNet dataset for their valuable contribution to this work.
12
+
13
+ @article{Qiu_Zhao_Li_Lu_Peng_Gao_Zhu_2022,
14
+ title={ValueNet: A New Dataset for Human Value Driven Dialogue System},
15
+ volume={36},
16
+ url={https://ojs.aaai.org/index.php/AAAI/article/view/21368},
17
+ DOI={10.1609/aaai.v36i10.21368},
18
+ abstractNote={Building a socially intelligent agent involves many challenges, one of which is to teach the agent to speak guided by its value like a human. However, value-driven chatbots are still understudied in the area of dialogue systems. Most existing datasets focus on commonsense reasoning or social norm modeling. In this work, we present a new large-scale human value dataset called ValueNet, which contains human attitudes on 21,374 text scenarios. The dataset is organized in ten dimensions that conform to the basic human value theory in intercultural research. We further develop a Transformer-based value regression model on ValueNet to learn the utility distribution. Comprehensive empirical results show that the learned value model could benefit a wide range of dialogue tasks. For example, by teaching a generative agent with reinforcement learning and the rewards from the value model, our method attains state-of-the-art performance on the personalized dialog generation dataset: Persona-Chat. With values as additional features, existing emotion recognition models enable capturing rich human emotions in the context, which further improves the empathetic response generation performance in the EmpatheticDialogues dataset. To the best of our knowledge, ValueNet is the first large-scale text dataset for human value modeling, and we are the first one trying to incorporate a value model into emotionally intelligent dialogue systems. The dataset is available at https://liang-qiu.github.io/ValueNet/.},
19
+ number={10},
20
+ journal={Proceedings of the AAAI Conference on Artificial Intelligence},
21
+ author={Qiu, Liang and Zhao, Yizhou and Li, Jinchao and Lu, Pan and Peng, Baolin and Gao, Jianfeng and Zhu, Song-Chun},
22
+ year={2022},
23
+ month={Jun.},
24
+ pages={11183-11191}
25
+ }