Affective computing（一）

（一）Bhattacharya S, Bhattacharya S, Chang S F. Predicting Viewer Perceived Emotions in Animated GIFs[C]// ACM International Conference on Multimedia. ACM, 2014:213-216.

When a media sample is presented to human subjects, their perceived emotion is the emotion that they think the sample expresses instead of the emotion they feel, which is otherwise called their induced emotion.Perceived emotion is an important phenomena to study because it is more concrete and objective than induced emotion, where labels are less reliable due to their subjectivity.
感知情绪是受试者认为被呈现样本表现出的情绪，诱导情绪是受试者看到被呈现样本时产生的情绪。
感知情绪：客观明确；诱导情绪：主观不可靠

提取以下四类特征：
Color histograms：compute frame-level color histograms in
HSV color space for its classical use in vision and affect computing
Face expression：The training set of consisted of 28,708 48 × 48 face images over seven emotions: angry, disgust, fear, happy, sad, surprise and neutral. We perform face detection using OpenCV’s Haar-like cascade and apply facial expression recognition on the largest face for a 6-D vector of SVM score outputs as a feature
Image-based aesthetics：GIF frames are divided into 3×3 cells from which cell-level statistics are computed including the dark channel, luminosity, sharpness, symmetry, white balance, colorfulness, color harmony, and eye sensitivity.Together these
form a 149-D feature vector for each frame
SentiBank： the representation is a set of a 1,200 D Linear SVM outputs where the SVMs are trained using a taxonomy of “adjective noun pairs” (ANPs)

分类方式——multi-tasking learning (MTL), also known as multitask
regression (MTR), 解决情绪之间的相关问题（如惊讶和害怕）
our goal is to learn the weight matrix W comprised of t tasks via the optimization min W{ L(W) + Ω(W) }, where L(W) is the empirical training loss and Ω(W)is a regularization encoding task-relatedness
对情绪之间的相关性建模——constrain regressors of different emotions to share a low dimensional subspace
共享低维子空间意思是具有低秩的权重矩阵
min W { L(W)+λ ·rank(W) } 将这个NP难问题化为：

一些结论：所有特征中，脸部表情分类能力最强；分类效果最好的表情是happiness和amusement，最差的是embarrassment，原因是快乐的情绪会有脸部笑容，而尴尬的情绪会捂住脸。

公开数据集：
DEAP dataset
MAHNOB-HCI Tagging dataset
FilmStim
LIRIS-ACCEDE
International Affective Picture System (IAPS)

---

（二）Analysis of EEG Signals and Facial Expressions
for Continuous Emotion Detection

The goal of this work was to detect emotions continuously
from EEG signals and facial expressions

---

（三）Categorical and dimensional affect analysis in continuous input: Current trends and future directions

• Introduction: continuous input

1.连续情感信号遇到的问题：The first challenge is the requirement for soft real-time processing and responsiveness
2.连续情感信号处理的目标：this special issue focuses on affect analysis in continuous input and aims at discussing the issues and the challenges pertinent in sensing, recognizing and responding to continuous human affective behavior from diverse communicative cues and modalities.

• Affect representation

3.对比categorical方法和dimensional方法：In the categorical approach, where each affective display is classified into a single category, complex mental/affective states or blended emotions may be too difficult to handle；
in the dimensional approach, emotion transitions can be easily captured, and observers can indicate their impression of moderate and naturalistic emotional expressions on several continuous scales.
4.一些dimensional的方法：
Circumplex of Affect：把基本的情感表征成双极性——arousal (relaxed vs. aroused) and valence (pleasant vs. unpleasant)
the PAD emotion space （emotional primitives）：the 3D-emotional space of pleasure–displeasure, arousal–non-arousal, and dominance–submissiveness
（To guarantee a more complete description of affective coloring, some researchers include expectation as the fourth dimension, and intensity as the fifth dimension）
5.情绪的相关性问题需要进一步探究
6.评价理论：In the appraisal-based approach emotions are generated through continuous, recursive subjective evaluation of both our own internal state and the state of the outside world (relevant concerns/needs)
This approach divides the emotion recognition process into two mappings: expressive features to appraisal variables, and appraisal variables to emotion label.——将appraisal variables当做中间过渡层

• Affect analysis in continuous input: a generalized view

7.连续信号和非连续信号的区别：In non-continuous input, data is usually segmented such that it is constrained to contain one affective event (e.g.,head nod/shake), expression (e.g., smile) or affective state (e.g., pain) with beginning and end.
In continuous input instead the automatic analyzer ultimately needs to determine the starting and ending times of affective events, expressions and affective states (the segmentation)
也就是在the window size or the unit of analysis 之中进行判断

8.时间窗大小问题：the challenge for future research is to find an appropriate unit of analysis.

• Modalities and cues

9.Bio signals的种类：galvanic skin response that is used as an indication of a person’s level of physiological arousal electromyography (the electrical potential mostly originated from muscular cell activities), that is correlated with negatively valenced emotions；heart rate that increases with negatively valenced emotions such as fear；heart rate variability that indicates a state of relaxation or mental stress, and respiration rate (how deep and fast the breath is) that becomes irregular with more aroused emotions like anger or fear；amygdala；there exists a correlation between increased blood perfusion in the orbital muscles and stress levels for human beings
10.视觉信号/语音信号