Session Co-Chairs:
Fu-Hai Frank Wu, National Tsing Hua University, Taiwan
Session description:
Recent years, the deep learning(DL) techniques have applied to audio(speech, music, sound etc.) and video(including image). There are differences and commons for these two fields in term of input data type, usually the video is in raw formats (mostly RGB pixel data) and the audio in pre-processed format s, for example spectrogram). Besides, the focus of data augmentation, including synthetic data, strategies are different to be effective in the training phase. In the respective of DL architecture, convolution kernel sizes and pooling strategies are generally distinct. The kernel size of audio DL is rectangular with the long-end in time axis and the other is square due to the import localized characteristic of image in fully convolutional network. The countermeasure beside the rectangular kernel size for the long-term characteristic of audio could be the adoption of recurrent network, for example long-short term memory(LSTM). The DL for audio and video is a broad topic, besides the discriminant problems we mention, it is obvious that tons of issues could be addressed. We also welcome the research of cross-domain inter-activities, although mostly audio IR borrow the DL outcome from the computer vision.
The special session will gather the researchers in the field of DL for audio information retrieval and computer vision to share the research progress, new finding , and state-of-the-art algorithms . We hope to explore and enumerate the common methods could be shared by studying the individual field. We expect to inspire and foster cross-domain improvement and increase the multi-modality research.
The topics of interest include, but are not limited to:
music lyrics and other textual data, web mining, and natural language processing
multi-modality
corpus creation
musical rhythm, beat, tempo
optical music recognition
text in scene
music synthesis and transformation
automatic classification
indexing and querying
pattern matching and detection
human-computer interaction
gaming
action recognition
recognition, detection, categorization,indexing
segmentation, grouping and shape representation