Abstract

Action recognition with skeleton data has recently attracted much attention in computer vision. Previous studies are mostly based on fixed skeleton graphs, only capturing local physical dependencies among joints, which may miss implicit joint correlations. To capture richer dependencies, we introduce an encoder-decoder structure, called A-link inference module, to capture action-specific latent dependencies, i.e. actional links, directly from actions. We also extend the existing skeleton graphs to represent higher-order dependencies, i.e. structural links. Combing the two types of links into a generalized skeleton graph, We further propose the actional-structural graph convolution network (AS-GCN), which stacks actional-structural graph convolution and temporal convolution as a basic building block, to learn both spatial and temporal features for action recognition. A future pose prediction head is added in parallel to the recognition head to help capture more detailed action patterns through self-supervision. We validate AS-GCN in action recognition using two skeleton data sets, NTU-RGB+D and Kinetics. The proposed AS-GCN achieves consistently large improvement compared to the state-of-the-art methods. As a side product, AS-GCN also shows promising results for future pose prediction.

Keywords

Computer scienceAction recognitionGraphConvolution (computer science)Artificial intelligencePattern recognition (psychology)Skeleton (computer programming)InferenceConvolutional neural networkTheoretical computer scienceArtificial neural network

Affiliated Institutions

Related Publications

Publication Info

Year
2019
Type
preprint
Pages
3590-3598
Citations
1197
Access
Closed

External Links

Social Impact

Social media, news, blog, policy document mentions

Citation Metrics

1197
OpenAlex

Cite This

Maosen Li, Siheng Chen, Xu Chen et al. (2019). Actional-Structural Graph Convolutional Networks for Skeleton-Based Action Recognition. , 3590-3598. https://doi.org/10.1109/cvpr.2019.00371

Identifiers

DOI
10.1109/cvpr.2019.00371