overview overview overview overview
overview overview overview overview
overview overview overview overview

Abstract

How human interact with objects depends on the functional roles of the target objects, which introduces the problem of affordance-aware hand-object interaction. It requires a large number of human demonstrations for the learning and understanding of plausible and appropriate hand-object interactions. In this work, we present AffordPose, a large-scale dataset of hand-object interactions with affordance-driven hand pose. We first annotate the specific part-level affordance labels for each object, e.g. twist, pull, handle-grasp, etc, instead of the general intents such as use or handover, to indicate the purpose and guide the localization of the hand-object interactions. The fine-grained hand-object interactions reveal the influence of hand-centered affordances on the detailed arrangement of the hand poses, yet also exhibit a certain degree of diversity. We collect a total of 26.7K hand-object interactions, each including the 3D object shape, the part-level affordance label, and the manually adjusted hand poses. The comprehensive data analysis shows the common characteristics and diversity of hand-object interactions per affordance via the parameter statistics and contacting computation. We also conduct experiments on the tasks of hand-object affordance understanding and affordance-oriented hand-object interaction generation, to validate the effectiveness of our dataset in learning the fine-grained hand-object interactions.

New!!

Video

About

Data

AffordPose dataset collects a total of about 26.7K affordance-driven hand-object interactions, involving 641 3D objects from 13 different categories and 8 types of affordance.

In addition, we also rendered 36 images(224*224) for each interaction data.

Dataset License

AffordPose Dataset Copyright License is CC BY-NC-ND 4.0 license.

Additional Notes:

Our Data Copyright License for non-commercial scientific research purposes including rendered images, hand information, object information and segmentations.

Without Author1's or Author2's prior written permission, any use for commercial purposes is prohibited, including, without limitation, incorporation in a commercial product, use in a commercial service, production of other artifacts for commercial purposes, or training methods/algorithms/neural networks/etc. for commercial use of any kind.

Data Download

  • Download the AffordPose datasets.7z(855M) or Rendered images.rar(7.29G). You can download specific categories or all the data according to your needs. The data are following structure: 
    └── AffordPose
            ├──bottle
            │   ├──3415
            │   │   ├──3415_Twist
            │   │   │   ├── 1.json
            │   │   │   ├── ...
            │   │   │   └── 28.json
            │   │   │
            │   │   └──3415_Wrap-grasp
            │   │       ├── 1.json
            │   │       ├── ...
            │   │       └── 28.json
            |   |
            |   └── ...
            |
            └── ...

  • A .json file contains the following information: 
    ├── xxx.json
       ├── rhand_mesh            # the hand mesh
       ├── dofs                  # the joint configurations of the hand
       ├── rhand_trans           # the translation of the paml
       ├── rhand_quat            # the rotation of the paml
       ├── object_mesh           # the object mesh, and the verts are annotated with affordance label
       ├── trans_obj             # with the default value: (0,0,0)
       ├── quat_obj              # with the default value: (1,0,0,0)
       ├── afford_name           # the object affordance corresponding to the interaction
       └── class_name            # the object class

    • Here's a breakdown of the relationships between the affordance numbers and corresponding name:
    1: Handle-grasp, 3: Press, 4: Lift, 5: Wrap-grasp, 6: Twist, 7: Support, 8: Pull, and 9: Lever.

    Acknowledgements

    If you find our work useful in your research, please cite: 
                  
@InProceedings{Jian_2023_ICCV,
  author    = {Jian, Juntao and Liu, Xiuping and Li, Manyi and Hu, Ruizhen and Liu, Jian},
  title     = {AffordPose: A Large-Scale Dataset of Hand-Object Interactions with Affordance-Driven Hand Pose},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month     = {October},
  year      = {2023},
  pages     = {14713-14724}
}


    © Juntao Jian 2023. The website template was borrowed from OakInk.