Graphics, Imaging, and Games Lab

  • Home
  • About Us
  • Contact
  • Login
Home

From the Lab

  • Benchmark
  • Projects
  • Publications
  • Image Galleries
  • Blogs
  • Lab Members
  • Prospective Students
  • Search

Predictive and Generative Neural Networks for Object Functionality

Ruizhen Hu
Zihao Yan
Jingwen Zhang
Oliver van Kaick
Ariel Shamir
Hao Zhang
Hui Huang
Predictive and Generative Neural Networks for Object Functionality. Trans. on Graphics, 2018.
Journal Article

Related Projects
High­level Shape Representations for Content Creation

Download the PDF

Abstract

Humans can predict the functionality of an object even without any surroundings, since their knowledge and experience would allow them to "hallucinate" the interaction or usage scenarios involving the object. We develop predictive and generative deep convolutional neural networks to replicate this feat. Specifically, our work focuses on functionalities of man-made 3D objects characterized by human-object or object-object interactions. Our networks are trained on a database of scene contexts, called interaction contexts, each consisting of a central object and one or more surrounding objects, that represent object functionalities. Given a 3D object in isolation, our functional similarity network (fSIM-NET), a variation of the triplet network, is trained to predict the functionality of the object by inferring functionality-revealing interaction contexts involving the object. fSIM-NET is complemented by a generative network (iGEN-NET) and a segmentation network (iSEG-NET). iGEN-NET takes a single voxelized 3D object and synthesizes a voxelized surround, i.e., the interaction context which visually demonstrates the object's functionalities. iSEG-NET separates the interacting objects into different groups according to their interaction types.


BibTeX

@article{hu18iconnet,
author = {Ruizhen Hu and Zihao Yan and Jingwen Zhang and Oliver van Kaick and Ariel Shamir and Hao Zhang and Hui Huang},
title = {Predictive and Generative Neural Networks for Object Functionality},
journal = {ACM Trans. on Graphics (Proc. SIGGRAPH)},
volume = {to appear},
year = 2018,
}
 

  • Home
  • About Us
  • Contact
  • Login