Recovering the 3D pose of an object or human body relative to the camera is a fundamental problem in computer vision with many applications in robotics and augmented reality. Traditional methods attempted to solve this problem by aligning 3D models to 2D images with hand-crafted features, which lack robustness to appearance change and background clutter. Recently, deep learning has shown remarkable performances in feature learning. In this talk, I will summarize our recent efforts to combine geometric methods with deep learning for robust 3D pose estimation. First, I will introduce a pixel-wise voting network that robustly localizes keypoints and estimates 6DoF object pose even under severe occlusion. Next, I will discuss 3D human pose estimation and show that the lack of training data can be alleviated by using diverse supervision signals. Finally, I will present an approach for multi-person motion capture in a crowd scene with several calibrated cameras.