gaze::pipeline::GazeCapture Class Reference

Uses iTracker to track Gaze. More...

#include "gaze/pipeline_steps/gaze_capture.h"

Inheritance diagram for gaze::pipeline::GazeCapture:

Public Member Functions
void	process (util::Data &data) override
void	visualize (util::Data &data) override
Public Member Functions inherited from gaze::PipelineStep
	PipelineStep ()
std::string	get_name ()
virtual	~PipelineStep ()=default
Public Member Functions inherited from gaze::gui::Visualizeable< widget_type >
std::shared_ptr< dlib::drawable >	init (dlib::drawable_window &parent, int width, int height, std::string text="No visualization.") final
void	remove_widget () override
Public Member Functions inherited from gaze::gui::VisualizeableBase
virtual	~VisualizeableBase ()=default

Additional Inherited Members
Protected Attributes inherited from gaze::PipelineStep
int	number
std::string	name
Protected Attributes inherited from gaze::gui::Visualizeable< widget_type >
std::shared_ptr< widget_type >	widget

Detailed Description

Uses iTracker to track Gaze.

This step does nothing if not compiled using the -DWITH_CAFFE=ON option.

If it is compiled, it holds a pointer to an instance of itracker::ITracker, which it passes extracted eyes, the face, and a face mask to. It then maps the result to the target area on the screen.

Definition at line 29 of file gaze_capture.h.

Member Function Documentation

void gaze::pipeline::GazeCapture::process ( util::Data &  data )

overridevirtual

Estimates the gaze point using iTracker.

Parameters

data	The data object to be updated.

Returns

via data the modified data object.

Implements gaze::PipelineStep.

Definition at line 73 of file gaze_capture.cpp.

References gaze::util::Data::estimated_gaze_point, gaze::util::get_eyes_chip_details(), gaze::util::Data::image, gaze::util::Data::landmarks, left(), mask, right(), and toMat().

                                        {
  if (data.landmarks.num_parts() < 5) {
    return;
  }

#ifdef HAS_CAFFE
  // Gather input
  std::vector<dlib::chip_details> chips =
    util::get_eyes_chip_details(data.landmarks);

  auto right_dlib = dlib::sub_image(data.image, chips[0].rect);
  auto left_dlib = dlib::sub_image(data.image, chips[1].rect);
  auto face_dlib = dlib::sub_image(data.image, data.landmarks.get_rect());
  cv::Mat right = dlib::toMat(right_dlib);
  cv::Mat left = dlib::toMat(left_dlib);
  cv::Mat face = dlib::toMat(face_dlib);

  dlib::matrix<unsigned char> mask_in =
    dlib::zeros_matrix<unsigned char>(data.image.nr(), data.image.nc());
  dlib::set_subm(mask_in, data.landmarks.get_rect()) = 1;
  dlib::matrix<unsigned char, 25, 25> mask_out;
  dlib::resize_image(mask_in, mask_out, dlib::interpolate_nearest_neighbor());
  cv::Mat mask = dlib::toMat(mask_out);

  cv::Vec2d prediction_cam = this->itracker->predict(left, right, face, mask);
  // Note that the prediction is in cm, thus DIV100 is needed
  double x = (prediction_cam[0] / 100 + this->camera_offset[0])
           / this->screen_size_m[0] * this->target_size[0];
  double y = (-prediction_cam[1] / 100 + this->camera_offset[1])
           / this->screen_size_m[1] * this->target_size[1];
  data.estimated_gaze_point = cv::Vec2d(x, y);
#endif
}

void gaze::pipeline::GazeCapture::visualize ( util::Data &  data )

overridevirtual

Visualizes the estimated gaze point.

Parameters

data	The data object to be visualized.

Implements gaze::gui::VisualizeableBase.

Definition at line 107 of file gaze_capture.cpp.

References std::cout, std::endl(), and gaze::util::Data::estimated_gaze_point.

                                          {
#ifdef HAS_CAFFE
  std::cout << data.estimated_gaze_point << std::endl;
#endif
}

---

The documentation for this class was generated from the following files:

• include/gaze/pipeline_steps/gaze_capture.h
• src/gaze/pipeline_steps/gaze_capture.cpp