gaze  0.1.0
Perform gaze tracking with common webcams.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Groups Pages
Classes | Functions
gaze::util Namespace Reference

Helper classes within the gaze library. More...

Classes

struct  Data
 Wraps the data acquired per frame into a single instance. More...
 

Functions

dlib::rectangle get_eye_region (int eye, dlib::full_object_detection object_detection)
 
double parse_aspect_ratio (std::string aspect_ratio_string)
 
double clamp (double value, double min, double max)
 
void fill_displacement_tables (dlib::matrix< double > &table_x, dlib::matrix< double > &table_y, int size)
 
YAML::Node get_config ()
 
YAML::Node get_config (int pipeline_step_number)
 
dlib::point cv_to_dlib (const cv::Point &to_convert)
 
dlib::rectangle cv_to_dlib (const cv::Rect &to_convert)
 
cv::Rect dlib_to_cv (const dlib::rectangle &to_convert)
 
std::vector< dlib::chip_details > get_eyes_chip_details (const dlib::full_object_detection object_detection)
 
template<typename T >
void normalize_and_threshold_gradients (dlib::matrix< T > &horizontal, dlib::matrix< T > &vertical, double relative_threshold=-1)
 
std::ostreamoperator<< (std::ostream &ostr, const Data &data)
 
dlib::point cv_to_dlib (const cv::Vec3d &to_convert)
 
template<typename T , int N>
dlib::vector< T, N > cv_to_dlib (const cv::Vec< T, N > &to_convert)
 
template<typename T , int N>
cv::Vec< T, N > dlib_to_cv (const dlib::vector< T, N > &to_convert)
 

Detailed Description

Helper classes within the gaze library.

This namespace contains a data access class (util::Data).

Function Documentation

double gaze::util::clamp ( double  value,
double  min,
double  max 
)

If value is smaller than min, min is returned. If value is bigger than max, max is returned. Else, the value is returned.

Parameters
valueThe value to clamp.
minThe minimum allowed value.
maxThe maximum allowed value.
Returns
The clamped value.

Definition at line 36 of file gaze_point_calculation.cpp.

References max(), and min().

Referenced by gaze::pipeline::GazePointCalculation::process().

36  {
37  if (value < min) {
38  return min;
39  } else if (value > max) {
40  return max;
41  }
42  return value;
43 }
_Tp max() const
_Tp min() const
dlib::point gaze::util::cv_to_dlib ( const cv::Point to_convert)

Converts a cv::Point to a dlib::point.

Parameters
to_convertThe point to convert.
Returns
The converted point.

Definition at line 8 of file dlibcv.cpp.

References Point_< int >::x, and Point_< int >::y.

8  {
9  return {to_convert.x, to_convert.y};
10 }
dlib::rectangle gaze::util::cv_to_dlib ( const cv::Rect to_convert)

Converts a cv::Rect to a dlib::rectangle.

Parameters
to_convertThe rectangle to convert.
Returns
The converted rectangle.

Definition at line 12 of file dlibcv.cpp.

References cv::Rect_< _Tp >::height, cv::Rect_< _Tp >::width, cv::Rect_< _Tp >::x, and cv::Rect_< _Tp >::y.

12  {
13  return {to_convert.x, to_convert.y,
14  to_convert.width, to_convert.height};
15 }
dlib::point gaze::util::cv_to_dlib ( const cv::Vec3d to_convert)

Converts a cv::Point to a dlib::point.

Parameters
to_convertThe point to convert.
Returns
The converted point.
cv::Rect gaze::util::dlib_to_cv ( const dlib::rectangle to_convert)

Converts a dlib::rectangle to a cv::Rect.

Parameters
to_convertThe rectangle to convert.
Returns
The converted rectangle.

Definition at line 17 of file dlibcv.cpp.

17  {
18  return cv::Rect(to_convert.left(), to_convert.top(),
19  to_convert.width(), to_convert.height());
20 }
Rect2i Rect
void gaze::util::fill_displacement_tables ( dlib::matrix< double > &  table_x,
dlib::matrix< double > &  table_y,
int  size 
)

Filles the displacement table to precalculate values for vector $d_i$ Timm2011.

The function returns without modifying table_x or table_y if size is smaller than the current table size.

The size will always be set to size or size + 1, whichever is odd. This ensures that the center value is 0 as a reference point.

After a call to this function with a sufficiently big size argument, table_x and table_y contain values such that reading from both tables at the same indices yields the respective components for a unit vector geometrically pointing from the center of the matrix to the read position.

A $3 \times 3$ table_x would thus look like this:

\[ \left( \begin{array}{ccc} -0.707 & 0 & 0.707 \\ -1 & 0 & 1 \\ -0.707 & 0 & 0.707 \end{array} \right) \]

And the corresponding $3 \times 3$ table_y would be table_x's transposed version:

\[ \left( \begin{array}{ccc} -0.707 & -1 & -0.707 \\ 0 & 0 & 0 \\ 0.707 & 1 & 0.707 \end{array} \right) \]

Accessing table_x and table_y at (0, 0) yields $(-0.707, -0.707)$ which is a unit length vector pointing from the center (1, 1) to (0, 0).

Parameters
table_xThe x components of $d_i$.
table_yThe y components of $d_i$.
sizeThe size to grow this table to.

Definition at line 23 of file pupil_localization.cpp.

References std::numeric_limits< class >::min(), and std::sqrt().

Referenced by gaze::pipeline::PupilLocalization::process().

26  {
27  // Ensure size is bigger than the current size (i.e. if an update is needed
28  // at all)
29  if (size <= table_x.nr()) {
30  return;
31  }
32 
33  // Ensure size is odd
34  size += (1 - size & 1);
35  int half_size = (size - 1) / 2;
36 
37  // Resize tables
38  table_x.set_size(size, size);
39  table_y.set_size(size, size);
40 
41  // Fill with indices
42  for (int row = 0; row < table_x.nr(); ++row) {
43  dlib::set_rowm(table_x, row) = dlib::range(-half_size, half_size);
44  }
45  table_y = dlib::trans(table_x);
46 
47  // Calculate norms and store to table_x, then transpose to table_y
48  const double epsilon = std::numeric_limits<double>::min();
49  for (int row = 0; row < table_x.nr(); ++row) {
50  for (int col = 0; col < table_x.nc(); ++col) {
51  table_x(row, col) = table_x(row, col) /
52  (std::sqrt(table_x(row, col) * table_x(row, col)
53  + table_y(row, col) * table_y(row, col))
54  // Avoid div0
55  + epsilon);
56  }
57  }
58  table_y = dlib::trans(table_x);
59 }
size_t size() const
static constexpr _Tp min() noexcept
complex< _Tp > sqrt(const complex< _Tp > &)
YAML::Node gaze::util::get_config ( )

Tries to read "gaze_pipeline.yaml" or on failure "gaze_pipeline.default.yaml" and returns the resulting YAML::Node.

Returns
the yaml configuration

Definition at line 12 of file config.in.cpp.

Referenced by gaze::pipeline::FallbackStep::FallbackStep(), gaze::pipeline::HeadPoseEstimation::get_and_maybe_read_distortions(), get_config(), gaze::pipeline::HeadPoseEstimation::read_or_set_camera_matrix(), and gaze::pipeline::SourceCapture::SourceCapture().

12  {
13  YAML::Node config;
14  try {
15  config = YAML::LoadFile("gaze.yaml");
16  } catch (YAML::BadFile) {
17  config = YAML::Load("@DEFAULT_CONFIGURATION_FILE@");
18  }
19  return config;
20 }
YAML::Node gaze::util::get_config ( int  pipeline_step_number)

Calls get_config() and gets the data from

pipeline:
- ...
- ...

Where pipeline_step_number is the bullet to read.

Parameters
pipeline_step_numberthe pipeline step to read
Returns
the sub node for the respective pipeline step.

Definition at line 22 of file config.in.cpp.

References get_config().

22  {
23  return get_config()["pipeline"][pipeline_step_number];
24 }
YAML::Node get_config()
Definition: config.in.cpp:12
dlib::rectangle gaze::util::get_eye_region ( int  eye,
dlib::full_object_detection  object_detection 
)

Returns the bounding box around an eye region. Works for both, the 5 and the 68 landmarks model.

Parameters
eye0 for left eye, 1 for right eye.
object_detectionThe detected face landmarks.
Returns
the bounding box.

Definition at line 239 of file eye_like.cpp.

References get_eye_region(), one(), and rectangle().

Referenced by get_eye_region().

240  {
241  int index_ex;
242  int index_en;
243  if (object_detection.num_parts() == 5) {
244  if (eye == 0) {
245  index_ex = 0;
246  index_en = 1;
247  } else {
248  index_ex = 2;
249  index_en = 3;
250  }
251  } else {
252  if (eye == 0) {
253  index_ex = 45;
254  index_en = 42;
255  } else {
256  index_ex = 36;
257  index_en = 39;
258  }
259  }
260 
261  auto get_rectangle = [](dlib::point one, dlib::point two)
262  -> dlib::rectangle {
263  dlib::rectangle result(one, two);
264  double scale = (one - two).length() * 1.5;
265  return dlib::centered_rect(result, scale, scale);
266  };
267 
268  return get_rectangle(object_detection.part(index_ex),
269  object_detection.part(index_en));
270 }
void rectangle(InputOutputArray img, Point pt1, Point pt2, const Scalar &color, int thickness=1, int lineType=LINE_8, int shift=0)
static softfloat one()
std::vector< dlib::chip_details > gaze::util::get_eyes_chip_details ( const dlib::full_object_detection  object_detection)

Determines a bounding box around the eyes from a given object detection. This function is designed to work with 5 feature landmarks.

Parameters
object_detectionA full object detecetion as acquired from a dlib::shape_predictor.
Returns
a vector of dlib::chip_details to use with dlib::extract_image_chips.

Definition at line 12 of file pipeline_utils.cpp.

References one(), std::vector< _Tp, _Alloc >::push_back(), and rectangle().

Referenced by gaze::pipeline::GazeCapture::process(), gaze::pipeline::PupilLocalization::process(), gaze::pipeline::GazePointCalculation::process(), and gaze::pipeline::PupilLocalization::visualize().

13  {
15  if (object_detection.num_parts() < 5) {
16  return details;
17  }
18 
19  auto get_rectangle = [](dlib::point one, dlib::point two)
20  -> dlib::rectangle {
21  dlib::rectangle result(one, two);
22  double scale = (one - two).length() * 1.5;
23  return dlib::centered_rect(result, scale, scale);
24  };
25 
26  int index_ex_left;
27  int index_en_left;
28  int index_ex_right;
29  int index_en_right;
30  if (object_detection.num_parts() == 5) {
31  index_ex_left = 2;
32  index_en_left = 3;
33  index_ex_right = 0;
34  index_en_right = 1;
35  } else { // 68 landmarks
36  index_ex_left = 45;
37  index_en_left = 42;
38  index_ex_right = 36;
39  index_en_right = 39;
40  }
41 
42  details.push_back(dlib::chip_details(
43  get_rectangle(object_detection.part(index_ex_right),
44  object_detection.part(index_en_right))));
45  details.push_back(dlib::chip_details(
46  get_rectangle(object_detection.part(index_ex_left),
47  object_detection.part(index_en_left))));
48 
49  return details;
50 }
void rectangle(InputOutputArray img, Point pt1, Point pt2, const Scalar &color, int thickness=1, int lineType=LINE_8, int shift=0)
void push_back(const value_type &__x)
static softfloat one()
template<typename T >
void gaze::util::normalize_and_threshold_gradients ( dlib::matrix< T > &  horizontal,
dlib::matrix< T > &  vertical,
double  relative_threshold = -1 
)

Normalizes a horizontal and a vertical gradient matrix to unit length vectors.

If a relative threshold is provided, all values where the gradient magnitude $ M $ is lower than $ \theta \max M $ are set to 0.

Parameters
horizontalA gradient, e.g. from dlib::sobel_edge_detector
verticalA gradient, e.g. from dlib::sobel_edge_detector
relative_thresholdoptional threshold to set small values to 0.

Definition at line 79 of file pupil_localization.h.

References magnitude(), mean(), sqrt(), and threshold().

Referenced by gaze::pipeline::PupilLocalization::process().

82  {
83  dlib::matrix<T> magnitude;
84  magnitude = dlib::sqrt(dlib::squared(horizontal) +
85  dlib::squared(vertical));
86 
87  // Thresholding
88  if (relative_threshold >= 0) {
89  T threshold = dlib::mean(magnitude)
90  + dlib::stddev(magnitude) * relative_threshold;
91  for (int row = 0; row < horizontal.nr(); ++row) {
92  for (int col = 0; col < horizontal.nc(); ++col) {
93  if (magnitude(row, col) < threshold) {
94  horizontal(row, col) = 0;
95  vertical(row, col) = 0;
96  }
97  }
98  }
99  }
100 
101  // Normalization
102  horizontal = dlib::pointwise_multiply(
103  horizontal, dlib::reciprocal(magnitude));
104  vertical = dlib::pointwise_multiply(
105  vertical, dlib::reciprocal(magnitude));
106 }
Scalar mean(InputArray src, InputArray mask=noArray())
void magnitude(InputArray x, InputArray y, OutputArray magnitude)
double threshold(InputArray src, OutputArray dst, double thresh, double maxval, int type)
complex< _Tp > sqrt(const complex< _Tp > &)
std::ostream& gaze::util::operator<< ( std::ostream ostr,
const Data &  data 
)
inline

Prints a human readable short information about the data object to the stream.

Parameters
ostrthe stream.
datathe data object.
Returns
the stream.

Definition at line 130 of file data.h.

References std::endl(), gaze::util::Data::eyes, gaze::util::Data::image, gaze::util::Data::landmarks, cv::Mat::size, and gaze::util::Data::source_image.

131  {
132  ostr << "Source image size: " << data.source_image.size() << std::endl
133  << "dlib image size: " << data.image.size() << std::endl
134  << "Face: " << data.landmarks.get_rect() << std::endl
135  << "Eyes detected: " << data.eyes.size() << std::endl;
136  return ostr;
137 }
basic_ostream< _CharT, _Traits > & endl(basic_ostream< _CharT, _Traits > &__os)
double gaze::util::parse_aspect_ratio ( std::string  aspect_ratio_string)

Parses a string representing an aspect ratio. If the string contains : or /, it is split there and the aspect ratio is the quotient between the values left and right: a:b is equal to a/b. If non of the tokens is present, the string is parsed as a double.

Parameters
aspect_ratio_stringThe string to parse.
Returns
the parsed aspect ratio.

Definition at line 20 of file gaze_point_calculation.cpp.

References basic_string< char >::find(), and basic_string< char >::substr().

20  {
21  double aspect_ratio;
22  int delim = aspect_ratio_string.find(":");
23  if (delim < 0) {
24  delim = aspect_ratio_string.find("/");
25  }
26  if (delim > 0) {
27  double width_rate = std::stod(aspect_ratio_string.substr(0, delim));
28  double height_rate = std::stod(aspect_ratio_string.substr(delim + 1));
29  aspect_ratio = width_rate / height_rate;
30  } else {
31  aspect_ratio = std::stod(aspect_ratio_string);
32  }
33  return aspect_ratio;
34 }
basic_string substr(size_type __pos=0, size_type __n=npos) const
size_type find(const char *__s, size_type __pos, size_type __n) const noexcept