10. Support Vector Machines

Here, support vector machines will be used only to classify objects which can be categorized into one of exactly two classes. As with other classification and regression methods, support vector machines as a method can be used more generally. However, we will work to understand the mathematical derivation of the binary-classification SVM.

Author

Johanna Hardin

Published

November 11, 2024

Artwork by @allison_horst.

Agenda

November 11, 2024

  1. linearly separable
  2. dot products
  3. support vector formulation

November 13, 2024

  1. not linearly separable (SVM)
  2. kernels (SVM)
  3. support vector formulation

Readings

Reflection questions

  • How is an SVM built (how do we find the model)?

  • Why is it often advantageous to transform the data into a higher dimensional space?

  • What is the kernel trick and how is it related to the SVM decision rule?

  • Can SVMs work on data that are not linearly separable (even in high dimensions)? How?

  • What are the advantages of the SVM algorithm?

  • What are the disadvantages of the SVM algorithm?

Ethics considerations

  • What type of feature engineering is required for Support Vector Machines?

  • Do Support Vector Machines produce a closed form “model” that can be written down or visualized and handed to a client?

  • If the model produces near perfect predictions on the test data, what are some potential concerns about putting that model into production?

Slides

Additional Resources

:::