Machine learning often deals with Euclidean data, however many modern applications require dealing with data naturally described by other structures, such as: graphs, strings and sequences, trees, curves, probability distributions, time series and dynamical systems. We tackle the problem of dealing with structured data from a theoretical, algorithmic and practical point of view. We consider a number of applied scenarios, including inferring structure in unstructured textual content from the web, identifying subgraphs based on prior knowledge for recommender systems, forecasting the evolution of time-dependent phenomena.