Grow with Data

𝗪𝗵𝗮𝘁 𝗮𝗿𝗲 𝗦𝘂𝗽𝗽𝗼𝗿𝘁 𝗩𝗲𝗰𝘁𝗼𝗿𝘀 𝗶𝗻 𝗦𝗩𝗠? • Support vectors are the data points nearest to the hyperplane, the points of a data set that, if removed, would alter the position of the dividing hyperplane. • Using these support vectors, we maximize the margin of the classifier. • For computing predictions, only the support vectors are used.

𝗪𝗵𝗮𝘁 𝗮𝗿𝗲 𝘀𝗼𝗺𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀 𝗼𝗳 𝗧𝗶𝗺𝗲-𝗦𝗲𝗿𝗶𝗲𝘀 𝗗𝗮𝘁𝗮 𝘄𝗵𝗶𝗰𝗵 𝗰𝗮𝗻 𝗯𝗲 𝗺𝗶𝗻𝗲𝗱? • 𝗦𝗲𝗻𝘀𝗼𝗿 𝗱𝗮𝘁𝗮: Sensor data is often collected by a wide variety of hardware and other monitoring devices. Typically, this data contains continuous readings about the underlying data objects. For example, environmental data is commonly collected with different kinds of sensors that measure temperature, pressure, humidity, and so on. Sensor data is the most common form of time series data. • 𝗠𝗲𝗱𝗶𝗰𝗮𝗹 𝗱𝗲𝘃𝗶𝗰𝗲𝘀: Many medical devices such as an electrocardiogram (ECG) and electroencephalogram (EEG) produce continuous streams of time series data. These represent measurements of the functioning of the human body, such as the heartbeat, pulse rate, blood pressure, etc. Real-time data is also collected from patients in intensive care units (ICU) to monitor their condition. • 𝗙𝗶𝗻𝗮𝗻𝗰𝗶𝗮𝗹 𝗺𝗮𝗿𝗸𝗲𝘁 𝗱𝗮𝘁𝗮: Financial data, such as stock prices, is often temporal. Other forms of temporal data include commodity prices, industrial trends, and economic indicators.

𝗛𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝗵𝗮𝗻𝗱𝗹𝗲 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗩𝗮𝗹𝘂𝗲𝘀 𝗶𝗻 𝗧𝗶𝗺𝗲-𝗦𝗲𝗿𝗶𝗲𝘀 𝗗𝗮𝘁𝗮? • It is common for time series data to contain missing values. Furthermore, the values of the series may not be synchronized in time when they are collected by independent sensors. • The most common methodology used for handling missing, unequally spaced, or unsynchronized values is linear interpolation. • The idea is to create estimated values at the desired time stamps. These can be used to generate multivariate time series that are synchronized, equally spaced, and have no missing values.

𝗛𝗼𝘄 𝗶𝘀 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗽𝗿𝗶𝗻𝗰𝗶𝗽𝗹𝗲 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁 𝗮𝘅𝗶𝘀 𝘀𝗲𝗹𝗲𝗰𝘁𝗲𝗱 𝗶𝗻 𝗣𝗖𝗔? In Principal Component Analysis (PCA) we look to summarize a large set of 𝗰𝗼𝗿𝗿𝗲𝗹𝗮𝘁𝗲𝗱 𝘃𝗮𝗿𝗶𝗮𝗯𝗹𝗲𝘀 (basically a high dimensional data) into a smaller number of representative variables, called the 𝗽𝗿𝗶𝗻𝗰𝗶𝗽𝗮𝗹 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀, that explains most of the variability in the original set. The 𝗳𝗶𝗿𝘀𝘁 𝗽𝗿𝗶𝗻𝗰𝗶𝗽𝗮𝗹 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁 axis is selected in a way such that it 𝗲𝘅𝗽𝗹𝗮𝗶𝗻𝘀 𝗺𝗼𝘀𝘁 of the variation in the data and is closest to all n observations.

𝗛𝗼𝘄 𝘄𝗲 𝗰𝗮𝗻 𝘂𝘀𝗲 𝗣𝗖𝗔 𝗳𝗼𝗿 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝘀𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻? Feature selection refers to 𝗰𝗵𝗼𝗼𝘀𝗶𝗻𝗴 𝗮 𝘀𝘂𝗯𝘀𝗲𝘁 of the features from the complete set of features. In PCA, we obtain 𝗣𝗿𝗶𝗻𝗰𝗶𝗽𝗮𝗹 𝗖𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀 𝗮𝘅𝗶𝘀, this is a 𝗹𝗶𝗻𝗲𝗮𝗿 𝗰𝗼𝗺𝗯𝗶𝗻𝗮𝘁𝗶𝗼𝗻 of all the 𝗼𝗿𝗶𝗴𝗶𝗻𝗮𝗹 𝘀𝗲𝘁 of feature variables which defines a new set of axes that explain most of the 𝘃𝗮𝗿𝗶𝗮𝘁𝗶𝗼𝗻𝘀 in the data. Therefore while PCA performs well in many practical settings, it does not result in the development of a model that relies upon a 𝘀𝗺𝗮𝗹𝗹 𝘀𝗲𝘁 of the original features and so for this reason, 𝗣𝗖𝗔 𝗶𝘀 𝗻𝗼𝘁 𝗮 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝘀𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝘁𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲.

𝗪𝗵𝗮𝘁 𝗶𝘀 𝘁𝗵𝗲 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝗮 𝗠𝘂𝗹𝘁𝗶𝗰𝗹𝗮𝘀𝘀 𝗽𝗿𝗼𝗯𝗹𝗲𝗺 𝗮𝗻𝗱 𝗮 𝗠𝘂𝗹𝘁𝗶𝗹𝗮𝗯𝗲𝗹 𝗽𝗿𝗼𝗯𝗹𝗲𝗺? • 𝗠𝘂𝗹𝘁𝗶𝗰𝗹𝗮𝘀𝘀 𝗰𝗹𝗮𝘀𝘀𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 means a classification task with more than two classes; e.g., classify a set of images of fruits which may be oranges, apples, or pears. Multiclass classification makes the assumption that each sample is assigned to one and only one label: a fruit can be either an apple or a pear but not both at the same time. • 𝗠𝘂𝗹𝘁𝗶𝗹𝗮𝗯𝗲𝗹 𝗰𝗹𝗮𝘀𝘀𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 assigns to each sample a set of target labels. This can be thought of as predicting properties of a data-point that are not mutually exclusive, such as topics that are relevant for a document. A text might be about any of religion, politics, finance or education at the same time or none of these.

𝗚𝗶𝘃𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀 𝗼𝗳 𝘂𝘀𝗶𝗻𝗴 𝗖𝗹𝘂𝘀𝘁𝗲𝗿𝗶𝗻𝗴 𝘁𝗼 𝘀𝗼𝗹𝘃𝗲 𝗿𝗲𝗮𝗹 𝗹𝗶𝗳𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀. • 𝗜𝗱𝗲𝗻𝘁𝗶𝗳𝘆𝗶𝗻𝗴 𝗰𝗮𝗻𝗰𝗲𝗿𝗼𝘂𝘀 𝗱𝗮𝘁𝗮: Initially we take known samples of a cancerous and non-cancerous dataset, and label both the samples dataset. Then both the samples are mixed and different clustering algorithms are applied to the mixed samples dataset. It has been found through experiments that a cancerous dataset gives the best results with unsupervised non-linear clustering algorithms. • 𝗦𝗲𝗮𝗿𝗰𝗵 𝗲𝗻𝗴𝗶𝗻𝗲𝘀: Search engines try to group similar objects in one cluster and the dissimilar objects far from each other. It provides results for the searched data according to the nearest similar object which is clustered around the data to be searched.

𝗛𝗼𝘄 𝘁𝗼 𝗸𝗻𝗼𝘄 𝘄𝗵𝗲𝘁𝗵𝗲𝗿 𝘆𝗼𝘂𝗿 𝗺𝗼𝗱𝗲𝗹 𝗶𝘀 𝘀𝘂𝗳𝗳𝗲𝗿𝗶𝗻𝗴 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺 𝗼𝗳 𝘃𝗮𝗻𝗶𝘀𝗵𝗶𝗻𝗴 𝗚𝗿𝗮𝗱𝗶𝗲𝗻𝘁𝘀? • The model will improve very slowly during the training phase and it is also possible that training stops very early, meaning that any further training does not improve the model. • The weights closer to the output layer of the model would witness more of a change whereas the layers that occur closer to the input layer would not change much (if at all). • Model weights shrink exponentially and become very small when training the model. • The model weights become 0 in the training phase.

𝗛𝗼𝘄 𝘄𝗼𝘂𝗹𝗱 𝘆𝗼𝘂 𝗰𝗵𝗼𝗼𝘀𝗲 𝘁𝗵𝗲 𝗔𝗰𝘁𝗶𝘃𝗮𝘁𝗶𝗼𝗻 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻 𝗳𝗼𝗿 𝗮 𝗗𝗲𝗲𝗽 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹? • If the output to be predicted is real, then it makes sense to use a 𝗟𝗶𝗻𝗲𝗮𝗿 𝗔𝗰𝘁𝗶𝘃𝗮𝘁𝗶𝗼𝗻 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻. • If the output to be predicted is a probability of a binary class, then a 𝗦𝗶𝗴𝗺𝗼𝗶𝗱 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻 should be used. • If the output to be predicted has two classes, then a 𝗧𝗮𝗻𝗵 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻 can be used. • 𝗥𝗲𝗟𝗨 𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻 can be used in many different cases due to its computational simplicity.

𝗪𝗵𝗮𝘁 𝗮𝗿𝗲 𝗘𝗻𝘀𝗲𝗺𝗯𝗹𝗲 𝗺𝗲𝘁𝗵𝗼𝗱𝘀 𝗮𝗻𝗱 𝗵𝗼𝘄 𝗮𝗿𝗲 𝘁𝗵𝗲𝘆 𝘂𝘀𝗲𝗳𝘂𝗹 𝗶𝗻 𝗗𝗲𝗲𝗽 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴? • Ensemble methods are used to increase the 𝗴𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗽𝗼𝘄𝗲𝗿 of a model. These methods are applicable to both deep learning as well as machine learning algorithms. • Some ensemble methods introduced in neural networks are 𝗗𝗿𝗼𝗽𝗼𝘂𝘁 and 𝗗𝗿𝗼𝗽𝗰𝗼𝗻𝗻𝗲𝗰𝘁. The improvement in the model depends on the type of data and the nature of neural architecture.