Neural Networks

[!WARNING] this is a work in progress. SNNs and Surrogate gradient descent Spiking neural networks (SNNs) are biologically inspired models that compute via discrete, sparse spikes, rather than continuous activations (non differentiable non linearity). This event driven framework not only captures rich temporal patterns (such as inter spike intervals and cross neuron synchrony) but also powers energy efficient neuromorphic hardware. Surrogate gradient descent (SuGD) answer the most important challenging question: “how to convert precise spike timing into effective learning signals ?” ...

[!WARNING] This is a work in progress and may contain inaccuracies or incomplete information. Introduction RNNs are first working initiative to process sequential data by maintaining a hidden state that captures information about previous elements (some early varient of context) in the sequence. They are widely used in various applications such as language modeling, speech recognition, and time series prediction. Different types of sequential data include: Text data (e.g., sentences, documents) Time series data (e.g., stock prices, weather data) Audio data (e.g., speech signals, music) Video data (e.g., frames in a video sequence) Biological sequences (e.g., DNA, protein sequences) sequence of actions (e.g., user behavior, robot movements) Different ways to implement RNNs include: ...