AI Governance: A Research Agenda (2018) by Allan Dafoe. Centre for the Governabnce AI, Future of Humanity Institute, University of Oxford. Eye on the prize ya’ll This review is filled with measured calls to ideation sweeping across a host of AI ethical concerns, yet the sentiment that rings out as most primary is the call to vision stated thus, “AI ideal governance aspires to envision, blueprint, and advance ideal institutional solutions for humanity’s AI governance challenges.

NLP vs. NLU vs. NLG: the differences between three natural language processing concepts. Eda Kavlakoglu (IBM Watson Blog, 2020). Terms Natural Langauge Processing (NLP) takes unstructured data and converts it into structued data that machines can use to complete tasks for humans. Natural Languaage Understanding (NLU) uses syntactic (how words are arranged to create well-formed sentences) and semantic (the meaning conveyed) analysis of text and speech to determine the meaning of sentences.

A growing neural gas network learns topologies by Bernd Fritzke (1995). Overview Hebbian learning variations for learning topologies in high-dimensional data. Key concepts topological learning: Given some high-dimensional data distribution, find a topological structure which closely reflects the topology of the data distribution. competitive Hebbian learning (CHL): For each input signal x connect the two closest centers (measured by Euclidean distance) by an edge. vector quantization (VC): Often used for data compression, quantization involves dividing a large set of points (vectors) into groups having approximatley the same number of points closest to them.

The central question this paper asks is “how should we distribute our ML programs over a cluster?“. For some of us, the answer is “Spark”, and we call it a day. But if we aren’t satisified with the current state of performance, and would like to, instead, lean into the performance edge between ML models and systems, and explore the possibilities of improved performance, then we need to dig a little further.

DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning by Min Du, Feifei Li, Guineng Zheng, & Vivek Srikumar (2017). Overview Inspired by natural language modeling, the authors present a deep learning approach to anomaly detection in system logs. Key concepts system logs: A file that records the system states and significant events for debugging software. syslog: The system log standard maintained by Internet Engineering Task Force (IETF).

Jessica Taylor, Eliezer Yudkowsky, Patrick LaVictoire, and Andrew Critch. Machine Intelligence Research Institute. Alignment in this context means making sure agents arrive at and optimize objective functions that are in the spirit of what was intended; that is that goals are reached while making sure no one gets hurt. One of the key takeaways from this overview is that our solutions must scale with intelligence, so for any new discovery, how long will it “hold” in lock step with advances in intelligence?

Adam: A Method for Stochastic Optimization (2015). Diederik P. Kingma and Jimmy Lei Ba. Conference paper at ICLR 2015. Overview This method computes individual adaptive learning rates for different parameters from estimates of first and second moments of the gradients. Combines AdaGrad (which works well with sparse gradients) and RMSProp (works well in on-line and non-stationary settings.

Introduction to Gaussian Processes (1998) - David Mackay Overview “From a Bayesian perspective, a choice of a neural network model can be viewed as defining a prior probability distribution over non-linear functions, and the neural network’s learning process can be interpreted in terms of the posterior probability distribution over the unknown function. (Some learning algorithms search for the function with maximum posterior probability, and other Monte Carlo methods draw samples from this posterior probability).

Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016). Shahriari et al. Proceedings of the IEEE Introduction “Mathematically we are considering the problem of finding a global maximizer (or minimizer) of an unknown objective function f, where X is some design space of interest; …” ”…in global optimization, X is often a compact subset of R^d but the Bayesian optimization framework can be applied to more unusual search spaces that involve categorical or conditional inputs.

Dušan Teodorovic. Swarm Intelligence systems for transportation engineering: principles and applications. Transportation Research, 2008. This is one of the most comprehensive articles I have found that attempts to connect swarm intelligence heuristics to transportation systems. Most of the article is dedicated to introducing four multi-agent systems that leverage information sharing models to optimize search techniques, and I will recap those here. Ant Colony Optimization (ACO) Ants leave pheromone trails, and an ant will use the strength of the signal to weight their choice of path, as well-trod paths traveled by ants heading to a food source have a stronger pheromone signal.