Review of MIT’s WHY!

Conceptual Contributions

It is an interesting and challenging question to resolve how we determine the “why” behind an image. It is posited that it is a result of “Theory of the Mind” and psychophysics researchers hypothesize that our capacity to reliably infer another person’s motivation stems from our ability to impute our own beliefs to others, that is, if we experience a certain emotion doing something, we infer that others experience that emotion as well. This paper seeks to computationally deduce the motivation behind people’s actions in images. This problem is challenging in many regards as it is…

Conceptual Contributions

In DeViSE, the authors tackle the issue that visual recognition systems are often limited in their ability to scale to large numbers of classification categories in part due to difficulty in acquiring such a balanced dataset as well as the traditionally rigid nature of classification within defined classes. The authors propose a new deep visual-semantic embedding model (DeViSE) to use text data to train visual models and to constrain their predictions. DeViSE leverages both labeled image data as well as unannotateed text data. The model uses the textual data to leaern a. semantic relationship between the labels of the image…

Review of this article

Conceptual Contributions

This paper aims to improve image captioning, which is one of the primary challenges in the intersection of CV and NLP. Tangible process improves applications from aiding visually impaired users to human computer interaction. Current state of the art captioning models typically include an end-to-end large neural network where images are passed to a CNN as a vector and then captions are retrieved from an output vector using a Recurrent Neural Network.

This paper introduces a novel framework for image captioning that is restricted to the set of objects that the model is able to detect…

Semantic interpretation, or making inferences on the meaning of atext is a fundamental step in language understanding. Drawing from our human ability to envisionmental images when prompted by a description, this paper introduces a novel approach for automaticallydenoting similarities between descriptions of everyday situations via sets of images and captions detailedmore below. Existing approaches include standard distributional approaches to lexical similarity, which is the idea that “linguistic expressions that appear in similar contexts have a similar meaning.” …

An Overview of Segmenting Scenes by Matching Image Composites

Introduction

One of the major goals in computer vision is semantic object segmentation, which is essentially identifying different objects that have semantic meaning (toothbrush or tree) within an image. This is a difficult endeavor because objects in semantic classifications take on a variety of forms and can rely on context spread throughout an image.

Beyond basic methods such as canny edge detection, where we look for a large difference (in intensity) between a series of neighboring pixels, we wish to identify entire objects. This presents technical challenges such as how do we…

Convolutional Neural Networks (CNNs), which is a class of deep learning neural networks, have become the go-to method for 2D image detection/classification as it produce accurate results without taking too much computing power or time. You can find out more about how it works here and the original motivation, which was inspired by human brain activity here. However, with the increased popularity of self driving cars, omnidirectional images, and other 3D maps (such as wind maps, drones, temperature maps, etc.) we arrive at the challenge of conducting image recognition on spherical images.

Note: This paper was a bit difficult for me and I relied heavily on other sources to help me understand the material. A lot of the mechanisms and why’s of res nets I still don’t understand to the point where I feel confident explaining it to others, so I’m going to keep this one short and as a brief overview.

In this review, I will discuss the motivations for this work, their solution and how the residual network helps without going into details much. If you would like a more detailed explanation of the exact mechanisms and procedure, I recommend…

A Review of Joseph Halpern’s A Modification of the Halpern-Pearl Definition of Causality

Waves: An Introduction

Just as the waves ebb and flow, the principles of causality, cause and effect push and pull the world around us into the way that it is. Despite the fact that causality in principle is something that is astoundingly intuitive and simple to understand, defining it precisely and mathematically turns out to be quite a challenge. Bothering philosophers and logicians alike for millennia (way back to Aristotle), defining causality in a rigid sense can be traced to Hume in the1700s and more recently, to Joseph Halpern and…