We know that we can measure a model’s robustness to perturbation by applying perturbation to training points and checking if the outputs are the same:

The lp ball around an image is said to be the adversarial ball, and a network is said to be E-robust around x if every point in the adversarial ball around x classifies the same. source, Part 3

But how is this done concretely?

An excerpt from Thomas Dietterich’s recent blog post:

In order for a system to understand, it must create linkages between different concepts, states, and actions. Today’s language translation systems correctly link “water” in English to “agua” in Spanish, but they don’t have any links between “water” and “electric shock”.

Much of the criticism of the latest AI advances stems from two sources. First, the hype surrounding AI (generated by researchers, the organizations they work for, and even governments and funding agencies) has reached extreme levels. It has even engendered fear that “superintelligence” or the “robot apocalypse” is imminent. Criticism is essential for countering this nonsense.

Second, criticism is part of the ongoing debate about future research directions in artificial intelligence research and to allocation of government funding. On the one side are the advocates of connectionism who developed deep learning and who support continuing that line of research. On the other side are the advocates of AI methods based on the construction and manipulation of symbols (e.g., using formal logic). There is also a growing community arguing for systems that combine both approaches in a hybrid architecture. Criticism is also essential for this discussion, because the AI community must continually challenge our assumptions and choose how to invest society’s time and money in advancing AI science and technology. However, I object to the argument that says “Today’s deep learning-based systems don’t exhibit genuine understanding, and therefore deep learning should be abandoned”. This argument is just as faulty as the argument that says “Today’s deep learning-based systems have achieved great advances, and pursuing them further will `solve intelligence’.” I like the analysis by Lakatos (1978) that research programmes tend to be pursued until they cease to be fruitful. I think we should continue to pursue the connectionist programme, the symbolic representationalist programme, and the emerging hybrid programmes, because they all continue to be very fruitful.

Criticism of deep learning is already leading to new directions. In particular, the demonstration that deep learning systems can match human performance on various benchmark tasks and yet fail to generalize to superficially very similar tasks has produced a crisis in machine learning (in the sense of Kuhn, 1962). Researchers are responding with new ideas such as learning invariants (Arjovsky, et al., 2019; Vapnik & Ismailov, 2019) and discovering causal models (Peters, et al., 2017). These ideas are applicable to both symbolic and connectionist machine learning.

I work in a research department where one of my seniors are authoring a position paper on trustworthy AI, and we came into a discussion regarding the phrase “…understanding the theoretical basis for (human) intelligence has gone hand in hand with improvements in the capabilities of real systems.” Even though it was referenced to Russel and Norvigs book, I think the statement is misleading and a bit sensationalist, as more or less none of the recent advances in deep learning I can think of are inspired by neuroscience. In fact, the two seems more detached now than ever.

I’ve tried searching for good papers to back my claim on this, but I have not been able to find any. Am I wrong? Are there any good material on the subject of the connection between applied AI and the theory of human intelligence?

Abstract

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

Links to article (open-access): https://www.sciencedirect.com/science/article/pii/S0893608019300784

Direct PDF link: https://www.sciencedirect.com/science/article/pii/S0893608019300784/pdfft

Hi all.

Iam trying to train a convolutional variational autoencoder (CVAE) on computed tomography (CT) IMAGES (176X224 px) . The training data is normalized between 0 and 1 and Iam using approximately the same model structure as in keras autoencoder tutorial.

https://keras.io/examples/variational_autoencoder/

I only changed the depth and the size of the latent space to 128.

For the loss function I use Mse and KL, with a weight annealing for the KL part.

When I train the network it seems like it is learning something, but if I try to reconstruct images after training, the output images are just noisy.

I have no clue what it is Iam doing wrong.

Any advice would be really great.

Cheers,

M

Here’s the github repo.

Here’s another discussion on Hacker News

Right now it’s still experimental, but it seems to have a rather interesting way to deal with matrices (and tensors) by expanding einsum.

Hi Everyone!

I’ve recently started playing with ML, so far I’m thrilled with the results. I’ve been using https://app.nanonets.com which allows you to easily create and train models. The only thing that I don’t like about this is that I’m tied to this platform. I’d like to be able to achieve the same results using a third-party framework that I can host on my server, not having to depend on a platform.

I’ve done a few experiments, training a model to recognize castles, training a model to recognize panels on comic strips, so far so good.

https://imgur.com/a/zt574ai

I’m looking for a framework that allows me to do the same thing. The thing that I like about nanonets is that its simple. You just upload the images, label the content that you want to recognize and train the model, through a very simple and friendly UI.

Does anyone know a framework like this? That’s easy to use but doesn’t depend on a platform.

Thanks!

I’ve encountered node2vec which encapsulate deep walk, but besides that, I haven’t seen other methods. Is node2vec the best out there?