Re:

progress?

I would think of it more of a space / field effects , Not recursive algorithm s

Last week I got to know Steve Hyman, Daniel Kahneman and Bob Horvitz. Telefonica invited all of us to a two day workshop with Pablo Rodriguez, Ken Morse and a few others, where we were meant to advise them on how to use Al for health applications. I told them that I think the goal of therapeutic invention is not to increase happiness, but integrity. Happiness is merely an indicator, not the benchmark. Current apps tend to subvert the motivation of people, but I don't think that this is necessary or the best strategy. Humans are meant to be programmable, not subverted. They perceive their programming as "higher purpose". If we can come from the top, supporting purpose, instead of from the bottom, subverting attention, we might be more successful. Downside might be that we create cults.) Of the bunch, Hyman managed to be the most interesting (Kahneman was very charismatic but mostly tried to see if he could identify an application for his system one/system two theory). Gary Marcus was there, too, but annoyed everyone by being too insecure to deal with his incompetence.

Did I tell you that I discovered that Deep Learning might be best understood as Second order AI?

First order Al was the classical Al that was started by Marvin Minsky in the 1950ies, and it worked by figuring out how we (or an abstract system) can perform a task that requires intelligence, and then implementing that algorithm directly. It yielded most of the progress we saw until recently: chess programs, data bases, language parsers etc.

Second order Al does not implement the functionality directly, but we write the algorithms that figure out the functionality by themselves. Second order Al is automated function approximation. Learning has existed for a long time in Al of course, but Deep Learning means compositional function approximation.

Our current approximator paradigm is mostly the neural network, i.e. chained normalized weighted sums of real values that we adapt by changing the weights with stochastic gradient descent, using the chain rule. This works well for linear algebra and the fat end of compact polynomials, but it does not work well for conditional loops, recursion and many other constructs that we might want to learn. Ultimately, we want to learn any kind of algorithm that runs efficiently on the available hardware.

Neural network learning is very slow. The different learning algorithms are quite similar in the amount of structure they can squeeze out of the same training data, but they need far more passes over the data than our nervous system.

The solution might be meta learning: we write algorithms that learn how to create learning algorithms. Evolution is meta learning. Meta learning is going to be third order Al and perhaps trigger a similar wave as deep learning.

I intend to visit NYC for a workshop at NYU on the weekend of the 16th.

We just moved into a new apartment; the previous one had only two bedrooms and this one has three, so I can have a study. It seems that we are as lucky with the new landlords as with the previous ones.

Bests, and thank you for everything!

Joscha

understanding is a multi dimensional space the language is a projection in that space. or an arrow in category theory. the focal point has history . so like the play appears different from every seat in the theatre the integaration over each point projects his understanding on the language.

What do you think of as space/field effects? The universe or learning?

Btw., did you ever come across Schmidhuber's idea of a Goedel Machine?

Hmm. I'd say that there is a multidimensional space in which understanding is projected. Understanding is the creation of a mapping between the features of a domain and a function that I already know how to compute, so I can simulate the domain. Shallow understanding involves mapping of a particular feature configuration, deeper understanding explores the latent variables of the feature set. There is usually more than one way of creating such a mapping, and when we have found several, we can also identify relationships between the mappings. Category theory systematizes that.

When you describe your understandings (such as the path of light through space), it seems to me that your perspective is descriptive, i.e. you look at the emergent pattern that is generated by your understanding, without looking at the structure of the generator itself. I try to understand that generator, i.e. how to create a structure that can produce the desired pattern. This constructive perspective is what computationalism is all about.

Btw, Google has just announced that they think they might be getting closer to quantum supremacy: https://www.technologyreview.com/s/610274/google-thinks-its-close-to-quant-m-supremacy-heres-what-that-really- means/

If they ever get there, I will be forced to revise a part of my current preliminary model of how the world works, which would be very exciting. It would probably mean that digital physics must be wrong, and finite automaton computationalism must be only treated as a theory about models built on constructionist formal languages, and I might get converted to Scott Aaronson's views.