Machines Like Us – Science News

By Steve Grand

Norm tells me a few people have commented on the bit in his interview with me where I say that simulations are not yet a substitute for the real world. The general opinion seems to be that this is close to not being true, mostly thanks to better physics engines. Well, I'm a huge fan of simulations -- in fact I'm working on a 3D, physics-based simulation now -- but I still think there's an awful long way to go yet and robots are much easier to build for certain things. I thought it might be worth starting a casual debate on this subject.

Physics engines certainly mimic collisions and forces pretty well these days, but there's a lot more to perception (and hence intelligence) than physics.

Vision, for instance, occurs when quadrillions of photons interact with tens of millions of rod and cone cells, after refracting and diffracting from thousands of surfaces. Anything more abstract than that misses the point about how vision works. We can mimic the effect in a 3D engine by rendering the scene from the organism's point of view to produce a bitmap, and that's a fair starting point. If 3D graphics are good enough to fool human eyes, they'll do nicely for a robot too. Each eye in the scene requires a render, though, so a single organism slows the frame rate by a factor of three (two for the eyes and one render for us, the viewers). It's not cheap, computationally speaking, but it's doable.

But then you have to add in touch -- a badly underrated sense. We use touch for all sorts of things (including calibrating our vision and muscles). We have millions of touch sensors, capable of telling the difference between the pressure of a doorknob, the pressure of a finger, the brush of someone's hair, the shape of a gesture, etc. Calculating zillions of sub-poly intersections is a pretty demanding thing, several orders of magnitude harder than any current physics engine attempts to handle.

And then there's hearing (thousands of reflections interfering with each other -- we use all that information to detect location, context, etc.), and smell (gazillions of diffusing molecules). We can in principle compute all these things with reasonable resolution, but not in anything like realtime. And don't forget that the robot also has to do its own processing work on all this massively parallel data, probably on the same CPU (or if not then we have communications bandwidth to worry about).

I suspect people will argue that we don't need to go to this level of detail, and we can abstract a lot of this sensory information (as I did in Creatures). But if we really want to understand how brains work (which I do, and I think we have to figure this out before we have any hope of creating true intelligence) then I think this noisy complex data is vital to the task.

Abstraction is what gave AI false hope. Even today, computer vision people tend to start with an RGB bitmap from a static, linearly-mapped camera, and abstract it straightaway into vertices, surfaces, etc. But the human eye turns a scene into a map of contrast ratios, then convolves the signals deeply, so that the information is smeared over wide areas, splits it into three or more parallel representations and modulates the data through rapid eye movements. This is what brains actually use to see with, not a static bitmap. And we can be damn sure that evolution has created brains that capitalise specifically on this particular form of representation. Starting with a bitmap or geometric abstraction of a scene skips a fundamental fact about the visual system -- a fact that's almost certainly crucial to its function.

Robots are hard to build, but they're a lot easier than building a complex 3D engine with suitable veracity (I've done both, so I'm not making this up). 99.99999% of the bandwidth and processing come for free, via the real world -- atoms bump into other atoms, photons diffract and interfere, electrical forces pull and push -- leaving us with the comparatively simple task of processing the sensory data and driving the motors (the real world then ensures that our robot will fall over ineptly, with much better veracity than a simulation).

Cheating is a bigger problem than any of us would like to admit. We take shortcuts for good pragmatic reasons, but each shortcut removes us one more step from the world that real organisms live in, and hence makes it that much harder to understand how they work.

You don't need any of this if you disagree with my general stance on AI though. If you think we can create intelligent machines without reference to the one natural example we have, and those machines can be highly abstract devices, which process discrete symbols and think without perceiving, then you don't need the real world OR a faithful simulator. In that case, stick to your blocksworld, feed your model some Prolog declarations and pray! I just don't think it's going to work...