OT: interesting paper on AI

So, one computer can play with another computer?

You bet! I think they even have coding contests where one AI player faces off against others.

That's a step beyond the old core wars.

That's cool, he had to train it with his own gameplay, and then the AI segmented his gameplay input into sequences of 10 consecutive commands ("motifs") then randomly put them together to find good branches of commands in the statespace, with a lookforward time of of several hundred+ commands, and constant branch growth to take into account future events (ie falling off a cliff in the next 50 commands)

The feedback used accomplished by the "lexicographic ordering" of the nintendo RAM bytes, ie he found the pattern of bytes in RAM that consistently would indicate a successful gameplay, ie progressing towards the game's goal etc, which could be indicated by specific memory locations incrementing or decrementing.

It doesn't run in real time due to all the statespace branching analysis required to be done etc.

He talks about improving it by removing the motifs or allowing the AI to train itself, which would be an obvious first step to improve it.

I like his technique of state space branching, I am working on something like that myself, but I am planning on unsupervised initial learning to generate something comparable to the "motifs" he used by partitioning the statespace variables into easier to analyze chunks to find good motifs.

cheers, Jamie

In honor of the recently departed Minksy:

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

"What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said.

Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened.

:)