Humanity has collectively built the Internet into an immense treasure trove of information, designed for visual consumption by humans. But preliminary signs indicate we can push the existing implementation far: with the right settings, our client can coax GTA V to run at 20 frames per second over the public internet. As we scale to larger games, there’s a decent chance we’ll start using additional backend technologies. We were all quite surprised that we could make VNC work so well. We can also save the VNC traffic for future analysis. There are even VNC implementations in JavaScript, which allow humans to provide demonstrations without installing any new software-important for services like Amazon Mechanical Turk.Įasy to debug. We can observe our agent while it is training or being evaluated-we just attach a VNC client to the environment’s (shared) VNC desktop. VNC as a standard. Many implementations of VNC are available online and some are packaged by default into the most common operating systems, including OSX. use supervised learning to mimic what the human does), before switching to RL to optimize for the given reward function. We’ve found demonstrations to be extremely useful in initializing agents with sensible policies with behavioral cloning (i.e. We can use human performance as a meaningful baseline, and record human demonstrations by simply saving VNC traffic. For instance, it can play any computer game, interact with a terminal, browse the web, design buildings in CAD software, operate a photo editing program, or edit a spreadsheet.įamiliar to humans. Since people are already well versed with the interface of pixels/keyboard/mouse, humans can easily operate any of our environments. General. An agent can use this interface (which was originally designed for humans) to interact with any existing computer program without requiring an emulator or access to the program’s internals. Here are some important properties of our current implementation: After experimenting with many combinations of VNC servers, encodings, and undocumented protocol options, we now routinely drive dozens of environments at 60 frames per second with 100ms latency-almost all due to server-side encoding. We wrote a batch-oriented VNC client in Go, which is loaded as a shared library in Python and incrementally updates a pair of buffers for each environment. Each screen buffer is 1024x768, so naively reading each frame from an external process would take 3GB/s of memory bandwidth. Our design goal for universe was to support a single Python process driving 20 environments in parallel at 60 frames per second. If we are to make progress towards generally intelligent agents, we must allow them to experience a wide repertoire of tasks so they can develop world knowledge and problem solving strategies that can be efficiently reused in a new task. In a standard training regime, we initialize agents from scratch and let them twitch randomly through tens of millions of trials as they learn to repeat actions that happen to lead to rewarding outcomes. One apparent challenge is that our agents don’t carry their experience along with them to new tasks. Systems with general problem solving ability-something akin to human common sense, allowing an agent to rapidly solve a new hard task-remain out of reach. For instance, AlphaGo can easily defeat you at Go, but you can’t explain the rules of a different board game to it and expect it to play with you. However, despite all of these advances, the systems we’re building still fall into the category of “Narrow AI”-they can achieve super-human performance in a specific domain, but lack the ability to do anything sensible outside of it. A reinforcement learning system, AlphaGo, defeated the world champion at Go. They are also learning to generate images, sound, and text. Computers can now see, hear, and translate languages with unprecedented accuracies. The area of artificial intelligence has seen rapid progress over the last few years.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |