Imagine you could play video games – and immerse yourself in virtual worlds – with 3D graphics comparable to those found in blockbuster films like Transformers or WALL•E. And then imagine you could experience and control those graphics in real-time from any internet-enabled device, whether it be a desktop computer, set-top box or even iPhone.
Sound far-fetched? It doesn’t to Jules Urbach, founder and CEO of a Los Angeles-based company called OTOY, who has been working with microprocessor manufacturer AMD since 2006 to make the idea of server-side graphics processing a reality. If all goes as planned, 3D rendering will become just another computer task that jumps from the client to the cloud. Call it gaming as a service (GaaS) if you will. No more Xboxes, no more PlayStations, and no more souped-up PC towers. Just a monitor, some controls, and a way to receive and display frames generated by a powerful server farm.
But let’s take a step back for a second. Before it’s even possible to deliver movie-quality graphics through a thin client, there must be a way to produce those graphics – and in real-time. Movie producers have the luxury of knowing ahead of time just how they want their frames to look. Visual effects studios like Industrial Light and Magic don’t have to respond to user inputs, so they can spend hours rendering each and every frame. Game producers, however, rely on engines that must respond quickly to user behavior and serve up graphics at near-instantaneous speeds. That reliance constitutes perhaps the main reason why in-game graphics have lagged behind their big-screen counterparts for years.
Just the other week, however, AMD announced an initiative called Cinema 2.0 that promises to narrow the gap between movies and games with a new RV770 GPU. To demonstrate the power of AMD’s new consumer graphics cards, Urbach and his art teams in Spain, Canada and the US pulled together a set of videos that approximate the CGI you’d expect from movies. He took us through an overview of that work here:
Most of his demos focus on recreating Autobots and Decepticons from the Transformers movie. And the results are very impressive, even if they don’t quite match those found on the big screen. The stills at the bottom of this post are from voxel-based animations that were rendered in real-time, such as the one embedded at the top of this post.
But the rendering of machinery poses far fewer challenges than producing humanoid models that suspend disbelief. To achieve the organic in addition to the inorganic, Jules has worked on a project called LightStage that takes panoramic shots of real humans in motion and turns them into animated 3D models. Watch below as Jules explains how the Lightstage works:
All of this is just an extension of what has been done by technologists so far to mimic reality within virtual experiences. Urbach’s bold and particularly innovative proposal is that he can deliver these experiences through the browser. While we’ve seen 3D games delivered through the browser before, this time it’s very different.
First of all, OTOY-powered graphics can potentially go far beyond those found on any consumer device because they aren’t actually rendered by whatever hardware is sitting on your desk, resting in your hand, or laying on your living room floor. In the video below, Urbach shows how AMD graphics cards installed on the server (rather than the client) can be hooked up to work in parallel and deliver highly complex graphics from afar, in the form of pure frames.
The main limitations are bandwidth and server power (i.e. how fast the client can receive frames generated by the server, and how fast the server can generate those frames for all its concurrent users). Urbach claims that his technology can deliver up to 220 frames per second (fps), which is overkill for most monitors and the human eye. As for lag, he experiences 12-17 milliseconds on the west coast (where his current test server is located) and 100 ms in Japan. The compression codec used to deliver these levels of performance was developed internally, although with help from AMD’s engineering team.
The second main difference is that Urbach’s technology doesn’t require any browser plugin whatsoever (although it can take advantage of those, too). OTOY-powered graphics can be delivered via Ajax, Flash, Java, or ActiveX. Surprisingly, the Ajax-powered version in Safari works fastest. That’s good news for future iPhone 3G owners since this graphics delivery system is compatible with that mobile device (and any other device with a full-featured browser, such as Android). The possibilities here make one wonder whether the days of PSPs and other portable gaming devices are limited. And they suggest that the future of the web-enabled cell phone is bright indeed.
Expect one of the first commercial implementations of LightStage and this server-side rendering technology to come in the form of a virtual world. Urbach also plans to release a full suite of developer tools to those who want to leverage OTOY for their own applications.
OTOY is privately funded, although much of its GPU-based hardware will be provided by AMD. More stills from real-time renders and LightStage results are provided below.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
“We’re sorry this video is no longer available”
Yes – all the youtube videos are down…
stop putting up illegal content guys lol
note to self: make youtube videos public before posting them
why all videos are down already ?
Nath
http://www.them...rfulcompany.com
This is fantastic. The developments we can expect to see in the next 2-3 years will be absolutely astounding.
Cool – Mark H please delete both my posts if you want.
No, the main limitation is lag more than anything else if you want a smooth interactive experience. With all the other services that have moved from the client so far this has not been an issue, with this it would be a major one.
Mark,
That always gets me too. It is hard, as when you are logged in they play just fine for you, so if you preview the post it looks fine. I would like YT to have a little ‘private’ red dot or something to let me know
Cheers,
Dion
(I work for Google so I should try to make that happen!)
Now this is innovation. This is a brilliant idea if it works well. The biggest thing is the response time though. If you click and rotate around a model, it has to respond almost instantly. That seems extremely difficult.( That means it has to call the server and deliver frames right away, even crappy ajax apps lag today).
Damn hard concept too do well, but if they do, then that would be awesome. Then 3d really will be in the cloud and the web will change forever.
how are they going to send every high resolution frame (at least 1280×1024) with at least 32 frames per second through any Americans broadband connection and not have lag?
As it is now, when playing any game online where there is a multiplayer component, the client renders all the graphics, and the server simply sends back data points for the client to turn into nifty graphics (where people are, where the bullets are heading, etc). most of those games lag as well.
Try this, take a high resolution image (above 1280×1024) and email it to your friend…how long does it take to send the email? Or even FTP it somewhere….
its just not gonna be smooth…
not to mention games are written for DirectX 9/10…what happens when a new game or DirectX version comes out? Do all the servers update themselves with the latest version? have fun applying every patch for every game to correct bugs seen on a small number of client puters.
This may be able to be accomplished for pokey little flash like games, but high end games, I think not…
power gamer
“Try this, take a high resolution image (above 1280×1024) and email it to your friend…how long does it take to send the email? Or even FTP it somewhere….”
1)The DOWNLOAD Speed is way faster than UPLOAD speed, example , Download speed 6 to 8 megs/Sec , upload 500Kilobytes/sec.
2) Good Image compression example JPG vs BMP
“what happens when a new game or DirectX version comes out? ”
1)Well is better for a Game Dev. Company to update a few Servers than 1,000s of Client Computers.
2)The reason why they make so many fixes and Patches is to satisfy the needs of many different Types of PCs (Drivers, OS, Hardware)
These guys are going to need a big server farm
Why ?
What woud possibly motivate you in financing a massive data center to run this. Google Reader’s response time is (at best) 1 second between clicks. Just think how much more processing this is going to need and how much faster the lag time would be. Just the fact that you’re routing in the internet alone would mean lag time issues making games unplayable – forget what hardware is running behind it.
Then why would you do it? The thin client business model never succeeds as client devices are getting more and more sophisticated. Distributed processing wins hands down.
The web has only succeeded because it involves centralizing and connceting data – not centralizing horsepower.
This is the technology of the future – perhaps the next generation – but one becomes fearful of a society that will be unable to distinguish reality and virtual reality because of extreme realism.
Also considering the fact that most of these games deal with violence and extreme property destruction, one wonders if people will become hypnotized by this sort of action packed violence starting from very early ages
What would really be great is this technology being used in the Education field, Medical Field or the Space Science field – but understanding the economy we live in, it will probably be entertainment oriented.
@The World is BEAUTIFUL
Violent video games are cool, haven’t you ever played Grand Theft Auto?
Sounds great for rendering commercials or anything expensive. I’m excited that someone is commercializing DeBevec’s Lightstage technology. It’s as good as it gets.
But for doing virtual worlds or games, you need to scale the server technology well beyond “real-time” performance (in the video, they seem to get maybe 5-15 fps). At 60 frames per second, you’re good for your home machine. A server with 1000 simultaneous users, though, would need to render at the equivalent of 60,000 frames per second and handle all that data simultaneously.
We’ll get there. But at current colo and bandwidth costs and only one (almost) real-time user per server or servers, this is currently only good for applications where a company can charge upwards of $1-$5/hr for their service, and that rules out most games and virtual worlds and keeps this very ’boutique.’
The good news is that Moore’s Law will take care of this in only a few years, so it’s smart to be ready. But rendering on your home machine will also improve at the same rate.
No plugins? Java and activeX are plugins. with ajax you can only serve images. anything else would require a plugin. I suppose it could be done in flash, but performance would be rather useless… well, wait and see, rather sceptic about the not having to download anything additional…
@Peter – If you don’t have plugins installed, the graphics will be served up as images using JavaScript. Any controls will also be powered by JS.
because we all want to play World of Warcraft at youtube quality.
Colo energy costs are going to be ENORMOUS for this. MUCH more environmentally friendly to let NVidia and AMD distribute this processing for you. That is, to put all those transistors in geographically distributed PC’s, consoles, and mobile phones, instead of putting them all in a set of colocated electricity sucking servers. (I know, it’s best not to have them at all, but I’m being pragmatic here).
Obviously, they have not yet sat down and calculated the wattage that will be necessary to run, and especially cool, that many GPUs. Even with the expected advances over the next 5 years, we are still talking about the energy equivalent of a small city being siphoned off the grid. Realize, I am being EXTREMELY generous with that statement. Since this guy will probably be using GPUs made for PCs and not mobile GPUs for laptops or cell phones. God help them if they locate in a municipality where that energy comes from petroleum products. Natural gas say. Or coal. There goes all that extra energy we thought we’d have. Good news is the energy companies will charge them through the nose for it. Bad news is the game company will probably charge you.
Incidentally, by the time you are finished paying the fees necessary to support this thing, you will wish you had just bought a graphics card. Or by that time, even your mobile CPU will have 32 cores anyway, your cell phone will have at least 16. Again, VERY conservative estimates, it will probably look more like 96 and 32, and game engines will do ray-tracing. I don’t know about you, but I will feel silly having paid BOTH for all those cores in my new iPhone or Nokia n, AND paid to have someone else’s server render my graphics.
Sometimes I think not a lot of analysis goes into the planning before launching off on these ventures.
Most of the comments I see here are all complaining because it doesn’t seem practical. Sure, I agree, but most businesses are based on making the inconvenient convenient, or the impractical practical. I particularly like L0pht’s tagline the best: “Making the theoretical practical since 1992.”. It’s easy to criticize someone who is doing something that’s never been done before.
If they are able to do something like this, I would gladly pay $20 a month to play in their virtual world. I love the idea. Especially if they can deliver this to an Xbox 360, or my regular PC. Hell, sign me up for the beta. I want this on my Nokia N810, my Laptop EEE pc, my home PC, and my Xbox 360.
If you want to see a practical example of this technology, take a look at “streammygame.com” which is a home computing example. Imagine this taken to the next level, it seems a little more believable now doesn’t it?
The will be an intermediate step before this will be streamed to homes (and watchable) because of the obvious bandwidth limitations.
There are two things you are going to see first. One is the multiplex cinema siting on a server farm (eliminating bandwidth latency) and rendering movies (or parts of movies) in real time.
This would enable some really cool things like audience interaction and geographic localization. Think about a storyline hitting closer to home because the movie is based in your home town. What if King Kong climbed the Eiffel Tower, Sears building or CN Tower in the climax to the film? What if every outdoor scene in a chick flick (sorry) had a recognizable local landmark or two? Would it make the impact much more personal and help draw viewers into the world? Of course it would.
The second thing you are going to see is the resurgence of the video game arcade. Again, the data center would be on-site so latency would not be an issue. You could also place these face scanning machines in each arcade so for every game you play, you are the central character.
Just imagine video arcades and movie theaters becoming huge again!
Of course, big studios (with big money) are looking for something precisely like this, so they can create an experience that people simply cannot have at home (like you can with today’s movies and a decent TV+Stereo).
Now, lets just hope they don’t choose Windows as their platform. Can you imagine a 30 foot tall blue screen of death?
The horror…
I believe a mixture of client side GPU, and Server side GPU delivered assets to the client will create an amazing new virtual world. There is a reason why things are done on the client, too numerous to name here.
This is a great idea and will definetely foster a new method of thinking int he games and animation world, BUT to think that everything will just go to the server will never happen in our life time if at all.
This is a cool technology that i will actively monitor, thanks for the info !
ever heard of server-push (ala comet), whereby a neverending connection is established with the server from the client and the server uses that connection to continually push images to the clients screen.
This technique will greatly increase response time and enable this technology.
(btw this in not ajax, its a hidden iframe)
also its a game, the better game developers produce their best work under constraints such as these
did people forget their brains? most user’s gaming computers are more powerful than the average server. and definitely has better graphics hardware. oh, and it can get an image to the screen without having to go through a dsl connection
This article is stupid.
Real time rendering server side might be useful for delivering small low resolution 3d applications to phones and browsers but to say this technology is somehow going to make real time rendering equivalent to Hollywood rendering is bizarre. What has delivering 3d images as frames using ajax got anything to do with making real time rendering any better?
Real time rendering hasn’t changed all this does is moves the rendering hardware cost from the client to the server. You still need the same amount of processing power but you add latency.
Great idea but not sure if there is a business case for realtime 3D games. Of course 3D scene authoring, OpenSource Renderman farms for indie movie developers – this might actually get the guys who need non-realtime huge rendering architecture to have a serious look at Otoy.
- We are already in a time when the graphics processors are virtually more powerful than the processors. Hell, we are even talking about realtime raytracing engines. I don’t even know how a distributed server-based rendering can even get to give the experience and the flexibility as an onboard graphics card.
- This might be a case for mobiles but we again talking about 10MBps speeds as yet?
- Latency kills gameplay. In 3D games its not as much the problem of hi-res rendering as is the responsiveness of the system. Game engines have been using low-latency socket architecture (like UDP) and send only the very basic player location information and other team information only to clients and let the client worry about the rendering. Even the linux Xserver sends the commands to the clients and lets the client do the rendering for itself if it can. Assuming 25 frames of 1 million pixels (assuming a byte per pixel) we are talking about atleast 25MBps bandwidths for the nearest of places. Not to speak of the huge cost of having to save each and every frame (from the user PoV) into a image, compress it, stream it back. We might be there but just not yet. But in that time my local nVidia or AMD card might produce that punch anyday and definitely cheaply than these render farms.
- As one commenter already noted, the environment costs of this is going to be monstrous.
- Not to speak of having to integrate streaming visuals from the server as part of the game engine itself.
- Not to sound rude, but I think OTOY is never intended to
Great business case for SaaS for high-latency render farms for CG studios, indie movie guys. Gaming – joking right?
Company feels like a plug for AMD to compete with nVidia for corporate clients.
Total and complete BS. never trust a Frenchman, and a French Canadian at that….liar.
alan bernard is NOT a liar!!!
Any one familiar with 3D engines and “light stages” will know that they are selling a LOAD of bs here.
Very Impressive!
To all those writing it off you don’t get it or have fanboy blinders on not realising this is the future of PC gaming or a similar service.
1) Only a tiny handful of elite computer users can ever afford the cost of dedicated GPU’s this closes off a huge customer base for PC gamers.
2) PC gaming is not doing so well but it’s not down & out yet, this service would kill the piracy complaints stone dead as the game is streaming to your computer.
3) A service like this streaming games to any computer on the plant would remove much of the complexity and problems associated with PC gaming definitely putting PC gaming back as king of the hill.
4) Any crappy PC could be used to game on, no more shelling out for expensive hardware though expect resistance from PC hardware/gamer enthusiasts as this will deliver a death blow to this faction.
5) This is already happening, Sony’s remote play feature with the PS3 does the rendering and the game in your PS3 can be played on a PSP & it does a pretty decent job of it.
This is the future of PC gaming as the old/current ways just cant go on if anyone who truly wants to see PC gaming prosper.
Great point, this could also be the beginning (some dread) of the one-console-future (OMG)
If the Internet Bandwidth is there to support a HQ experience this will pown nay-says and soon enough every MMO will hop on as well
multi-player
I am not a developer but I like games, please please Second Life is too pixelated it sucks, invent a world combining face recognition software and OTOY. I think it would be great, imagine talking to people seeing their expressions, in a virtual world with virtual property business model. ok.
in essence, combine the following youtube videos…
http://www.yout...feature=related
http://blip.tv/play/AcfBVou8cA
Well, I AM a developer, and I only have a couple of things to say:
1. Zero piracy. Ever.
2. Big problem with XBox and PS3 in the early days was that games got bigger and more expensive to make, with a tiny install base of consumers to pay for it. With this system, the instant new tech is invented and installed in the server farm, EVERYONE HAS IT. You can develop your next-next-gen game timed to hit that market from day one knowing that umpteen million people will immediately be able to buy and play it. That is huge on a completely unprecedented level.
As far as i know this Jules Urbach founder of Otoy works for 3D Blasphemy the makers of the original videos that have been compiled in this video. I can’t blow up the pictures at 3DBlashphemy.com, the pictures won’t seem to load but you can see in the picture someone who looks a lot like this Jules Urbach, i am sure it is the same guy, which then proves this is all for real.
Welcome to our website… We offer a range of architectural presentation services at amazing prices.
Our goal is to consistently create photo-realistic Architectural Rendering and 3D models.
We deliver the final product on-time, on-budget.
web:www.cn9d.net.cn
Welcome to our website… We offer a range of architectural presentation services at amazing prices.
Our goal is to consistently create photo-realistic Architectural Rendering and 3D models.
We deliver the final product on-time, on-budget.
http://www.cn9d.net.cn
This looks great but the details might be closer to science fiction for real time gaming in the near term. I saw a service by Playce that may offer more for high fidelity 3D games via the browser http://www.playcelive.com/ – in the near term at least.
why the site http://otoy.com is not working?!
Isn’t that an old video? I’ve seen like 10 minutes of that video once.
Beware.. this is quite alot of bullshit from such big players. Consider this: latency to your _average_ connection is between 100-200ms, play on a local ISP you might get down to 20-40ms. But these guys are talking about joining a ‘cloud’. They could be anywhere. The reality is, that 100-200ms is something that wont disappear without _infrastructure_ changes. You can claim all sorts of crap about realtime interactive movies, but if you dont have the infrastructure to do it (ie network) then its all just smoke and mirrors people.
I dont see a seismic change in network infratructure occurring across a large scale anytime soon.. so dont expect this to work either.
As a communications technician within one of the worlds largest telcos, I must add a few points and expound upon points made by several other posters.
Firstly:
1) What is the speed of light, and what is the refraction index of optic fibre
2) What is the conversion speed of light into electrical signal via the use of transistors and optical recievers
3) What is the conversion speed of electrical signal into optical transmission via laser
4) Calculate the best possible scenario in regards to the above figures, for transmission times through a network of packet switching equipment over any large distance
Basically, the ONLY WAY THIS SYSTEM WOULD EVER BE VIABLE FOR GAMERS IS IN A LOCAL CONTEXT.
Until our computers and transmission equipment are capable of instantaneous transmission from A to B (currently a couple of projects btw) this technology is restricted to applications which can safely ignore high latency.
However, all this said, I would absolutely love this technology to be taken further, and applied in a local area hence keeping latency down.
FTW
hello everyone…recently our company initiate one of the project for 3D rendering for medical purpose.The requirement for this project is using thin client , that mean required grid computing across the internet. Did anyone know any kind of reference or any company that can help me on this ?
You can use BOINC (http://boinc.berkeley.edu/) and solicit your own own users. Or your could use Plura (http://www.plur...processing.com/) and they already have a ton of computers you can use.
For BOINC, you don’t pay your users but you have to convince them to join your project. With Plura you pay a tiny amount for each user, about $0.01-$0.02/cpu-hour, but you don’t spend any time looking for people to join your project.
Thank westide. Do you have any site for reference that currently using thin client for 3D rendering ?
3d rendering is the hottest topic now. seems every architect, design studio or rendering company favors it.
http://www.lifang3d.com, it is a professional 3d company in china providing high quality works and comparative price for every client. kindly visit it.
if you want to know more, feel free to chat with me: email: anne@lifang3d.com
msn: anne_szp@hotmail.com
skype: anne_lf3d
The real performanc test would be trying out highly computation intensive, interactive Volume Rendering through 3D Texture Mapping, and Hardware Acceleration.
a question.. OTOY is the first game that randering itself while gaming or existing other games that can that??
plz dont be angry about my bad english i come from germany ^^