Last week we posted a video that presented LivePlace, a 3D world with an incredible amount of detail. The impressive technology behind it is called OTOY, a streaming platform that allows developers to generate movie-quality renders “in the cloud”, which can then be streamed to more modestly-powered computers and even mobile phones. For more information on OTOY, see our intro post here.
The video was available to the public at LivePlace.com alongside the ambiguous headline “Live or Virtually Live?”, but apparently nobody was supposed to find it. Soon after we published the post, LivePlace removed the video from its servers. Brad Greenspan, the entrepreneur behind MySpace who owns LivePlace, says that the site was never meant to be seen by the public, explaining that it was for internal mockups, viral videos, and “something similar to a Funny or Die episode.” That explanation doesn’t sit well with me, but it’s unlikely we’re going to get anything more substantial out of Greenspan.
So what about that 3D virtual world – is it a sham?
Jules Urbach, founder of OTOY, explains that while he can’t comment on what Liveplace is doing (or why they released the video), virtual worlds running on the rendering engine in the video are on the way. He says this video isn’t representative of his system’s capabilities (which have actually improved since the footage was shot), and is actually just a number of random clips spliced together by Liveplace:
“The 14 mins of real time rendering in this material is streaming live to a Treo 700 at 240 kpbs. This was captured on March 2007, the server was running an ATI RX 1900 GPU. The tech has improved massively since then (as has the HW we now run on). There was never intention to show any part of this to the public until we could include voxel rendering and Lightstage based characters. I think anyone who liked what they saw, will find the final project much more impressive.
The whole aim of our work last month on the Ruby demo for AMD was to show that the quality of offline and real time work is identical starting with this generation of GPUs. The following presentations this month are just introducing Lightstage and how it makes characters (or any CG object) look 100% real in those real time environments.
The virtual worlds these technologies are going to be applied to was not meant to be discussed until later this year, after one further announcement regarding the server side platform being developed for OTOY.
We had nothing to do with editing or leaking this video and can’t comment on anything other than the OTOY technology, since this project is still under NDA.”
One concern readers had beyond the lack of consistency seen in the video is the possibility that it contains material pirated from other artists. The video begins with a brief clip of cars that is apparently taken from a artists’ portfolio and was originally created years ago. As it turns out, the footage is old, but Jules Urbach explains that the artist is now part of the OTOY team:
“JJ has been working with OTOY/JulesWorld on almost all of our major projects over the past 3 years (some of which are still under NDA). I couldn’t be prouder to count him as a great friend and partner.
JJ’s studio, BLR, is always properly accredited on all videos that our clients let us put our logos on, whether it is a for a real time project or linear VFX work. You can see the BLR logo on the real time Transformers OTOY clip that was on Techcrunch a few weeks back (originally from Daily Variety), and you will see it again in a November print ad campaign featuring our work.
Note: The VW beetle you see in the very beginning of the BCN street scene was is one of JJ’s first CG models and is his ‘baby’. It has appeared in nearly everything we’ve done together – from our ‘Bumblebee’ Transformers ad for Paramount, to our most recent Ruby voxel demo for AMD (you can find it on the right side of the street). It is also in one of the images from the TechCrunch piece on OTOY last month (rendered in real time on 512 Mb R770, pre-voxel renderer).
So what’s the bottom line? LivePlace doesn’t seem to have anything to do with the video provided or the city described, and shouldn’t have posted the footage in the first place. The impressive OTOY technology behind it is real, but we will have to wait to see what products will be taking advantage of it.
Here are more technical details Jules has provided:
- We sore voxel data in several ways, including geometry maps (see our Siggraph or Iceland presentations, where we show this method applied to the Ligthstage 5 structured light data, courtesy Andrew Jones ICT/Graphics lab)
- The datasets from the BCN and Ruby city scenes contain up to 64 data layers per voxel, including diffuse albedo, fresnel reflectance values, irradiance data, UV coordinates (up to 8 sets), normals, and, for static scenes, look up vectors for 1-20 bounces of light from up to 252 evenly distributed viewpoints (it is important to note that this data is always 100% optional, as the raycaster can do this procedurally when the voxels are close and reflection precision is more important than speed; however, with cached reflectance data, you might see the scene rendering at 100s-1000s of fps when the scene isn’t changing).
- A note on raytracing vs. rasterization: amplifying the tree trunk in Fincher’s Bug Snuff demo to 28 million polys using the GPU tessellator turned out to be faster than rendering a 28 million voxel point cloud for this object. So there is a threshold where voxels become faster than rasterziation at about 100 million polys. At least in our engine, on R7xx GPUs, using full precision raycasting at 1280×720. Below that point, traditional rasterization using the GPU tessellator seems to be faster for a single viewport.
- The engine can convert a 1 million poly mesh into voxel data in about 1/200th second on R770 (60 fps on R600 and 8800 GTX). This is useful for baking dense static scenes that are procedurally generated once, or infrequently, on the GPU. That is why some of the OTOY demos require the GPU tessellator to look right.
- Hard shadows in OTOY were done using rasterization until we got R770 in May. Now hard shadows, like reflections, can be calculated using raycasting, although shadow masks are still very useful, and raycasting with voxel data can still give you aliasing.
- We can use the raycaster with procedurally generated data (perlin generated terrain or clouds, spline based objects etc.). At Jon Peddie’s Siggraph event, we showed a deformation applied in real time to the Ruby street scene. It was resolution independent, like a Flash vector object, so you could get infinitely close to it with no stair stepping effects, and likewise, the shadow casting would work the same way.
- The voxel data is grouped into the rough equivalent of ‘triangle batches’ (which can be indexed into per object or per material groups as well). This allows us to work with subsets of the voxel data in the much the same way we do with traditional polygonal meshes.
- The reflections in the march 2007 ‘Treo’ video are about 1/1000th as precise/fast as the raycasting we now use for the Ruby demo on R770/R700.
- One R770 GPU can render about 100+ viewports at the quality and size shown in the ‘Treo’ video. When scenes are entirely voxel based, the number of simultaneous viewports is less important than the total rendered area of all the viewports combined.
- The server side rendering system is currently comprised of systems using 8x R770 GPUs ( 8 Gb VRAM, 1.5 Kw power per box).
2 words
“HOLY SHIT”
“Brad Greenspan, the entrepreneur behind MySpace who owns LivePlace, says that the site was never meant to be seen by the public, explaining that it was for internal mockups, viral videos, and “something similar to a Funny or Die episode.””
OMFG.
“Hi-
Thx for email!
I have to finish next project before I can do anything new. and then I am going to launch site for entrepreneurs so this sounds great!
Lets stay in touch” – Brad Greenspan to Me, October 2007
Brad had written me this in late October of 2007 in an email when I asked him to help on my social search engine Peeplr by providing some developers.
He said he was working on a new startup for entrepreneurs like YouNoodle or something???
So I’m not sure where this fits in but the plot thickens.
http://www.freemyspace.com
Brad Greenspan also runs and operates this website aimed at exposing alleged insider trading at Intermix/MySpace prior to the NewsCorp deal.
Anyone who has ever done business with Brad Greenspan knows what kind of toxic BS to expect out of him. Why do you think he was forced out of Intermix? Ask around. My advice: stay away.
Tcruncher2, Holy Shit indeed. Looks like he’s about to create Earth’s first Matrix world… not all that surprising when you think about it…
If GTA5 had this I think we’d all cry. This would make the most killer sandbox game ever.
Just curious, anyone know of the hotel/building with the red ’swing-like’ objects and the pool? Thats a real building, i’ve seen it before.
And who CARES if its a sham, HOLY CRAP even if it’s fake it’s still a huge landmark on any level: CGI or whatever. They obviously didn’t film it.
Those renders are amazing… and real time! OTOY looks sick. I just wonder how much it will cost to stream all of that lovely data and whether our measly computers will be able to handle it.
Here’s what I don’t understand, and perhaps somebody that knows gaming can explain it to me. Brad wants to create something like MySpace with this having millions of people, and scaling this type of PlayStation 3 level detail on the server side to millions.
How can you do that?
Even the Googleplex couldn’t handle that type of processing.
You can’t render an entire city in 3d at that detail with every possible camera angle then aggregate that? Can you?
i don’t get it – rendering as a service?
how the hell is this supposed to be beneficial to anyone? the point of offloading all that rendering power seems to be to put it into a browser, but then how do you make it interactive if every time you move you have to wait for the server to render the stuff and send it back down the pipe? even with client-side rendered graphics and network gaming, lag is a huge issue, how can it not be even more of an issue for something like this? what gives?
please explain to the dumb guy…
This has to be a hoax. There’s no way in hell.
Nobody can explain it.
Say they made people on a PC download an ActiveX object like Google Lively and ONLY did OTOY rendering with mobile devices and stuff like the Macbook Air.
It would still overload their server capacity unless they had a server farm 1,000,000 the size of Googles.
http://ventureb...es-in-3-months/
“MySpace’s mobile site (WAP) ets more than 1.7 million daily unique mobile visits in the United States alone”
1.7 million daily uniques on a PS3 style rendering engine where the server has to render 1.7 million camera angles on an entire city as if 1.7 million people were playing WoW simultaniously on 1 server farm would be instant death.
There’s no way. I simply don’t believe it.
I know they’re going to offload PC clients with an ActiveX plugin, but even just the mobile clients which need server rendering would kill the servers dead.
I can’t believe so many of you are falling for this scam. Just think about how much hardware you’d need to support 1000 people playing this game. Then think about how much hardware you’d need to support 1,000,000 people. Do you think that OTOY would have enough money to install that many GPUs?!
Finally, if you just simply can’t figure out it’s a scam from the logical argument above, go see the thing in action and report back if it looks ANYTHING like what they claim: http://otoy.com/city/
(you’ll need a windblows machine for it)
Hi, I’m Eddie from Singapore.
Well, at least he starts something, even though it might looks weird to many. Only 1 thing I would like to comment, the so called “cinematic quality” 3D rendering doesn’t look so cinematic to me at all.
the next commenter is gay
doh!
Guilty as charged!
the previous commenter was gay
I enjoy hearing people saying “it’s absolutely impossible” and then one year later being embarrassed for not seeing the obvious at that point.
Explain your reasoning then?
How are millions of scenes going to be rendered simultaneously to flv video to be played in a flash player without sinking to 1FPS and having severe frame skipping.
A machine or even a farm of GPUs/CPUs can only render so many FPS, even if it’s a huge number of FPS through something like OGL. Once you add users, it becomes simple math to figure out how many can be on at once for 50-60 FPS.
It’s called division.
Then you have the video encoder, unless the flv frames can be written directly.
Say they have some crazy number, like they can do 1,000,000 FPS with the farm(dreamworks). Let’s just say that. That would only allow 20,000 people to play the 3d Sims game at 50 FPS. Then you have to add the video encoding overhead, and the streaming overhead.
There are tons of people from Google, Yahoo and Microsoft that read Techcrunch. One of them would have popped up and said, Eureka, that’s easy with Microsoft TM Direct X, you just use MSDN Pro and X framework.
They would be bragging about it practically. Or somebody from OGL would do the same.
Say the reasoning is that they’re rendering 3d at 320×240 frames for flv fit, to render 2 or 3 times as many frames, it would look like @ss.
The video clearly shows 800+ pixel resolution encoded down to 320×240
ATI RV770 with 1GB VRAM is less than $299. I’m sure they have some deal with ATI, so, $250 x 8 = $2K add $3K for the server itself – $5K/server. Maybe less.
First, I think the focus would be mobile devices as there’s no competition there. Even if they don’t figure out a way for businesses to subsidize the cost, people now are willing to pay.
Let’s assume that a server can handle 8 (number of GPUs) x 4 (users/GPU) = 32 *simultaneous* users. Of course, those users won’t pay 24×7. And even if there will be peak hours, let’s multiply 32 by a factor of just 2, i.e. 64 paying users/server. If the monthly fee on average is $10, this is $640/month/server, i.e. the server pays off in 8 months and then you start generating revenue. They may have pricing plans – free, mobile ($4.95), standard ($9.95), HD ($19.95), etc. Plan limitation could include number of hours per month, time of the day, peak/off-peak access, max resolution, max FPS, etc. Given people will be buying/renting space/stuff, there will be extra revenue. Advertising will bring additional revenue and eventually transaction fees from real purchases mad eat the VR shops will gradually become a major profit source, too. Buying stuff for yourself, gifts for others, or shopping together with friends and then receiving the stuff in the real world… the possibilities there have almost no limits!
8 users per server is going to pose other scalability problems such as power, space, and maintenance.
Right now a typical web 2.0 application will server tens of thousands of users per server.
“If the monthly fee on average is $10, this is $640/month/server”
Firstly, consider that Brad Greenspan is doing this. It will instantly be compared to MySpace, just as Vidilife was.
That being said, there was a huge campaign by MySpace competitors to make them lose users about 3-4 years ago.
Rumors were spread that Tom was going to start charging people money to use MySpace. Tom would repeatedly come on and disclaim those rumors.
They’ve conditioned people along with Facebook so well to expect social networking to be free, that a pay model will never work now. Lively and 2nd life added to that in the context of city space.
So now you have these expensive 8 GPU servers with no revenue except for some ad placement in the virtual world, which isn’t that lucrative, and traditional internet adverts which are very unprofitable on social networks.
Damn you’re slow, Chris. He said 32 users per server, not 8, and that would be simultaneous users, which means that each server would probably be able to handle 64-128 paying users, since they wouldn’t all be online at once.
And comparing myspace to this is idiocy, although I know myspace is your most favorite thing in the world and you aspire to build something so shitty and ugly. People will absolutely pay for something like this…look at WoW, EVE, and others. Even on 2nd life, people often pay for stuff, just not to actually play.
Please, learn to read, and practice on another site.
well I thought the video was amazing, I just wish home for the PS3 could have looked like that…
If they can actually pull this off it will indeed be very amazing – I’m excited, I would love to be on the beta testing team, I think the big question is how the world will be.
Is it going to be like second life where you can create stuff and sell it and will there be things in the world like apartments that will be for sale?
I would love to beta test this… just looks amazing
I read once that California alone uses more energy than the entire nation of China. Looking at this startup, OTOY, I can see why. 1.5Kw PER SERVER! Let’s suppose everything they say is true. They will need only 1000 of these servers to serve 1,000,000 people. That is, 1000 people can be served per server. That is 1.5MW! 1.5MWhrs EVERY hour for a SINGLE game with 1,000,000 players that uses this technology. World of Warcraft has 9,000,000. Myspace has God knows how many concurrent users, tens of millions probably.
No offense, but I hope not a lot of publishers decide to use this stuff.
I found all replies on TechCrunch are all trapped in an assumption: “For each client, the server has to render the whole scene again and again”
I guess in fact they don’t need to do that. Here is my guess.
1. Given a city model, it can be rendered according to camera location, orientation.
2. If users are only allow to walk on the street. Which reduce the possible rendering camera position and possible orientations.
3. If user can only move in fixed speed.
4. Given 1, 2, 3. The possible camera positions, orientations are limited.
5. The render farm render all possible scenes in system. They are marked by positions, camera orientation.
6. Assemble all rendering scenes according to users’ camera position, orientation and pace and stream the whole scene out.
7. We can add one more parameter, time of the scene. So a city can have 24 hours and the change of Sun, Moon.
So the problem is reduced to how to assemble those scene fast enough and streaming them. And because it will be a movie, so one important thing is to find the difference between two scene and compress them quickly.
If they want to add users’ avatars, the avatars may not need to be ray-traced. But just a object overlay in the scene.
“5. The render farm render all possible scenes in system. They are marked by positions, camera orientation.”
NO.
http://www.gara...e.php?qid=52582
I have some experience with game programming on both DX and OGL, and I know for a fact you can not render every single possible scene in even a tiny 3d space.
I was thinking that since the FLV flash video is only 320×240 that they could make a 1280×960 back buffer render per GPU
The 1280×960 square of video memory could render 16 @ 320×240 .flv sized scenes.
So instead of 1 GPU handling 1 buffer at a time, it could do 16 @ 320×240 .flv sized ones to be streamed to a flash player with red5 or FMS. So 1 GPU could do 16 users.
But that would look like @ss. What they are showing is high res encoded down to FLV, and that would be terribly expensive resourcewise. I don’t believe it.
Step back, people. Chris has spoken.
my money is also on overlayed avatars with pre-calculated 3d scenes on the background. The user wont be able to move the point-of -view.
I went to high school with Jules. He’s a genius, and I use that term sparingly. He turned down Harvard to continue working on video games he started in high school. He’s been at this stuff for going on 20 years now. Believe it. I hadn’t heard much about his most recent projects, as they were shrouded in mystery. I’m happy to see he’s taken all of his Flash work to the next level.
“When scenes are entirely voxel based, the number of simultaneous viewports is less important than the total rendered area of all the viewports combined.”
Weird that they’d go the voxel rendering route. Hardware usually doesn’t accelerate the drawing of voxels. Graphics hardware can draw triangles very quickly. Usually any voxel algorithm would need to be resolved to a triangulation method which then can be handed to the hardware for quick drawing. But even so in this case, it doesn’t seem to make much sense. Anything you can do with voxels you can do with triangulated models. Unless they plan to have the ability to tunnel holes and have deformable terrain of sorts; I can’t see why they’d go the voxel route.
More info:
http://www.flip...eleration.shtml
(note that the link is from 2000.)
There was an old game that used voxels called outcast. Do a youtube search for it to see how it looked back in the day.
For the folks who aren’t familar with voxel technology and are screaming about how it “can’t be done!” — do a bit of googling. The entire model is basically pre-rendered and has radiosity and raytracing from multiple light sources baked into it. It’s also the only way I can think of that you could support something that visually stunning from multiple perspectives, and I’m glad Jules came out and confirmed that it’s at least partially voxel based.
@Mo — Just a guess, but I’d say that they’re using specialized drivers (along the lines of how Nvidia’s “Cuda” stuff can utilize gpu resources).
The entire thing isn’t “pre-rendered” per-se. To entirely render each frame from every possible viewpoint would be impossible.
I think it is more likely that they just have a very complex and dynamic baking system (”dynamic baking”, is that an oxymoron?) which has all the necessary lighting, texture, and model data ready to go, and the final renderer has actually very little to do (cast avatar shadow and calculate reflections). The dynamic part is that the baking changes as the primary light sources change (sun, street lamps, etc) and any user-created changes.
In other words the only thing it calculates is when something changes. This means only reflections and your own shadows (and avatars+shadows of others) when you are flying around a city, since everything else is baked in until the time of day changes. Supposedly they can even have on-the-fly terrain changes, which means only that is calculated dynamically until it stops changing, and then it bakes itself in.
ah, sorry – that’s a better description of what I meant. The “pre-rendering” is pre-rendered & prepped for final. I’d love to see more of what otoy’s engine can do.
>At Jon Peddie’s Siggraph event, we showed a deformation applied in real time to the Ruby street scene.
That is untrue. I was there. He was unable to show anything in real time – he said his computer and network connection weren’t working. All he showed were pre-rendered videos. So much for cloud rendering – all you need is a 3G wireless card – or not, if it’s a whole sham, which it seemed like to me.
Great post!