Open Point-Cloud Rendering (Unlimited Detail in Games)

[loganryan99]

I think i'm going to start a new project - an open-source approach to point-cloud rendering. For those who don't know what point-cloud rendering is, i'll explain it later in this post. Open-Source software has always been one step behind the commercial competition, mostly due to funding. If we can eventually build up a community around an open point-cloud engine, we might actually be first to bring a solution to the table.

What is Point-Cloud Rendering?

Point-Cloud rendering is exactly how it sounds. Instead of objects having polygons, objects are instead rendered as a series of pixels. With traditional rendering, polygons are always being processed, even ones you're not looking at yet. With point-cloud rendering, we can throw this wasteful approach away. If you have a 1080p TV, you would have 1,382,400 (roughly) pixels. If each point in a point-cloud rendering system is rendered as one single pixel, that means all you will EVER be doing is setting pixels to a certain color on the screen. In fact, your computer is doing it right now as you read this post. When we eliminate polygons, we eliminate all of the calculations required to process the image on the screen, thus making the time to render an image of whatever detail you want instant - because a 1,382,400 pixel image is just that. With this new system, every image takes the same amount of power to display, which is roughly none. This eliminates most of the need for a GPU outright.

A good way of explaining how it works is uncompressed video. An uncompressed video file is simply a list of pixels and what color they should be set to. You can render a video file of a giant city like New York with life-like detail using the same amount of resources it takes to render a blue screen. My point with this is, the level of detail in a video file can be as much as you want without making it slow down, so we can do the same thing with gaming. A point-cloud rendering system is essentially a video file that you can edit on the fly! This means that you can be as detailed as you want, and it will NEVER slow down.

The Challenges of Point-Cloud Rendering

Point-Cloud rendering still is far from being problem free. The main challenges are figuring out which points need to be rendered, and storing all of the points.

So, we have a giant 3D list of points and colors, but how do we know which one's we are looking at? Assuming the camera perspective is a trapezoid (it most likely will be), we can do D[1/2(POV1 + POV2)] where

D = Render Distance, POV1 = the width of the beginning point of the POV (How wide your vision is close up), and POV2 = How wide your vision will expand (on the x axis) on the faraway parts of the scene. We can then see which points have a position that is inside of our theoretical trapezoid. While this is still very intensive, I think it is one of the best solutions to the problem. We can also do some pre-processing (such as eliminating points that are behind the camera), then only check the remaining points.

Our second problem is that there are tons of points, how are we supposed to hold all of those in RAM? I've got multiple solutions to this. Say we have a model of a house, and all of the point-cloud data that we need to render it. We are going to use this house five times on our map, so instead of having the house's points stored five times in RAM, we can store the house model separate and off-set the position of it's points to put it where want on the map. We could do the same thing with all of the repeated models, such as grass, characters, trees, etc. My next solution would be to have two points that represent the two corners of a rectangle, so on any rectangular object, we can specify it's top-left and bottom-right points, then list all of the colors for the points inside of the rectangle's colors and not their position. This obviously won't work on a large scale, because we're rendering in unlimited detail, so having a flat rectangle is rare. But, doing this on a tiny scale (every little tiny rectangle in the map) we can save a lot of space. There's really a ton of ways to handle this, and i'm probably not coming up with any efficient ideas (i'm not a compression type person).

This was going to be a project, but I realized an open-source one already existed, so i'm going to focus on something else.

[zahlio]

Interesting..

 

How ever the limits of game assets, will make alot of developers not choose this methods, as the game details will go down.

 

Also it would require alot of calculations on what the player actually sees that (finding the points) that it might not be as fast as it could be, because the latency would be way to much. (At least with current hardware).

 

However what could be done was:

 

A server with lets say 100 GB of RAM, would process your date (the clients simply sends the x, y, z and camera position) the server would then process this information and give you the information of what "group" of points you would need to see. You then simply get a series of numbers, which the client reads and display to the end user.

[loganryan99]

Interesting..

 

How ever the limits of game assets, will make alot of developers not choose this methods, as the game details will go down.

 

Also it would require alot of calculations on what the player actually sees that (finding the points) that it might not be as fast as it could be, because the latency would be way to much. (At least with current hardware).

 

However what could be done was:

 

A server with lets say 100 GB of RAM, would process your date (the clients simply sends the x, y, z and camera position) the server would then process this information and give you the information of what "group" of points you would need to see. You then simply get a series of numbers, which the client reads and display to the end user.

 

Hopefully as SSDs get faster and faster, loading assets will become less of an issue. Until then, sadly, "unlimited" is still pretty limited (as you mentioned).

[zahlio]

Also when developing a game you don't want to be restricted to users who only have a SSD. So i belive this methods of rendering is way in the future, but an interesting topic to discuss. 

[rufee]

I discussed this with a friend some years back.

There is a company "Euclideon" that claims they have a working example, you can find some videos on youtube about them.

Basically i see problems on how animations and most importantly collision detection would work in this engine without overloading the CPU, i mean you have millions of points each sticking out more then the other rather than a plain wall with triangle engines now. Even if you are not looking directly at some object the CPU still has to work on stuff that is happening behind you.

The other minus is real time lighting would be impossible and even prerendering levels would take forever due to the huge amount of detail that could be put in, but that depends on the developer. And "Unlimited" is still limited to the amount of HDD space you have.

Id love to see this technology evolve, but for now this would make a good looking engine with more resource requirement just like new engines of today.

[zahlio]

I discussed this with a friend some years back.

There is a company "Euclideon" that claims they have a working example, you can find some videos on youtube about them.

Basically i see problems on how animations and most importantly collision detection would work in this engine without overloading the CPU, i mean you have millions of points each sticking out more then the other rather than a plain wall with triangle engines now. Even if you are not looking directly at some object the CPU still has to work on stuff that is happening behind you.

The other minus is real time lighting would be impossible and even prerendering levels would take forever due to the huge amount of detail that could be put in, but that depends on the developer. And "Unlimited" is still limited to the amount of HDD space you have.

Id love to see this technology evolve, but for now this would make a good looking engine with more resource requirement just like new engines of today.

 

Completely agree...

 

There is alot of issues with this technology. And i think that the implementation of all the things that current engines has (real time lighting etc.) would make this technology so inefficient that its not even worth it.

[Darkfeign]

There are well-founded culling methods for rendering engines and while I'm not sure I understand what you're proposing fully, would point-cloud rendering not require crazy amounts of data storage for things like shadows and other lighting effects in a scene? You would have to store all possible positions for light sources, and this seems highly inefficient for moving objects in a scene?

 

Just some thoughts. Like I said, I'm not entirely sure I understand the approach as I haven't come across this before.

[rufee]

Thats the point it probably will, immagine a triangle the computer only needs to know 3 points in space to create it, now fill that triangle with pixels and each pixel needs its coordinate. Maybe there is a better way to save on data, but in a nutshell that would be the problem.