Streaming a real-time rendered application as a video?

[appel]

Hi

I'm wondering if it's possible to, let's say, if you have a game that is using LWJGL, is it possible to "capture" the screen/window and stream it as a video on the internet? (Allowing the user to send input to the server to change the video).

It would then work as a server-side-rendered game instead of what is normal, client-side rendered game.

I cannot explain exactly why I need to do this, as this involves a research project at my company.

In short:

OpenGL application -> MPEG2 stream to internet


Any insight is helpful!

[DarkMortar]

Fraps?

[woogley]

no.. he means the server runs the app, takes a snapshot, and streams it to the client. thus the client is just watching a video thats being recorded on the server in realtime (so each client would have about the same performance regardless of hardware)

i havent seen anything like it, but good luck..

Games published by our own members! Check 'em out!

[noblemaster]

It's possible if you can deal with the following problems:
-probably 1 server per client (1000 clients => need 1000 servers)
-very fast connection to obtain the movie stream with decent quality
-latency: 1. sending data to server, 2. rendering on server, 3. encoding to movie stream, 4. sending stream over the network, 5. decoding and display of stream on client ...

[appel]

It's possible if you can deal with the following problems:
-probably 1 server per client (1000 clients => need 1000 servers)
-very fast connection to obtain the movie stream with decent quality
-latency: 1. sending data to server, 2. rendering on server, 3. encoding to movie stream, 4. sending stream over the network, 5. decoding and display of stream on client ...

Yes, that is probably true. I know about the problems But I'm interested in experimenting with this.

[noblemaster]

One of the projects I am working on consists of a surgical simulation framework that allows to stream video from a server to connected clients to facilitate distributed surgical training with haptics interaction. The project is written in C/C++. The server can be run in parallel mode on a super-computer with through Internet2.

From what I have seen, we got 1-2 frames a second tops. Note: I was told it can run faster ...

[moogie]

How about sending the captured image in a gzip stream? I would imagine that compression/decompression via gzip will be fast enough for such a system.

This is what i would try firstly to see if it is feasible:

-make a simple animation program e.g. a ball bouncing around the screen.
-change it so it is now rendering to a BufferedImage
-in the render loop, grab the image data e.g. byte[] data=((DataBufferByte)img.getRaster().getDataBuffer()).getData();
-write this data to the GZipOutputStream

on the client side I would simply

(assumption that we know the size of the image data buffer)
-block until the data is fully read.
-create a BufferedImage from this databuffer.
-display the image and then repeat.

of course this is very dumb and does not allow frame skipping etc, but as a starting point it may prove useful

Note you will have to make sure you have created a BufferedImage of one of the folowing type:

TYPE_3BYTE_BGR
TYPE_4BYTE_ABGR
TYPE_4BYTE_ABGR_PRE
TYPE_BYTE_BINARY
TYPE_BYTE_GRAY
TYPE_BYTE_INDEXED

[woogley]

what you could do is just send the pixels that have changed since the last frame. this would work well for most 2D applications (I believe this is how VNC/VPN is done). check out RealVNC (java program) or logmein pro or whatever those things are called

[appel]

Good tips. I'll look into them thanks.