gStreamer in LWJGL

[matheus23]

I trust 7zip is ok:  https://dl.dropbox.com/u/52666052/GStreamerMid2012.7z
zip has no compression D:

._O?

Zip is a file format used for data compression and archiving.

wikipedia site about zip

[Cero]

I trust 7zip is ok:  https://dl.dropbox.com/u/52666052/GStreamerMid2012.7z
zip has no compression D:

._O?

Zip is a file format used for data compression and archiving.

http://en.wikipedia.org/wiki/Zip_(file_format)

I was exaggerating. It has bad compression =P

[nsigma]

@Cero, just noticed your re-post - I ended up working off your pastebins instead.  Hopefully these rewrites work for you - my dev machine is Linux, and I haven't tested on Windows.  Fingers crossed! 

I've moved a few things around, in particular I've moved all the OpenGL code below render().  I'm fairly sure this was the cause of at least one segfault.  Also had to deal with a few other exceptions after cleaning that up.  Also added in some stuff so it doesn't trample all over Slick's rendering - was writing over the font texture.  Ideally, the video code would be in a SlickCallable safe block, but I couldn't get this working correctly and ran out of time.

You'll need to change the video dimensions and file name back, as I hadn't got your archive file at the time.  You'll also need to re-enable the library loader (commented out for Linux) - I'd put it early in your main() too.  Finally, you'll have to change the package - I was using within an old existing Slick project of mine.

http://pastebin.java-gaming.org/54a6b7d341c -- SlickApp
http://pastebin.java-gaming.org/54a6b8d341c -- GStreamer_TestPlayer

There's some extra buffer copying going on which we can get rid of if this works OK for you, although the performance seems fine here.

Best wishes, Neil

Games published by our own members! Check 'em out!

[Cero]

Works great.
So now we gotta iron out everything.

First of all, in VLC using Theora, it wasnt possible to get the length of the file, as I said, its a VLC bug.
Is it possible in GStreamer ?

This

1   2

System.out.println(playbin.getState()); System.out.println(playbin.isPlaying());

returns PLAYING & true even after the video has ended

So it it isn't possible, I'm going to use the same fix as in VLC - when playing a video I pass the length in ms, which I already know, and then I can shut it down like this.
For cut scenes the game has obviously to know when its ended...

The thing on my laptop is still odd - but it did work
I'm going to check that out again

And then we have to include linux and mac binaries of course... Downloading the linux version of processing...

[nsigma]

1   2

System.out.println(playbin.getState()); System.out.println(playbin.isPlaying());

returns PLAYING & true even after the video has ended

I think that's correct behaviour, though you'd really need to go through the GStreamer API to be sure.  The length is definitely picked up - the position element in Praxis works fine with your video, and that relies on duration.

Instead, try using

1   2   3   4   5   6

playbin.getBus().connect(new Bus.EOS() {             public void endOfStream(GstObject arg0) {                 // mark your video as done here            }         });

There is also Bus.ERROR which you might want to add as well - this will get notified when (you guessed it!   )

You should dispose of the playbin when you're done (though possibly not from the EOS callback itself)

1   2

playbin.setState(org.gstreamer.State.NULL); playbin.dispose();

And then we have to include linux and mac binaries of course... Downloading the linux version of processing...

I've no interest in shipping binaries for Linux (and neither does Processing).  You're on your own with that one if you want to go down that route.  It shouldn't be that difficult to adapt the library loader code, but most Linux users should have GStreamer installed already.

Mac I'll be looking at as soon as I get a chance to test, and will post back here, as I will if I sort out a better buffer handoff mechanism.  The rest is up to you now!

EDIT: The GStreamer-Java mailing list is also worth checking in with.

[Cero]

It shouldn't work without plugins.  Could it be finding a system install of GStreamer on that laptop?

There was indeed a GStreamer preinstalled that was interfering, works as expected now.

[nsigma]

It shouldn't work without plugins.  Could it be finding a system install of GStreamer on that laptop?

There was indeed a GStreamer preinstalled that was interfering, works as expected now.

Good to know, though it still shouldn't happen. The pre loader should override a system install. I'll do some more checking my end, and keep you up to date if anything changes.

[Cero]

The pre loader should override a system install

After I deinstalled GStreamer, without restarting, I could see that it still tried to find plugins on the C drive where it was installed, and said "cannot find plugins there"
it worked - so it loaded the right plugins; but yeah, that overriding seems tough

[Cero]

And then we have to include linux and mac binaries of course... Downloading the linux version of processing...

I've no interest in shipping binaries for Linux (and neither does Processing).  You're on your own with that one if you want to go down that route.  It shouldn't be that difficult to adapt the library loader code, but most Linux users should have GStreamer installed already.

Mac I'll be looking at as soon as I get a chance to test, and will post back here, as I will if I sort out a better buffer handoff mechanism.  The rest is up to you now!

Well if thats true, then there is no worry I guess.
If ubuntu for example comes with gstreamer... which, I mean, it comes with a working totem player, so yeah
Only Mac then.

[nsigma]

The pre loader should override a system install

After I deinstalled GStreamer, without restarting, I could see that it still tried to find plugins on the C drive where it was installed, and said "cannot find plugins there"
it worked - so it loaded the right plugins; but yeah, that overriding seems tough

hmm .. looks like GStreamer has a default selection of paths it searches for plugins.  Rather than adding the plugin path with the scan path later, it might be possible to pass it in to Gst.init as one of the String args.

ie. instead of new String[]{} have new String[]{"--gst-plugin-path=PATH"}

I haven't tried this yet!

This won't stop it loading from the system paths, but should put those plugins first - see more at http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstRegistry.html

It may be more trouble than it's worth, though.  As you found, it doesn't fail.

Games published by our own members! Check 'em out!

[Cero]

The pre loader should override a system install

After I deinstalled GStreamer, without restarting, I could see that it still tried to find plugins on the C drive where it was installed, and said "cannot find plugins there"
it worked - so it loaded the right plugins; but yeah, that overriding seems tough

hmm .. looks like GStreamer has a default selection of paths it searches for plugins.  Rather than adding the plugin path with the scan path later, it might be possible to pass it in to Gst.init as one of the String args.

ie. instead of new String[]{} have new String[]{"--gst-plugin-path=PATH"}

I haven't tried this yet!

This won't stop it loading from the system paths, but should put those plugins first - see more at http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstRegistry.html

It may be more trouble than it's worth, though.  As you found, it doesn't fail.

well just

1	Gst.init("VideoPlayer", new String[]{"--gst-plugin-path='vlib_gst/windows32'"});

doesnt work for me, cannot find gstreamer...
But yeah your LibraryLoader works fine, so I guess we dont have to worry

I just realized you have a Gst.init in LibraryLoaderImpl's load() and I had my own in my player, we only need one I guess

btw, what do you do on Linux ? just dont use LibraryLoader, and java-gstreamer finds everything on its own on Gst.init() ?

[nsigma]

well just

1	Gst.init("VideoPlayer", new String[]{"--gst-plugin-path='vlib_gst/windows32'"});

doesnt work for me, cannot find gstreamer...

Well, no, it won't!  The path needs to be absolute for a start, formatted for Windows, probably without quotes, and you also need to make sure it's in the first call to Gst.init() - ie. in my LibraryLoader code.

1	new String[]{"--gst-plugin-path=" + pluginDir}

might work.

I just realized you have a Gst.init in LibraryLoaderImpl's load() and I had my own in my player, we only need one I guess

btw, what do you do on Linux ? just dont use LibraryLoader, and java-gstreamer finds everything on its own on Gst.init() ?

Keep both Gst.init().  The second call will be practically a no-op, but not cause any issues.  The call in LibraryLoader is needed prior to the plugin registration, but if you remove it from your code and don't call the LibraryLoader on Linux then it will fail. 

GStreamer-Java works fine on Linux without the library loader code, finding the system installed GStreamer by itself.  It also works fine on Windows for me with a system-installed GStreamer (from OSS builds), but I think the bundling is nicer and easier for the end user.  Not used a Linux distro in years that doesn't come with GStreamer by default.

[Cero]

works fine on Ubuntu

1   2   3   4   5

File dir = new File("vlib_gst/windows32"); String pluginDir = dir.getAbsolutePath(); System.out.println("--gst-plugin-path=" + pluginDir); Gst.init("VideoPlayer", new String[]{"--gst-plugin-path=" + pluginDir});

No dice.
path sysout is correct, still doesnt work.

[nsigma]

No dice.
path sysout is correct, still doesnt work.

Assuming you've put this in my lib loader code, then I'll have to do some more exploring when I get a chance - which won't be for a little while.  Still, it's working even with a system installed GStreamer so I guess it's nothing to stress about!   

[Cero]

Still, it's working even with a system installed GStreamer so I guess it's nothing to stress about!   

I agree.

[nsigma]

As mentioned earlier, GStreamer (webcam), LWJGL and live-coding GLSL fragment shaders in Praxis LIVE.  Finally got around to some videos! 

This is captured on Ubuntu, but working identically on Windows too.

<a href="http://www.youtube.com/v/19RIt9SslfY?version=3&amp;hl=en_US&amp;start=" target="_blank">http://www.youtube.com/v/19RIt9SslfY?version=3&amp;hl=en_US&amp;start=</a>

There's another GStreamer (though not LWJGL) example here.

[ra4king]

That is pretty amazing!

[nsigma]

That is pretty amazing!

Hey ra4king, thanks, glad you liked it! 

This is actually one part of Praxis I hope might have some use for people around here - it makes a fun playground!  There's no syntax highlighting of GLSL yet, but hopefully can get that done soon with some basic code completion.

[gouessej]

I'm sorry but using hardware acceleration in Java with video and even shaders is not so new. Good luck with gStreamer and LWJGL. nsigma's demos are amazing, I agree with ra4king.

[ra4king]

I'm sorry but using hardware acceleration in Java with video and even shaders is not so new.

?!? Where in the world did he say that this is brand new never-before-seen stuff?!

[matheus23]

I'm sorry but using hardware acceleration in Java with video and even shaders is not so new.

?!? Where in the world did he say that this is brand new never-before-seen stuff?!

JogAmp...

[nsigma]

I'm sorry but using hardware acceleration in Java with video and even shaders is not so new.

As @ra4king says, I'm definitely not claiming that.  It's not even new for me - been playing with this for at least 18 months, and was reading various blog posts about it before I started.  I posted the video more as an encouragement to anyone looking at this thread and trying to get it working that it does work!

What is potentially new (I'll at least claim more unusual  ) in Praxis LIVE is its architecture (http://praxisintermedia.wordpress.com/2012/07/26/the-influence-of-the-actor-model/), which enables not only editing the shader and controlling the parameters live, but the potential to connect audio to control the shader, or MIDI, or build a custom GUI, or a fragment of (live-compiled) Java code .. all while the video is still playing.

nsigma's demos are amazing, I agree with ra4king.

Thank you! 

PS.  Going on holiday in 5 min, so don't expect much else from me in the near future.  There is a thread for Praxis if you're interested in anything specific about it - I was not intending to spam this useful thread. 

[Cero]

GStreamer in LWJGL "Showcase" package

For now only win32 and 64 libs.

[Cero]

playbin.getVideoSize() return NullPointer
haven't tried anything but Theora, might be only with Theora, still sucks D=

[nsigma]

If you're calling it before the playback is initiated that would be expected. Anyway, the listener gives you the video dimensions.

[Cero]

If you're calling it before the playback is initiated that would be expected. Anyway, the listener gives you the video dimensions.

Can't figure it out, only other way I found was Video.getVideoSize and such but it requires a Pad.

I obviously need the info before I play the video to do create the Buffer with BufferUtils.createByteBuffer

[nsigma]

If you're calling it before the playback is initiated that would be expected. Anyway, the listener gives you the video dimensions.

Can't figure it out, only other way I found was Video.getVideoSize and such but it requires a Pad.

I obviously need the info before I play the video to do create the Buffer with BufferUtils.createByteBuffer

Surely you can create the buffer in the listener?  Check if null and create.  Check if null in update() too and just return.  You'll also have to synchronize on something else.  There are other ways, but that's probably easiest if you don't know the video size in advance.

[Cero]

Some changes.
Should scale properly now, does not really have resizing in mind, since in a game you have one resolution and video will just adapt to it.

SlickApp  http://pastebin.java-gaming.org/89b509a2c2e
GStreamerPlayer  http://pastebin.java-gaming.org/9b50a0c3e26

[Cero]

So for linux, I tried to get the libs aswell

I went into /usr/lib and copied everything that seemed useful, and later compared it with out mac and win folders of gstreamer and deleted everything that seemed unrelated... a lot of course
if in /usr/lib there was just a gstreamer folder with everything in there, it wouldnt be a problem, but its kinda like system32 on win
Anyway I'm pretty sure I have a linux32 folder for gstreamer now, which should work and may only contain more unnecessary stuff.
Me personally, I will delete everything but the theora and vorbis decoders later in real usage anyway because of license issues

Haven't had the chance to test it out though yet.
Does anybody know which linux distro has a live-cd version, java installed, but no gstreamer ?
well propably not =D

going to get just some linux distro, deinstall gstreamer and install java...