Cortado works!

[bobjob]

"Weasel word", not "weasel".

http://en.wikipedia.org/wiki/Weasel_word

shit. Well I learnt something today.
sry for being so ready to assume the worst of ya 

[Momoko_Fan]

Anyway... I'm hacking all the Cortado stuff out of Cortado, and just getting something really basic working. Eventually. I hope.

Thanks! This would be VERY useful for me. If you could make it work with LWJGL then it will be good. I already have a working OGG/Vorbis -> LWJGL-OpenAL so I don't need the sound part.

[nsigma]

Can you name a commercial software licence that works in the same way? They typically work on one of two bases: either you can use it but not distribute it; or you can distribute it, sometimes with fixed royalties per instance distributed. The most they're likely to insist I do with my own code is obfuscate the entire project.

IMO they're the same thing.  In the first case, you can do exactly that with GPL code.  In the second, you pay a royalty of "£10 per user" or you pay a royalty of "release all your code to each user".  Either is a cost to you, and you make a decision whether that cost is worthwhile, and the author protects their investment in the code.  Even more liberal licences have some cost, even if it's only attribution!  There are a lot of companies releasing dual licensed code as well (GPL and commercial) so you get to take your pick, and they benefit from the protection.  If you're intending on making money from your code then GPL probably doesn't offer the best solution, if you make money tangentially to your code (as I do) then the cost to me of using GPL code is better value.  I'm no free software zealot, just pragmatic!

And my only objection to the word "contaminate" is because it's too linked to the idea of the GPL being "viral".  This is a reaction from companies who thought that the only valid transaction for software licensing was monetary.  People make huge numbers of transactions all the time that are not monetary!

So, in a vague attempt to get this (or me!) back on topic

I would like to see a theora video solution for Java that's LGPL, and fulfils all our needs - that's why I delurked in the first place!  I don't believe the GPL is anywhere near as ambiguous as stated, and I don't think that hacking around the GPL code and hoping no-one notices is a viable or ethical option either.  I'm also happy to help out or do the refactoring myself, though I don't have any free time for a month.

I think that attempting to use the Java port of GStreamer from Cortado is a good idea, as it offers a ready built framework which could be made to work with other pure Java codecs in future.  I am also a great disbeliever in reinventing the wheel without good cause!  Might be worth an email to Fluendo first to see if they won't just re-licence the offending classes though.

I'm without t'internet for the rest of the week, so don't expect any further postings from me for a bit ... yeah, I heard that sigh of relief at the back! 

Best wishes, Neil

Games published by our own members! Check 'em out!

[princec]

Anything I produce will be under no license at all (as in, public domain), or the LWJGL-style BSD license.

Cas

[elias4444]

I'd recommend using a BSD style license... just so no one steals your code and licenses it themselves (it's actually happened to me).

[princec]

I suppose so. Though I honestly couldn't care less. I'd only steal it back...

Cas

[Chagma]

Has anyone made any progress on this, i.e. getting synchronized video and audio playback from Java?  I found Xuggler but it makes use of native libraries and has synchronization issues on Linux.  I need a pure Java implementation that just works!

-Chagma

[princec]

Well, as I say, Cortado works, but it's got complex licensing problems and doesn't integrate with OpenGL too well. I'll probably have something working in a few weeks but it will be heavily hacked to my own frameworks unfortunately.

Cas

[Chagma]

Does "my own frameworks" include LWJGL?  If you can get something that synchronises Theora video and audio and outputs the video as buffered images or such then I'll be able to do the rest for my applet.  I am at the point where I need to make a big decision: whether to go with the technology I have developed or abandon it in favour of something else entirely.  It all comes down to being able to play back Theora in an applet.  If it can't be done then it's back to the drawing board for me.

[Serethos]

CommanderKeith noticed, that you might be interested in this alternative:

http://www.java-gaming.org/index.php?topic=21813.new%3btopicseen#new

I had quite a long and disappointing trip finding a video framework available for all
important platforms. Now I have dicovered Xuggler and it works..

Games published by our own members! Check 'em out!

[princec]

Looking now... seems promising if I can get the size down a bit.

Cas

[Chagma]

I have reservations about Xuggler namely:

• They say themselves that video playback is experimental at best and not to be used in production.
• They say themselves that video/audio synch doesn't work on Linux and they have no solution.
• It has a massive footprint.
 
 
 
 

[i30817]

What are these guys doing:
http://code.google.com/p/jcodec/

h264 ? Isn't it patented to hell and back? And with the MIT license?

[Chagma]

That looks interesting.  Some recent activity but no releases yet.

[princec]

I tried contacting them but no reply.

Cas

[i30817]

How could you? They have no mail or group. Anyway you can check out the source i guess.

[princec]

The author's got a gmail address somewhere. I dug around a little and found it but no reply from it, so it might be just a "login" sort of address rather than on in actual use.

Cas

[xinaesthetic]

Has anyone tried this library?
http://www.mat.ucsb.edu/~a.forbes/PROCESSING/jmcvideo/jmcvideo.html

The jmcvideo library is a wrapper for playing videos and grabbing video data for any of the formats that the JMC library supports. It works on Windows, OSX, and Linux, using native bindings when possible. The framerates for playback are very fast, up to 300fps fullscreen for multiple YouTube quality .flv videos playing simultaneously. There are two versions of the library, one that interfaces with Processing and one directly for Java using the Java openGL bindings.

By 'the Java openGL bindings' they mean JOGL.

[Chagma]

The question is what is the license?  The JavaFX licenese prohibits separate distribution of any of its components.

[whome]

Has anyone tried this library?
http://www.mat.ucsb.edu/~a.forbes/PROCESSING/jmcvideo/jmcvideo.html  'the Java openGL bindings' they mean JOGL.

I downloaded and tried it, maybe its my +4 year old laptop with ancient OGL driver but it was not a fast playback. Must copy files to a desktop machine when at home and try again. Anyway, functionality point of view it does work and am slightly impressed.

Running demos was a pain first as I know nothing about Processing toolkit. Its a some sort of simplified programming language and graphics runtime environment running on Java. It has some very nice graphics demos, I recommend try it out.

This is what I did.
WindowsXP, Java6
* download processing 1.1 without java
* uninstall to c:/projects/processing-1.1
* git clone jmcvideo project to have sources and dependency libraries
* copy jmcvideo folder from jmcvideo.git/processing/libraries_windows/*  to processing-1.1/libraries/ folder
* download jmcvideo.jar v1.2 and overwrite old jar in a processing-1.1/libraries/jmcvideo/library/
* download few demos from jmcvideo site and unzip to c:/projects/examples/ folder
* run processing IDE and load .pde scripts

I tried few mpeg2, .mp4 and .mkv files. WMVPro video files was audio only and ocassionally crashed. Image quality is not that great but maybe I am spoiled with my DirectX EVR renderer Delphi players.

[Momoko_Fan]

I have jheora (from cortado) working inside jMonkeEngine3. It is easy to get started, but synchronization can be a real pain, especially if you're using multithreading. Often times the audio packets arrive very late in comparison to the video frames and you essentially have no choice but to buffer the video frames or you'll be deadlocked when trying to synchronize them.
Once I get proper syncing working, it will be possible to use this, it "just works". You throw in a theora/vorbis video and it plays just like that, even from a web-server. There are a ton of easy-one-click converters online for all video formats into theora/vorbis. Compression is decent, format is patent-free, only downside is jheora is under LGPL which means you gotta include the jar and source for it separately from app's jar.
So yeah, hopefully I'll be able to get the issues out, and then somebody can port it out of jME3 for general public use, or, there's always the option of everyone switching to jME3

[delt0r]

syncing is in general difficult to get right because you don't control the multiplexing. In some cases players simply fail if the interleaving is too bad and some have a "cache" option that permits the user to set how bad the plexing needs to be before failure.

Whats the performance like? My tests meant that HD was still a no go for jheora. While the C theora player is now doing quite a bit better than x264 for HD (720p) stuff.

ps I am writing a GLSL iDCT for fun, this may make jheora a bit faster if you skip the java iDCT.

[princec]

I've been doing syncing here @ work and basically the way it would appear to do it is you drive everything off of the audio, and buffer video frames behind. With the MPEG2 streams I'm dealing with the number of buffers I need is never more than 3 though, so it's not a big deal for us.

Cas

[Momoko_Fan]

syncing is in general difficult to get right because you don't control the multiplexing. In some cases players simply fail if the interleaving is too bad and some have a "cache" option that permits the user to set how bad the plexing needs to be before failure.

Yeah you gotta cache more if you want better sync.

Whats the performance like? My tests meant that HD was still a no go for jheora. While the C theora player is now doing quite a bit better than x264 for HD (720p) stuff.

It works fine but you gotta queue up more frames for less choppiness. I tried a 720p 30 fps video with a lot of action and it skipped frames in some places. Hopefully a 24 fps video with less moving things will do better.

ps I am writing a GLSL iDCT for fun, this may make jheora a bit faster if you skip the java iDCT.

Lol good luck with that  . DCT is actually used in a lot of places and in various ways inside a video codec, you might need to make a lot of mods to the jheora to make it work.

I've been doing syncing here @ work and basically the way it would appear to do it is you drive everything off of the audio, and buffer video frames behind. With the MPEG2 streams I'm dealing with the number of buffers I need is never more than 3 though, so it's not a big deal for us.

Here I am using 5 buffers and syncing to the system clock and it seems fine for the most part. When you try to sync to audio it's not really accurate because OpenAL queues up data in large pieces rather than in bytes.

[princec]

I'm queueing up audio buffers which contain the audio for 1 video frame; when I detect OpenAL has processed the audio for buffer (by polling every millisecond), I tell OpenGL to draw the next video frame, and by the time the buffer swap's occurred, OpenAL is well underway rendering the sound for that frame. This seems to be working out just great - sound seems to be perfectly synchronized with video.

Cas

[Riven]

I'm queueing up audio buffers which contain the audio for 1 video frame; when I detect OpenAL has processed the audio for buffer (by polling every millisecond), I tell OpenGL to draw the next video frame, and by the time the buffer swap's occurred, OpenAL is well underway rendering the sound for that frame. This seems to be working out just great - sound seems to be perfectly synchronized with video.

Cas

I can imagine it working perfectly, but also a tad slower, like losing a few millis every second in playback speed.

[delt0r]

Lol good luck with that  Tongue. DCT is actually used in a lot of places and in various ways inside a video codec, you might need to make a lot of mods to the jheora to make it work.

In theora its used in one place (I have read the spec and even hacked the jheora a bit a while ago.). This is not h.264. Its a much simpler codec and its decoding performance is quite a bit higher. In particular it doesn't do much intra frame prediction like h.264. This makes a iDCT in glsl theoretically possible. My 4x4 one is working pretty good. But the iDCT was only about 60% of the CPU usage the last time i profiled.

As for syncing, syncing of sound is what quite a few apps do. The reason is that it only matters that the sound matches the video, and a few milliseconds is not going to make any difference (1 ms per sec is about 3.6 seconds per hour.).  Even more important if your sound card clock is not matching the pc clock (some cards this is true) the voice will have drifted compared to the video, so you would need some way of time shifting the audio. Its easier just to match what matters, and use the sound card as the clock.

[princec]

I can imagine it working perfectly, but also a tad slower, like losing a few millis every second in playback speed.

It can't lose any time without gaps in the audio - of which there are of course none. So playback is at precisely the correct rate, which is very accurately controlled by OpenAL's underlying audio backend.

Cas

[Momoko_Fan]

I profiled jheora performance on the HD video:

31% - A method called "ReconInterHalfPixel2"
28% - DeBlocking filter
16% - Huffman and in general decoding from file
9%  - A method called "ExpandKFBBlock"
6%  - A method called "ReconInter"
5%  - YUV to RGB, OggDemux, audio

Notice that IDCT is not even in this list, its only 0.1%...
Btw, the decoding thread is seperate from the GL thread in my video player.

I'm queueing up audio buffers which contain the audio for 1 video frame; when I detect OpenAL has processed the audio for buffer (by polling every millisecond), I tell OpenGL to draw the next video frame, and by the time the buffer swap's occurred, OpenAL is well underway rendering the sound for that frame. This seems to be working out just great - sound seems to be perfectly synchronized with video.

Ah cool. I don't think I can do that tho. I have a better idea on how to make the audio clock more accurate.

[delt0r]

I don't have my profiling results handy. It was last year so the hardware was not that old. So I don't really trust your results. I will believe that halfpel and De blocking filter are high on the list since they really suck up a lot of raw mem bandwidth. But Huffman decoding is really fast even done badly while  YUV->RGB (720p?) in java only using 5%? Even that's surprising (but perhaps not on faster modern CPUs). 

What profiling tool did you use, how long did you collect stats for?  Note i added some of my own timing stuff since these days i am finding hard to get a accurate profiler.  Even then i replace the "slowdown" areas with a no op to see if it really does change the timings.