Java-Gaming.org Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (767)
Games in Android Showcase (229)
games submitted by our members
Games in WIP (854)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Video frame grabbing to bufferedImage  (Read 12125 times)
0 Members and 1 Guest are viewing this topic.
Offline JESTERRRRRR
« Posted 2015-06-25 01:23:16 »

Hi all. I am trying to grab frames from a video and then convert them to a BufferedImage. I will always be grabbing frames in order (so from frame 21 I'll either go to 20 or 22, never off to 100 or anything) and pausing occasionally.

I have achieved this using JCodec, but it is slow (between 3-15 frames a second)

BufferedImage frame = FrameGrab.getFrame(filmFile, frameNumber);

I'm guessing this is because it's working its way from the last ... keyframe or whatever the term is, whereas it would be quicker if I was simply going from frame 1 to frame 2 and so on.

I'm giving up on JCodec - can anyone else recommend what I should use to achieve this? I have read abut Xuggle, JMF and a few others but I keep reading about how they are abandoned.
(Video is mp4 if that's of any use)

Thanks
Offline CopyableCougar4
« Reply #1 - Posted 2015-06-25 01:42:31 »

If I'm reading the JCodec source correctly, each call to FrameGrab.getFrame() seems to decode the video all over again. I think that instead of looking for a new library you should see if you can find some way to not re-decode the file multiple times.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline CopyableCougar4
« Reply #2 - Posted 2015-06-25 01:50:24 »

If you're using the JavaSE AWT backend, then instead of calling
FrameGrab.getFrame(filmFile, frameNumber);


add the methods and static variables:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
private static FrameGrab grab;

public static void initialize(File file) {
   FileChannelWrapper ch = NIOUtils.readableFileChannel(file);
   grab = new FrameGrab(ch);
}

public static BufferedImage getFrame(double second) {
   return ((FrameGrab) grab.seekToSecondPrecise(second)).getFrame();
}


and then call
1  
2  
initialize(filmFile); // call once
BufferedImage frame = getFrame(frameNumber); // call each time


You would have to try it out but that may give you an FPS boost.

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline JESTERRRRRR
« Reply #3 - Posted 2015-06-25 02:38:16 »

Hey, thanks for the reply.

I was really hopeful but unfortunately it gave no speed boost. I tried it with both seekToSecondPrecise and seekToFramePrecise but it's still exactly the same speed.
Offline BurntPizza

« JGO Bitwise Duke »


Medals: 485
Exp: 7 years



« Reply #4 - Posted 2015-06-25 02:50:04 »

I have used Xuggle for this, so it does work, but I cannot remember how fast it was, sorry. I don't recall it being noticeably slow though.

I wouldn't be surprised if nsigma had a good solution to this built in to Praxis Live.
Offline CommanderKeith
« Reply #5 - Posted 2015-06-25 03:14:30 »

I have used Xuggle for this, so it does work, but I cannot remember how fast it was, sorry. I don't recall it being noticeably slow though.

I wouldn't be surprised if nsigma had a good solution to this built in to Praxis Live.

Yes that's true, I think @nsigma uses a java wrapper on gstreamer, perhaps this:
https://github.com/gstreamer-java

Also, @Riven's YUNPM does similar stuff:
http://www.java-gaming.org/index.php?topic=27100.0

Offline nsigma
« Reply #6 - Posted 2015-06-25 08:55:57 »

Yes that's true, I think @nsigma uses a java wrapper on gstreamer, perhaps this:
https://github.com/gstreamer-java

Yes, that's right, Praxis LIVE uses those GStreamer bindings (and I'm one of the repository admins).  I also use the same native GStreamer libs as Processing is using.  It's great and will do everything I need.  However, it's big too!  It might be overkill for what you need.  The video I posted yesterday in the "What I did today" thread is actually using Praxis LIVE's software pipeline rather than its OpenGL one, so is pretty much doing what you're trying to do.

@Riven 's YUMPM might well be worth looking at.  IIRC it's quite a bit smaller, and though there are potential issues in the approach (around timing, frame skipping, etc.) that might actually suit your purposes better (which are?)

The other obvious link missing is the Java bindings to VLC - https://github.com/caprica/vlcj - however, they are GPL licensed so may not suit.

And there's obviously JavaFX, which actually uses GStreamer under the hood, but I have no idea how low level you can get with it.

The odd thing that stands out about that API you're using is that it implies it's creating a new BufferedImage every frame.   You ideally want something that will give you direct access to the decoded pixel data.  There is a Swing player in the GStreamer examples which does something similar to what you want - note this code fills the pixels array of the BufferedImage from the passed in native pointer (IntBuffer).

Praxis LIVE - hybrid visual IDE for (live) creative coding
Offline JESTERRRRRR
« Reply #7 - Posted 2015-06-25 17:59:14 »

My intention is to have a video I can play forwards frame by frame, either at normal x1 speed, or slowed down. I wanted to convert to BufferedImage so I can draw these in a variety of ways (other libs I use make use of them). Ideally I'd like to be able to backwards frame by frame but that's not a priority, just being able to go forward frame by frame at say 60fps is ideal.

I actually have my images all saved as PNG files that I converted into a movie purely for the compression. If there is any alternative (looked into APNG but nothing useful) that would be fine, though I think this is the right way.

I'm currently using Xuggle but without meaning to sound lazy it's proving a pain up the ass to work with, also the whole GPL or build yourself for  LGPL scares me.

I found several mentions of pure Java decoders for mpeg which would be ideal; some were simply publications talking about it but no public release, and the other (http://viper-toolkit.sourceforge.net/products/jmpeg/) I have found literally nothing documentation wise and cannot work them.
Offline Riven
Administrator

« JGO Overlord »


Medals: 1356
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #8 - Posted 2015-06-25 19:47:44 »

With YUNPM you can read/display frames one at a time, at any speed. You have access to this input stream that simply feeds you byte[w*h*3] for every frame. Plain RGB8.

The audio sync, frame skipping and other features are merely feeding off this stream. The logic behind that is simply not active if you're only reading from this raw video stream. Same applies to audio, whatever media file you play, you get access to a plain inputstream with whatever sample rate you prefer.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline JESTERRRRRR
« Reply #9 - Posted 2015-06-25 21:48:14 »

Well I'm going to use YUNPM, will post on its own thread not this one anymore. Thanks for the help
Pages: [1]
  ignore  |  Print  
 
 

 
EgonOlsen (1278 views)
2018-06-10 19:43:48

EgonOlsen (1146 views)
2018-06-10 19:43:44

EgonOlsen (881 views)
2018-06-10 19:43:20

DesertCoockie (1290 views)
2018-05-13 18:23:11

nelsongames (1106 views)
2018-04-24 18:15:36

nelsongames (1337 views)
2018-04-24 18:14:32

ivj94 (2077 views)
2018-03-24 14:47:39

ivj94 (1230 views)
2018-03-24 14:46:31

ivj94 (2188 views)
2018-03-24 14:43:53

Solater (796 views)
2018-03-17 05:04:08
Deployment and Packaging
by mudlee
2018-08-22 18:09:50

Java Gaming Resources
by gouessej
2018-08-22 08:19:41

Deployment and Packaging
by gouessej
2018-08-22 08:04:08

Deployment and Packaging
by gouessej
2018-08-22 08:03:45

Deployment and Packaging
by philfrei
2018-08-20 02:33:38

Deployment and Packaging
by philfrei
2018-08-20 02:29:55

Deployment and Packaging
by philfrei
2018-08-19 23:56:20

Deployment and Packaging
by philfrei
2018-08-19 23:54:46
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!