coordinating sound f/x with visual f/x

[philfrei]

I'm working on my first 2D game. It is more a puzzle than an action game--tiles are being dragged and dropped, and one is trying to achieve a certain objective pattern.

When the puzzle is completed, I launch a visual animation (glows for the various tiles) and some sound effects. SOMETIMES, the performance stutters. I reopened the program today, and didn't recreate the problem in a dozen tries. But it sure was happening last night, and has been for a while.

The timing of the visuals is handled by a pre-exising util.Timer (my game loop timer) which does its thing by calling code which increments indexes into the arrays of graphics, then calling a repaint(). This code is "launched" by flipping a boolean. (The "glow cycle" references a premade set of graphics with varying alpha values at a slightly modified radial gradient.)

The timing of the sounds is handled by a freshly made util.Timer, where I create, on the fly a set of RemindTasks that each pop an index off of an array, and send a call to play the referenced object. Each of the sounds will have already been played once, prior to this call. But I haven't found a way to "index" into an array of sounds like I do with graphics, that allows a single WAV to be played while overlapping. So new sounds are created on the fly from the resource files.

Mostly this works just fine, but sometimes there is a jerkiness. I've seen various things written about problems with util.Timers and the Java.sound package, and am wondering if these are due to inherent limitations (related to garbage collection or time needed for loading my sound files), or to complexities in how these items have to be handled in order to use them correctly.

As I write this, it occurs to me maybe I don't need the second timer for launching the sounds as a sequence. I guess I could make the "run-once" remind tasks on the "game loop timer" and avoid the cost of making a new timer. Also, I might be able to make a special case for these sounds and eliminate situations where an overlap might occur, thus allowing for preloading them...

How difficult, in general is it to switch to another sound library? What is recommended? Is the reason "easier programming" (a great reason!) or is there a performance or reliability pickup in how sounds are played back by the JVM?

Are there other Timer() objects that run more reliably? I know there is a Swing.Timer, but I don't know how it differs in terms of performance and reliability with the util.Timer.

Thanks for input! If any of the above needs to be supplemented by code examples, you'll let me know, yes? I'm also working at trying to get an "in progress" Applet online. It is complicated as it involves loading a puzzle (the puzzles are beyond my capability to generate automatically) and some problem situations only arise upon completion (so I'll also need the instructions to be up and a way of showing/revealing the solutions). Hah. You get what seems to be 80% done, and then the last 20% takes 80% of the total project time. Yes?

[Captain Awesome]

Since you are calling repaint() I suppose you are using passive rendering, a JPanel perhaps?
If you use active rendering through a Canvas instead you will be able to paint stuff directly to the screen instead of waiting for the EDT to do it. That will probably make coordinating the sound with the graphics a lot easier

[philfrei]

Thanks. That is a reasonable suggestion, and yes I am drawing on a JPanel. I admit I'm kind of avoiding the Canvas option, though, because of an intuition that this animation task should be simple enough to work on the EDT, and if not, then I need to understand the EDT and its tolerances better, on general principles. I think that I'm enough of a beginner that I should first check if I'm doing something wrong rather than conclude that these Java functions are unreliable.

And, in fact, I did just come up with the probably cause. There is a sound event when a branch of the puzzle is completed, and an event when the entire puzzle is completed. If multiple branches are completed on the very last "play," two or three sound sequences are all being launched at the same time, instead of just the "game winning" sound sequence. So...duh. I will fix this and maybe the problems will disappear.

Games published by our own members! Check 'em out!

[CaptainJester]

You can't rely on the EDT for any time critical code.  You have no idea when it will run.

[philfrei]

You can't rely on the EDT for any time critical code.  You have no idea when it will run.

Well, I can see your point, but don't you think "no idea" is an extreme statement? I'd like to have a better understanding of the tolerances. Maybe if one is not "stressing out" the EDT with a lot of work, the amount of variability can be expected to be within X milliseconds with a given probability. The question is--what is that number X? And what sorts of things affect the size range of X. For example, if one is scrupulous with sending a bare minimum of tasks to the EDT, can one reasonably expect the timing to be within a dozen or so milliseconds of its target?

Anyway, I'm only doing a simple 2D game, and it turns out that eliminating the simultaneous calls (mentioned in my last post) has brought the timing back to within reasonableness. If I were doing high speed or 3D, I could see the need to avoid the EDT.

[erikd]

But you're still doing animations, and that will always be more reliable and just feel better with active rendering.
How exactly are you doing your sound effects?

[philfrei]

How exactly are you doing your sound effects?

I took some gongs and chimes I had previously made with an FM7 soft synth plus some effects, and exported something like a dozen .wav files of about 3 seconds each. I also have a pad that works as a loop (about 30 seconds). All of these are "resource" files, and played via javax.sound.sampled library code. Some use a WindChime effect (semi-random), some are tied to events--new threads are always spawned to allow overlapping Gongs, so each clip has a chance to decay naturally. Had a LOT of fun with memory leaks and JProfiler with that!

The looper for the pad has a second thread so the beginning can overlap the end (overlap = 125 msec? estimate) and sound seamless. It's kind of a neat acoustic environment, different from most games, and I hope to have it in the demo area within a couple days as a work-in-progress. ww.hexara.com/game.html [it takes about 3 minutes to load, due to 15MB of .wavs--less than 2 minutes worth!]

I'm having a bit of a problem with changing volumes on continuous sound, though. Have to use a really small increments. At the moment volume changes at a maximum rate of 0.02 (within allowable range of -80.0 to +6.0) every 20 msec or so, and it doesn't seem to be clicking.

I look forward to checking out other sound libraries, but wanted to learn how to do it with what java itself provides, first (this is a learning project, after all). I'm curious how overlaps & mid-sound volume changes will be handled.