Using Effects and Filters from JOAL? (Method "alGenFilters" not available)

[bridell]

Hi all,

We've managed to install JOAL 1.1.0 and run some basic stuff - positioning sources and the listener, loading wav files to play and that - it works fine once you figure out how to use everything (the JOAL javadoc is not exactly helpful...)

The thing we can't seem to get to work is using Effects and Filters. As soon as you try to use al.alGenFilters or al.alGenEffects it gives us an error:


Exception in thread "main" net.java.games.joal.ALException: Method "alGenEffects" not available
   at net.java.games.joal.impl.ALImpl.alGenEffects(ALImpl.java:766)


Question: is it at all possible to actually use filters and effects from JOAL? (And if so - how?)


It seems like we're not entirely off - al.alGetString(AL.AL_EXTENSIONS) responds with a list of available extensions:
EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE

PS - We are going to post all our code at http://openresearchplatform.org/ (see the "AW navigator" project), if you're interested in what we're doing, check it there...

/ Fredrik Bridell & Denis Romanovski

[Ultraq]

Well an error of '(OpenAL method name) not available' usually means that  OpenAL method is lacking in your hardware or drivers.  What sound hardware are you using, and have you installed drivers that allow the use of effects & filters?  Effects and filters require the OpenAL Effects Extension (EFX), whose extension string is (I think) AL_EXT_EFX

[Ken Russell]

Sorry, coming to this message a bit late.

I believe JOAL should be exposing the EFX extensions correctly. They were added on UltraQ's request a while ago. UltraQ, you've been successfully using them, correct?

According to efx.h, the extension string associated with these entry points is ALC_EXT_EFX, which I don't see in your extensions string.

Games published by our own members! Check 'em out!

[Ultraq]

I believe JOAL should be exposing the EFX extensions correctly. They were added on UltraQ's request a while ago. UltraQ, you've been successfully using them, correct?

Hmm, at least I thought they did work a while ago.  I just tried to re-create that EFX test case I made, and got pretty much the same error when using alGenAuxiliaryEffectsSlots(): "method unavailable".

As for the ALC_EXT_EFX string not showing-up, I think you have to use alc.alcGetString(ALC_EXTENSIONS).  Although I tried that on my X-Fi just now, and all it returns is the string "None", even though alc.alcIsExtensionPresent(device, "ALC_EXT_EFX") returns true.  Odd.

[Ken Russell]

Hmm. Right now we aren't using alcGetProcAddress / alGetProcAddress to look up entry points, but instead using the analogue of dlsym() on all platforms. I wonder whether newer versions of OpenAL require the use of these functions.

Do you think you might have a chance to look into this? The relevant code is in net/java/games/joal/impl/ALProcAddressLookup.java. It may be a little tricky to change over to using the alternate entry points; you'll need to remove the Ignore lines for alGetProcAddress and alcGetProcAddress in joal.cfg and joal-alc.cfg, and note the comments near the Ignore statements.

[Ultraq]

I'll give it a shot this weekend.

[Ultraq]

Arg, after all that setup and downloading, I can't seem to build JOAL.  Getting errors from the build.xml:

BUILD FAILED
build.xml:696: The following error occurred while executing this line:
build.xml:513: The following error occurred while executing this line:
build.xml:478: cl failed with return code 2

That was when using vc8 as the compiler.  I get a similar error using mingw (the error is at line 458 instead).

[Ken Russell]

What are the errors coming from the C compiler that are emitted above this in the build log? It's probably better if you just attach the entire build log. It builds fine on my machine with Microsoft Visual Studio .NET 2003 (VC7).

[Ultraq]

Alright, I managed to fix it up and build JOAL from the source.

Unfortunately, I seem to keep building bad DLLs:

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15

Exception in thread "main" net.java.games.joal.ALException: java.lang.UnsatisfiedLinkError: D:\Programming\Libraries\joal\joal_native.dll: Can't find dependent libraries    at net.java.games.joal.ALFactory.initialize(ALFactory.java:58)    at net.java.games.joal.ALFactory.getALC(ALFactory.java:83)    at TestAL.main(TestAL.java:12) Caused by: java.lang.UnsatisfiedLinkError: D:\Programming\Libraries\joal\joal_native.dll: Can't find dependent libraries    at java.lang.ClassLoader$NativeLibrary.load(Native Method)    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1751)    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1676)    at java.lang.Runtime.loadLibrary0(Runtime.java:822)    at java.lang.System.loadLibrary(System.java:993)    at net.java.games.joal.impl.NativeLibLoader$1.run(NativeLibLoader.java:69)    at java.security.AccessController.doPrivileged(Native Method)    at net.java.games.joal.impl.NativeLibLoader.load(NativeLibLoader.java:62)    at net.java.games.joal.ALFactory.initialize(ALFactory.java:54)    ... 2 more

The GlueGen and JOAL DLLs built on my machine are very different in size from the ones provided for download from the JOAL pages.  I get the above error regardless of whether I've commented the Ignore al(c)GetProcAddress lines in the joal(-alc).cfg files or not.

[Ken Russell]

What does "dumpbin /dependents joal_native.dll" report?

I'm guessing it's because of the C runtime library not being visible in a directory in your PATH.

Games published by our own members! Check 'em out!

[Ultraq]

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15

Dump of file joal_native.dll File Type: DLL   Image has the following dependencies:     MSVCR80.dll     KERNEL32.dll   Summary         1000 .data         6000 .rdata         1000 .reloc         4000 .text

msvcr80.dll is not on the PATH, but installed in C:\Windows\WinSxS\x86_Microsoft.VC80.CRT_...

I went and Googled the missing msvcr80.dll issue, and it seems there was some change in VC8 / VS2005 where you have to reference the msvcr80.dll via a manifest file built into the DLL.  gluegen-rt.dll and joal_native.dll did not have this manifest file (I checked using mt -inputresource:joal_native.dll;#1 -out:joal_native.dll.manifest for both gluegen-rt.dll and joal_native.dll), even though a manifest was generated during in /build/obj.
Anyway, I fixed that (copied the manifest to the directory of javaw.exe and renamed it javaw.exe.manifest).

I also had to fix a compile error in some of the generated java files, but it was an easy fix (couldn't resolve ByteBuffer and ByteOrder in 2 lines of ALImpl.java)

After all that, I still get Method "alGenAuxiliaryEffectSlots" not available in my test class after commenting the Ignore al(c)GetProcAddress line in the joal(-alc).cfg files and building JOAL.  Is there more to it than just commenting-out those lines?

[Ken Russell]

Yes, there's more to it. Class net.java.games.joal.impl.ALProcAddressLookup defines how the procedure addresses for the various OpenAL entry points are looked up. Note that right now it's using the GlueGen NativeLibrary class to do the lookup. The question is whether this can be changed to use alGetProcAddress / alcGetProcAddress. There may be a bootstrapping issue where alGetProcAddress / alcGetProcAddress may need to be looked up first using the NativeLibrary class, because otherwise their procedure addresses won't be available. Take a look at the generated code and let me know if you need help understanding what's going on in the generated glue code. Thanks for working on this; it will be a big help to have someone else know what is going on in the autogenerated code.

[Ultraq]

Phew, finally figured it out.
You were right about the bootstrapping issues: I tried at first to use alGetProcAddress() during the standard initialization of the function lookup tables, but alGetProcAddress() wasn't itself ready.  So I just wrote another method which I called from my app, to re-populate the addresses of the EFX extensions after an AL context was created.

More bad news though; for all of the functions, the address returned by alGetProcAddress() was the same (except for 2 in the example below).  Basically, it looked like this:

1   2   3   4   5   6   7   8   9   10   11   12   13

Looking-up EFX function pointers  - Address of alAuxiliaryEffectSlotf: ff8b  - Address of alAuxiliaryEffectSlotfv: ff8b  - Address of alAuxiliaryEffectSloti: ff8b  - Address of alAuxiliaryEffectSlotiv: ff8b  - Address of alDeleteAuxiliaryEffectSlots: ff8b  - Address of alDeleteEffects: 2068  - Address of alDeleteFilters: 2068  - Address of alEffectf: ff8b  - Address of alEffectfv: ff8b  - Address of alEffecti: ff8b  - Address of alEffectiv: ff8b  - ...

And so on.

I noticed some comments in the ALProcAddressLookup class, mentioning that the alGetProcAddress function was broken.  I took a look at the C++ demos from the OpenAL SDK which use EFX, and they manage to get their functions via alGetProcAddress, and run just fine:

1   2   3   4   5   6   7   8   9

// Get function pointers alGenEffects = (LPALGENEFFECTS)alGetProcAddress("alGenEffects"); alDeleteEffects = (LPALDELETEEFFECTS )alGetProcAddress("alDeleteEffects"); alIsEffect = (LPALISEFFECT )alGetProcAddress("alIsEffect"); alEffecti = (LPALEFFECTI)alGetProcAddress("alEffecti"); alEffectiv = (LPALEFFECTIV)alGetProcAddress("alEffectiv"); alEffectf = (LPALEFFECTF)alGetProcAddress("alEffectf"); alEffectfv = (LPALEFFECTFV)alGetProcAddress("alEffectfv"); ...

[Ken Russell]

Have you tried hacking the autogenerated C code and putting in a printf down there (like printf("alGetProcAddress(%s) = 0x%p\n", addr) to see whether the value is being inadvertently truncated somewhere along the way?

Are you sure you're calling the function from the thread with the OpenAL context current?

[Ken Russell]

Also, check to make sure you're picking up the same OpenAL implementation from the C++ examples and from JOAL. Note that JOAL comes with its own OpenAL implementation (taken from one of the OpenAL 1.1 redistributable installers) -- see make/lib/windows-i586, and this might be out of date or buggy.

[Ultraq]

Yes, the function is being used in the same thread, after the ALC.alcMakeCurrentContext() is called.
I've only included the joal_native.dll library, letting it find the OpenAL32.dll in my Windows\system32 directory.

The truncation was an error on my part, but even upon fixing that, the results seemed the same.  The printf() did reveal a lot: the addresses read by Java are definitely not what the alcGetProcAddress() finds in the C++ code:

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27

Looking-up EFX function pointers C++ - alGetProcAddress = 0x0B0F8493 Java - Address of alAuxiliaryEffectSlotf: 0x8b55ff8b C++ - alGetProcAddress = 0x0B0F8AA7 Java - Address of alAuxiliaryEffectSlotfv: 0x8b55ff8b C++ - alGetProcAddress = 0x0B0F87BC Java - Address of alAuxiliaryEffectSloti: 0x8b55ff8b C++ - alGetProcAddress = 0x0B0F8DE2 Java - Address of alAuxiliaryEffectSlotiv: 0x8b55ff8b C++ - alGetProcAddress = 0x0B0FA807 Java - Address of alDeleteAuxiliaryEffectSlots: 0x6855ff8b C++ - alGetProcAddress = 0x0B0FA250 Java - Address of alDeleteEffects: 0x10f12068 C++ - alGetProcAddress = 0x0B0FA482 Java - Address of alDeleteFilters: 0x10f12068 C++ - alGetProcAddress = 0x0B0F7C8F Java - Address of alEffectf: 0x8b55ff8b ...

[Ken Russell]

Can I take a look at your changes? Probably the easiest way would be for you to file a bug with the Issue Tracker on the JOAL home page and attach the output of "cvs diff" on your workspace. You'll need to be an Observer of the project if you aren't already.

[Ultraq]

Yeah, another pair of eyes would be good.  I'll sort-out that project observer thing, but in the interim, here's the changes:

The generated C++ alGetProcAddress() method, plus that printf() statement:

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29

/*   Java->C glue code:  *   Java package: net.java.games.joal.impl.ALImpl  *    Java method: java.nio.ByteBuffer dispatch_alGetProcAddress(java.lang.String fname)  *     C function: ALproc alGetProcAddress(const ALchar *  fname);  */ JNIEXPORT jobject JNICALL Java_net_java_games_joal_impl_ALImpl_dispatch_1alGetProcAddress0__Ljava_lang_String_2J(JNIEnv *env, jobject _unused, jstring fname, jlong procAddress) {   LPALGETPROCADDRESS ptr_alGetProcAddress;   const char* _UTF8fname = NULL;   ALproc _res;   ptr_alGetProcAddress = (LPALGETPROCADDRESS) (intptr_t) procAddress;   assert(ptr_alGetProcAddress != NULL);   if (fname != NULL) {       _UTF8fname = (*env)->GetStringUTFChars(env, fname, (jboolean*)NULL);     if (_UTF8fname == NULL) {       (*env)->ThrowNew(env, (*env)->FindClass(env, "java/lang/OutOfMemoryError"),                        "Failed to get UTF-8 chars for argument \"fname\" in native dispatcher for \"dispatch_alGetProcAddress\"");       return 0;     }   }   _res = (* ptr_alGetProcAddress) ((ALchar *) _UTF8fname);   printf("C++ - alGetProcAddress = 0x%p\n", _res);   if (fname != NULL) {     (*env)->ReleaseStringUTFChars(env, fname, _UTF8fname);   }   if (_res == NULL) return NULL;   return (*env)->NewDirectByteBuffer(env, _res, sizeof(ALproc)); }

New method in net.java.games.joal.impl.ALProcAddressLookup, called from my test case after an OpenAL context had been made current.  Tries to set the address values in the ALProcAddressTable class using AL.alGetProcAddress()

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42

public static void repopALProcAddrTable() {     String addrOfPrefix = "_addressof_";     AL al = ALFactory.getAL();     System.out.println("\nLooking-up EFX function pointers");     for (Field field: ALProcAddressTable.class.getFields()) {         // Skip non-address fields        String fieldname = field.getName();         if (!fieldname.startsWith(addrOfPrefix)) {             continue;         }         try {             String functionname = fieldname.substring(addrOfPrefix.length());             long fieldval = field.getLong(alTable);             // Skip fields which have already been valued            if (fieldval != 0) {                 continue;             }             // Get the address            ByteBuffer procAddress = al.alGetProcAddress(functionname);             long procAddressVal = 0;             if (procAddress.limit() == 8) {                 procAddressVal = procAddress.getLong();             }             else {                 procAddressVal = procAddress.getInt();                 if (procAddressVal < 0) {                     procAddressVal -= 0xffffffff00000000L;                 }             }             System.out.println("Java - Address of " + functionname + ": 0x" + Long.toHexString(procAddressVal) + "\n");             field.setLong(alTable, procAddressVal);         }         catch (Exception ex) {             throw new RuntimeException("Unable to repopulate ALProcAddressTable values");         }     } }

[Ken Russell]

Whoa, sorry -- this is very wrong. You need to specify "Opaque long ALproc" in your .cfg file.

[Tramboturbiener]

Hi all,

I tried to use Effects and Filters from JOAL but unfortunately unsuccessfully. And that is why I want to ask you whether this bug can be fixed in a shot time (one week or two) or is it not possibly? I'm sorry if that sounds impudent but I need this framework (and above all the Effects and Filters) for my diploma.

Thanks.

[Ultraq]

Well, good news: the change from AL.alGetProcAddress() to return a long via the C++ jlong type instead of trying to get the address from the direct buffer, fixed it up!  I guess the (*env)->NewDirectByteBuffer() kept returning the address of the buffer, not the address of the function itself?  I dunno, JNI isn't exactly my specialty.

@Tramboturbiener - if you want to fix this up on your own, you could probably do something similar to what I've been instructed to do.  It's probably not the most elegant solution, but I dunno if Ken can get it fixed right this instant (something to do with JavaOne)   In short:
 - you need to be able to build the project from the source.  There are instructions for that in the Readme.txt that comes with the source archives.
 - comment the lines that say something like Ignore al(c)GetProcAddress from the joal(-alc).cfg files
 - uncomment the line Opaque long ALproc from the joal.cfg file
 - add a method like that in the code snippet below to the net.java.games.joal.impl.ALProcAddressTable class
 - add a call to that method within the resetALProcAddressTable() method, after the line which has ProcAddressHelper.resetProcAddressTable(alTable, lookup);, of the ALProcAddressTable.
 - rebuild

I did some other non-code stuff, but that was more to do with getting the thing to work properly using the VC8 compiler more than anything else.

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35

/** * Retrieves the values of the OpenAL functions using alGetProcAddress(). */ private static void useALGetProcAddress() {     String addrOfPrefix = "_addressof_";     AL al = ALFactory.getAL();     System.out.println("\nLooking-up EFX function pointers");     for (Field field: ALProcAddressTable.class.getFields()) {         // Skip non-address fields        String fieldname = field.getName();         if (!fieldname.startsWith(addrOfPrefix)) {             continue;         }         try {             String functionname = fieldname.substring(addrOfPrefix.length());             long fieldval = field.getLong(alTable);             // Skip fields which have already been valued            if (fieldval != 0) {                 continue;             }             // Get the address            long procAddressVal = al.alGetProcAddress(functionname);             System.out.println("Address of " + functionname + ": 0x" + Long.toHexString(procAddressVal));             field.setLong(alTable, procAddressVal);         }         catch (Exception ex) {             throw new RuntimeException("Unable to repopulate ALProcAddressTable values");         }     } }

[Tramboturbiener]

@Ultraq: well, it was not easy, but now I can build the project.
 I don't really understand where the classes are. Do you mean this package (file):  ...\build\gensrc\classes\net\java\games\joal\impl\ALProcAddressTable.java ? It is a auto-generated file and I can write what I want, it will be rewriting by building-process. Anyway I have tried to edit this file. After rebuilding of the project I still have "not available Exception". The code of  ALProcAddressTable.java (before rebuilding):

1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51   52   53   54   55   56   57   58

public long getAddressFor(String functionName) {         String addressFieldName = com.sun.gluegen.runtime.ProcAddressHelper.PROCADDRESS_VAR_PREFIX + functionName;         try {             java.lang.reflect.Field addressField = getClass().getField(addressFieldName);             return addressField.getLong(this);         } catch (Exception e) {             // The user is calling a bogus function or one which is not            // runtime linked            throw new RuntimeException(                     "WARNING: Address query failed for \"" + functionName +                     "\"; it's either statically linked or is not a known " +                     "function", e);         }     }             /**      * Retrieves the values of the OpenAL functions using alGetProcAddress().      */     private static void useALGetProcAddress() {                 String addrOfPrefix = "_addressof_";         AL al = ALFactory.getAL();         System.out.println("\nLooking-up EFX function pointers");                 for (Field field: ALProcAddressTable.class.getFields()) {                         // Skip non-address fields            String fieldname = field.getName();             if (!fieldname.startsWith(addrOfPrefix)) {                 continue;             }             try {                 String functionname = fieldname.substring(addrOfPrefix.length());                 long fieldval = field.getLong(alTable);                                 // Skip fields which have already been valued                if (fieldval != 0) {                     continue;                 }                                 // Get the address                long procAddressVal = al.alGetProcAddress(functionname);                 System.out.println("Address of " + functionname + ": 0x" + Long.toHexString(procAddressVal));                 field.setLong(alTable, procAddressVal);             } catch (Exception ex) {                 throw new RuntimeException("Unable to repopulate ALProcAddressTable values");             }         }     }         /**      * my own method      */     private static void resetALProcAddressTable() {         com.sun.gluegen.runtime.ProcAddressHelper.resetProcAddressTable(alTable, lookup);         useALGetProcAddress();     }

Do I anything wrong or ist that a wrong file?

[Ultraq]

Sorry, I did name the wrong file.

The file to edit is ALProcAddressLookup.java, in src/java/net/java/games/joal/impl  It is not an auto-generated file.  I'll attach the file, just to make sure that I give the right instructions this time.  The lines I've added are at:
 - 69-71: This is just a System.out I to see what OpenAL functions were being done before the modifications.  You can remove this if you want.
 - 89-93: Most of this is a comment, and the only line of code is a call to that custom method to get the addresses of the EFX functions.
 - 127-161: The custom method I mentioned.

[Ken Russell]

Ultraq: thanks for investigating this. Very nice work. Just to confirm, using this technique the effects stuff is now working?

I've filed a bug about this on the JOAL Issue Tracker and will incorporate your changes as soon as possible.

[Tramboturbiener]

@Ultraq: Thank you!!! I fixed this and it works fine!!! All effects and filters are available!

P.S.: I had also the same problem with msvcr80.dll and I use Netbeans, so I must create java.exe.manifest (and not javaw.exe.manifest)

[Ultraq]

@Ken: Yes, I've even been able to translate one of the EFX examples from the OpenAL 1.1 SDK, into Java, just to make sure. (attached if anybody watching this thread is interested, although it needs the footsteps.wav sample from the SDK).  [EDIT]: And it seems it's working for Tramboturbiener too

@Tramboturbiener: If you want to avoid having to use a java.exe.manifest, you can incorporate the manifest files, which were generated in the /build/obj directory, into the DLLs themselves by using the following in the VS2005 command prompt:

mt -manifest gluegen-rt.dll -outputresource:gluegen-rt.dll;#2

(do the same for joal_native.dll)

[Ken Russell]

@Ultraq: sorry for the long delay. I hadn't taken a close look at your patches until now and they're perfectly correct. I misread them earlier and thought it was necessary to do a manual reset of the ALProcAddressTable from the end user's code, which would have been gross.

This and some other new functionality will be out soon -- tomorrow, if tonight's nightly builds succeed.

[Ultraq]

Oh cool, thanks Ken.  I was about to post that I hadn't made any progress on your suggestions in the other thread, but it seems you've sorted things out.