Java-Gaming.org Hi !
Featured games (81)
games approved by the League of Dukes
Games in Showcase (513)
Games in Android Showcase (119)
games submitted by our members
Games in WIP (577)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Java Audio Metronome | Timing and Speed Problems  (Read 9457 times)
0 Members and 1 Guest are viewing this topic.
Offline Tekkerue
« Posted 2014-06-17 23:28:55 »

Hi all,

I’m starting to work on a music/metronome application in Java and I’m running into some problems with the timing and speed.

My test program plays two sine wave tones together at regular intervals and updates two visual displays (one display for each tone). At slower speeds the two tones will play in sync for a little while, then slightly out of sync for a few beats, then they will play back in sync again, and this cycle continues. At faster speeds, the synchronization is all over the place.

From researching good metronome programming, I found that Thread.sleep() is horrible for timing, so I completely avoided that and went with checking System.nanoTime() to determine when the sounds should play.

I’m using AudioSystem’s SourceDataLine for my audio player and I’m using a thread for each tone that constantly polls System.nanoTime() in order to determine when to play the tones and update the visual display. I create a new SourceDataLine and delete the previous one each time a sound plays, because the volume fluctuates if I leave the line open and keep playing sounds on the same line...I’ve read that you’re not supposed to leave the SourceDataLine open when not playing anything, but I wanted to see what happened if I did. Unfortunately the process of creating a new player each time is costing a lot of time, I timed the process for creating the player and it was upwards of about 270ms. However, I’m not sure what else to do with it.

In theory this seemed like a good method for getting each sound to play on time, but in reality it seems like the audio hardware isn’t able to keep up with the method that I’m using and is causing timing problems.

At the moment this is just a simple test in Java, but my goal is to create my app on mobile devices (Android, iOS, Windows Phone, etc)...however my current method isn’t even keeping perfect time on a PC, so I’m worried that certain mobile devices with limited resources will have even more timing problems. I will also be adding more sounds to it to create more complex rhythms, so it needs to be able to handle multiple sounds going simultaneously without lagging.

Another problem I’m having is that the max tempo is controlled by the length of the tone since the tones don’t overlap each other. I tried adding additional threads so that every tone that played would get its own thread...but that really screwed up the timing, so I took it out. I would like to have a way to overlap the previous sound to allow for much higher tempos.

One final point...my first attempt I used a larger buffer (loading multiple tones separated by silence) and played the buffer using SourceDataLine. However, the problem with this method was that I wasn’t able to know exactly when each tone played so that I could update the beat counter visual display. I do have a working example using this method, so if needed I could post my code for this as well.

Any help getting these timing and speed issues straightened out would be greatly appreciated. Thanks!

Here is my code...

SoundTest.java
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;

import java.io.*;
import javax.sound.sampled.*;

public class SoundTest implements ActionListener {
   static SoundTest soundTest;
   
   
   // ENABLE/DISABLE SOUNDS
   boolean playSound1   = true;
   boolean playSound2   = true;
   
   
   JFrame mainFrame;
   JPanel mainContent;
   JPanel center;
   JButton buttonPlay;
   JLabel[] beatDisplay;
   
   int sampleRate = 44100;
   long startTime;  
   SourceDataLine line = null;  
   int tickLength;
   boolean playing = false;
   
   SoundElement sound01;
   SoundElement sound02;
   
   public static void main (String[] args) {      
      soundTest = new SoundTest();
     
      SwingUtilities.invokeLater(new Runnable() { public void run() {
         soundTest.gui_CreateAndShow();
      }});
   }
   
   public void gui_CreateAndShow() {
      gui_FrameAndContentPanel();
      gui_AddContent();
   }
   
   public void gui_FrameAndContentPanel() {
      mainContent = new JPanel();
      mainContent.setLayout(new BorderLayout());
      mainContent.setPreferredSize(new Dimension(400,400));
      mainContent.setOpaque(true);
     
      mainFrame = new JFrame("Sound Test");            
      mainFrame.setContentPane(mainContent);            
      mainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      mainFrame.pack();
      mainFrame.setVisible(true);
   }
   
   public void gui_AddContent() {
      JPanel center = new JPanel();
      center.setOpaque(true);
     
      buttonPlay = new JButton("PLAY / STOP");
      buttonPlay.setActionCommand("play");
      buttonPlay.addActionListener(this);
      buttonPlay.setPreferredSize(new Dimension(200, 50));
     
      beatDisplay = new JLabel[2];
     
      beatDisplay[0] = new JLabel("~ SOUND 1 ~");
      beatDisplay[0].setPreferredSize(new Dimension(400, 50));
      beatDisplay[0].setHorizontalAlignment(SwingConstants.CENTER);
      beatDisplay[0].setOpaque(true);
      beatDisplay[0].setFont(new Font("Serif", Font.BOLD, 40));
     
      beatDisplay[1] = new JLabel("~ SOUND 2 ~");
      beatDisplay[1].setPreferredSize(new Dimension(400, 50));
      beatDisplay[1].setHorizontalAlignment(SwingConstants.CENTER);
      beatDisplay[1].setOpaque(true);
      beatDisplay[1].setFont(new Font("Serif", Font.BOLD, 40));
     
      center.add(buttonPlay);
      center.add(beatDisplay[0]);
      center.add(beatDisplay[1]);
      mainContent.add(center, BorderLayout.CENTER);
   }
   
   public void actionPerformed(ActionEvent e) {
      if (!playing) {
         playing = true;
         
         if (playSound1)
            sound01 = new SoundElement(this, "Sound1", 0, 800);
         if (playSound2)
            sound02 = new SoundElement(this, "Sound2", 1, 1200);
           
         startTime = System.nanoTime();
         
         if (playSound1)
            new Thread(sound01).start();
         if (playSound2)
            new Thread(sound02).start();
      }
      else {
         playing = false;
      }
   }
}


SoundElement.java
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;

import java.io.*;
import javax.sound.sampled.*;

public class SoundElement implements Runnable {
   SoundTest soundTest;
   
   
   // TEMPO CHANGE
   // 750000000=80bpm | 300000000=200bpm | 200000000=300bpm
   long nsDelay = 750000000;
   
   
   long before;
   long after;
   long diff;
   
   String name="";
   int soundNumber;
   int clickLength = 4100;  
   byte[] audioFile;
   double clickFrequency;
   SourceDataLine line = null;
   long audioFilePlay;
   
   // GUI
   int beatCount=1;
   boolean green=true;
   
   public SoundElement(SoundTest soundTestIn, String nameIn, int soundNumberIn, double clickFrequencyIn){
      soundTest = soundTestIn;
      name = nameIn;
      soundNumber = soundNumberIn;
      clickFrequency = clickFrequencyIn;
      generateAudioFile();
   }
   
   public void generateAudioFile(){
      audioFile = new byte[clickLength * 2];
      double temp;
      short maxSample;
     
      int p=0;
      for (int i = 0; i < audioFile.length;){
         temp = Math.sin(2 * Math.PI * p++ / (soundTest.sampleRate/clickFrequency));
         maxSample = (short) (temp * Short.MAX_VALUE);
         audioFile[i++] = (byte) (maxSample & 0x00ff);        
         audioFile[i++] = (byte) ((maxSample & 0xff00) >>> 8);
      }
   }
   
   public void run() {
      createPlayer();
      audioFilePlay = soundTest.startTime + nsDelay;
     
      while (soundTest.playing){
         if (System.nanoTime() >= audioFilePlay){
            play();
            gui_UpdateBeatCounter();            
            destroyPlayer();
            createPlayer();
            audioFilePlay += nsDelay;
         }
      }
      try { destroyPlayer(); } catch (Exception e) { }
   }
   
   public void createPlayer(){
      AudioFormat af = new AudioFormat(soundTest.sampleRate, 16, 1, true, false);
      try {
         line = AudioSystem.getSourceDataLine(af);
         line.open(af);
         line.start();
      }
      catch (Exception ex) { ex.printStackTrace(); }
   }
   
   public void play(){
      line.write(audioFile, 0, audioFile.length);
   }
   
   public void destroyPlayer(){
      line.drain();
      line.close();
   }
   
   public void gui_UpdateBeatCounter(){
      soundTest.beatDisplay[soundNumber].setText("" + beatCount);
     
      if (green){
         soundTest.beatDisplay[soundNumber].setBackground(Color.GREEN);
         green = false;
      }
      else{
         soundTest.beatDisplay[soundNumber].setBackground(Color.YELLOW);
         green = true;
      }
     
      if (beatCount == 4)
         beatCount = 1;
      else
         beatCount++;
   }
}
Offline philfrei
« Reply #1 - Posted 2014-06-18 01:16:09 »

Check out this really useful article on Java timing for music and sound:

http://quod.lib.umich.edu/cgi/p/pod/dod-idx?c=icmc;idno=bbp2372.2007.131
Quote
REAL-TIME, LOW LATENCY AUDIO PROCESSING IN JAVA
Nicolas Juillerat, Stefan Muller Arisona, Simon Schubiger-Banz
ETH Zurich
Computer Systems Institute

Secondly, you will probably want to keep a SourceDataLine open and running constantly, for use as the output for a "mixer," in the studio audio sense of the word mixer, not the Java sense. Have it play "silence" when there is nothing going else to play.

If you don't want to write your own audio mixer, TinySound (on github, but also has posts here on JGO) has been written and already does this. With this library (pretty small, easy to read) you'll be able to overlap sounds.

I wrote my own audio playback library also, but haven't made it available, and added an "event system" for it a couple months ago. It checks a queue every frame for "trigger" commands. Something like this would be ideal for your use. To code this, you have to play your sounds one frame at a time, and check if on that frame count there's another sound that is scheduled to be played. This is the only way I know to be really accurate, as Java does a good job of keeping sounds playing correctly (when you are at the stage of progressively loading the SourceDataLine), but is less accurate (doesn't make real time guarantees) about issues that pertain to multiple threads or other issues mentioned in the article I cited.

There's a trick for improving the accuracy of the sleep timer in windows. If you run it the command Thread.sleep(Long.MAX_VALUE) (on its own thread, of course!) the Windows OS will use its higher resolution clock (same one used for nanotime) for sleep and getmillis. But issues pertaining to the buffer used for playback and to thread management guarantees will still hinder accuracy.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #2 - Posted 2014-06-18 02:42:25 »

Hi philfrei, thanks for the response! Smiley

If at all possible I'd prefer not to use 3rd party libraries, because like I mentioned before my goal is to eventually create my app on mobile devices. I'm starting out and getting a working example in Java because I'm familiar with Java, then I'll attempt to move over to mobile devices. I'd like to have my fingers on the inner workings of the low level audio stuff so I know how to translate it over onto different mobile platforms. I feel like I'd get totally stuck if I tried to use a 3rd party library because I'd have no idea how to make it work on other devices. I will certainly check out the code to see what I can understand and learn from it. I like the part about “small and easy to read”, so that’s encouraging. Some of the code I’ve looked at goes way over my head...I’m a musician long before a programmer. lol

I’ve thought about using “silence” in between playing the tones but I wasn't sure exactly how to go about executing it. Theoretically I could do this in each of the threads I’m running by feeding SourceDataLine zeros if the play method isn’t called. But then I’m not sure how to have it respond quickly when data needed to be played while preventing it from emptying the buffer and then running into the same problem I had when I left the line open when there was no data to play.

But your suggestion is to use a single SourceDataLine instead and then create a mixer that would handle combing all of the sounds and silence? Is something like that feasible for a DSP newbie? This is my first time working with actual audio...I’ve done a few music related MIDI applets in Java but MIDI had its own sequencer, so timing MIDI wasn’t difficult like audio is turning out to be.

Thanks again!
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline philfrei
« Reply #3 - Posted 2014-06-18 06:26:49 »

I am also a musician first. I got into Java only four or five years ago, am self taught, and am also new to DSP. I've managed to teach myself a few things but also have had lots of help!

I also preferred to write things myself, in order to learn and understand, so I appreciate your wanting to do so as well. I'm in the process of working on the problem of converting my Java sound programs to iOS and Android, also. So maybe there is room to put heads together on this, down the road. (I think I've located the equivalent of the SourceDataLine output in both realms, so my framework code (all core Java) should run on these other system, but haven't tried testing this yet.)

Do read the article I recommended! Many issues pertain to what Java can and can't do. I prefer not to get into specifics about handling latency issues until you at least give it a go. It wasn't the easiest read, but having it to refer to will be helpful.

A simple mixer is definitely feasible. The DSP being about as easy as DSP gets. The main task is the combining of PCM values, and that is just addition! Converting to and from bytes has to be done. There's some tweaking to make sure you don't overflow. Handling volume and panning can be kept very simple/stupid, at least initially.

Another excellent audio programmer will probably reply soon as well (he's in Australia): nsigma. He has an excellent set of tools written and repeatedly given great advice as well as being generous in opening up his source code.

I just remembered, I started this thread when I started my second pass on a mixer:
http://www.java-gaming.org/topics/simple-audio-mixer-2nd-pass/27943/view.html
The first post has a link with a jar with source code. The project has evolved a bit since then, and some of the decisions I made have not had unanimous support (!) but the code might still be helpful for you to look over.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #4 - Posted 2014-06-18 08:46:45 »

It's kind of funny when we say "self taught", since we have the best teacher you can ask for at our finger tips...the Internet. Smiley I did go to university for programming, but it was a "liberal arts" college that was mostly known for theater, so it was definitely NOT a tech school by any stretch of the imagination. LOL I feel like I've learned far more online than I learned in college as most of the teachers were "so-so". I created a music related MIDI applet for my final project and my teachers couldn't even help me with it since none of them worked with MIDI...after a couple months of trying to figure it out, got lucky and found an example that was similar enough to what I needed and I was able to make it work. I uploaded my finished app online and since you're also a musician, I'll go ahead and share the link here...it's a random chord progression generator that creates chord progressions using music theory and then plays them via MIDI.
http://www.guitarforbeginners.com/tekker/Chord%20Progression%20Generator%20V3/ProgressionGenerator.html

What sound programs have you made? Have you made any kind of tools for musicians?

I've also been looking at finding the mobile equivalents to SourceDataLine as well, I believe AudioTrack is the equivalent for Android...iOS has so ridiculously complicated that I actually haven't found the equivalent there yet. I've looked at several tutorials for iOS sound and it is insane the amount of overhead you have to do when working with lower level audio. Ugh!

When mixing the sounds, wouldn't you also need to have some kind of DSP limiter to make sure you don't clip when combining multiple sounds?

I will definitely take a look at the article you mentioned and the code you posted. I have downloaded the TinySound source code and quickly glanced through it, but I'll definitely have to dedicate more time to really studying it to see what all I can get out of it.

Thanks again for your help, you've definitely given me some new directions to look into and creating my own audio mixer sounds intriguing for sure...got more studying to do there!
Offline philfrei
« Reply #5 - Posted 2014-06-18 18:41:52 »

It's kind of funny when we say "self taught", since we have the best teacher you can ask for at our finger tips...the Internet. Smiley

Good point!

Quote
I did go to university for programming, but it was a "liberal arts" college that was mostly known for theater, so it was definitely NOT a tech school by any stretch of the imagination. LOL I feel like I've learned far more online than I learned in college as most of the teachers were "so-so".

I was at UC Berkeley as a Music Major, but took "breadth" courses in Comp Sci and in Psychology (emphasis Cognitive & Psychoacoustics--worked at a Hearing Lab for work-study). Did a bit of composing and sound design for the Dance Department, and the Drama Department and the Educational TV Office. The CompSci teachers were great, no complaints. But the material I chose: Pascal, Assembly (via PDP-11), Logic Design (e.g., TTL chips to make a Pong game!), are all pretty out of date now. I was there in the 1970-s & 1980's. (Dating myself!)

Quote
I created a music related MIDI applet for my final project and my teachers couldn't even help me with it since none of them worked with MIDI...after a couple months of trying to figure it out, got lucky and found an example that was similar enough to what I needed and I was able to make it work. I uploaded my finished app online and since you're also a musician, I'll go ahead and share the link here...it's a random chord progression generator that creates chord progressions using music theory and then plays them via MIDI.
http://www.guitarforbeginners.com/tekker/Chord%20Progression%20Generator%20V3/ProgressionGenerator.html

Cool! I'm looking forward to checking it out. By the way, you might check out krasse's related work, posted on JGO.
http://www.java-gaming.org/topics/some-reasonable-output-from-my-generative-music-system/27309/view.html

Quote
What sound programs have you made? Have you made any kind of tools for musicians?

I made a Theremin applet, with a real-time touchpad control for volume and pitch.
http://www.java-gaming.org/topics/java-theremin/24646/view.html
I made another touchpad that let's you play sounds forwards or backwards at varying speeds. Another tool (based on a friend's idea) makes semi-random sine waves that are harmonically related (whole-number-ratios), creating a kind of evolving cloud--sounds spacy--involved figuring out envelopes.

After getting the basic mixer working, I wrote my own version of the Java Clip that can play at varying speeds, but also can support multiple cursors, so you can do things like trigger multiple gunshots and not have the newest shot squelch the previous. I wrote an improved tool for looping these clips, in that one can specify that the ends overlap or not, and cross-fade or not.

I have yet another Clip variant that plays random slices of a sound. There's an example posted somewhere here, where a 4-second recording of a brook is broken into something like 200msec random slices that are played with a bit of overlap, creating the illusion of a continual, non-repeating sound from a small source file.
http://www.java-gaming.org/topics/what-music-do-you-listen-to-while-you-code/27824/msg/275109/view.html#msg275109

Related, a "SoundField" construction that lets one select a number of Clips and play them semi-randomly, at a certain probability for given time period. This was also used in the above link, but the current version has a much more efficient way of generating the random bells than using Perlin's algorithm.

There was a soundscape sort of piece involving wav files from an old analog synth that I own. There are five volume sliders that can be manipulated or left to drift randomly. I was attempting to use ogg compression, initially, but couldn't get it to work without memory leakage. So the file is a bit bulky as the resources are raw wav files.
http://hexara.com/ffw/ffway.jar

I have been delving more deeply into frequency modulation, as I am a big fan of the Yamaha DX7 (vintage 1980's synth, again dating myself). I've written an FM synthesizer, and have converted something like 8 of the DX7 patches to real-time Java synthesis, some bells, and pads and a keyboard so far.

I have an echo tool working, and a flanger/chorus algorithm working. Haven't gotten to filters yet, nor reverb.

The "sound field", FM and flanger can be heard in the puzzle game project I started but have yet to finish: Hexara.
http://www.java-gaming.org/topics/hexara-work-in-progress/23676/view.html

Making a listing like this (first time in a long time!) I see that I've been at it for a while now, but also have a definite tendency to R&D over completing and bringing products to market. I am in awe of programmers like nsigma who have accomplished so much that is practical and commercially/publicly available.

Quote
I've also been looking at finding the mobile equivalents to SourceDataLine as well, I believe AudioTrack is the equivalent for Android...
Bingo! I agree. I just haven't attempted a proof-of-concept example yet.

Quote
...iOS has so ridiculously complicated that I actually haven't found the equivalent there yet. I've looked at several tutorials for iOS sound and it is insane the amount of overhead you have to do when working with lower level audio. Ugh!

A good friend of mine is a contract iOS programmer, and he recently said he found the hook. But this is a case where we have to successfully port Java to Object-C. There are a couple tools that seem plausible, one that goes directly, another path that uses an intermediate JavaScript conversion. Whether the code I wrote is basic enough Java to allow this conversion and runs efficiently, remains to be seen! I don't know enough about this hook yet to pass on info to you. I only heard about it from him two weeks ago and am frustrated that I have to work on other things and can't get to this immediately. The electronic tanpura project I'm working on (also stalled) has some potential to be a neat little iOS app.

Quote
When mixing the sounds, wouldn't you also need to have some kind of DSP limiter to make sure you don't clip when combining multiple sounds?

Yes. But simple can be effective for 90% of the situations: just use a Math.min and Math.max function. That prevents overflow. As to clipping, I use my ears for diagnosis, and just turn down the volume of the contributing sounds. Again, crude but simple. A limiter/compressor is not something I've dealt with yet. Like filtering, it involve understanding things like convolution (I have a tenuous grip on this--understand the basic concept but weak on appropriateness of where/how to implement), is a bit more advanced than I can handle.

Quote
I will definitely take a look at the article you mentioned and the code you posted. I have downloaded the TinySound source code and quickly glanced through it, but I'll definitely have to dedicate more time to really studying it to see what all I can get out of it.

Thanks again for your help, you've definitely given me some new directions to look into and creating my own audio mixer sounds intriguing for sure...got more studying to do there!

It's endless, and easy to get continually distracted! Good that you have a concrete project (the metronome) to help provide focus.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #6 - Posted 2014-06-19 10:50:09 »

Ok, so I’ve been brainstorming about how to go about this and I have an idea in mind, but I’m not sure it’s 100% there yet. So I have just a few questions to make sure I understand this correctly...

Let’s say I did something like this pseudo-ish code:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
buffer = new byte[1000];

while (run){
   // LOAD BUFFER
   for (int i=0; i<buffer.length; i++)
      buffer[i] = getNextAudioByte();

   // WRITE LINE
   line.write(buffer, 0, buffer.length);
}


Q1) Does writing to the SourceDataLine (“line”) operate on its own thread?
In this example, after “dropping off” the audio data on the line, would the program immediately go back to filling the next buffer while the audio is still being played on the line?

Q2) Does loading the buffer and writing to the line execute in exactly the same amount of time?
I don’t think it does. I think filling the buffer would execute way faster than writing the audio to the line because it seems like the speed of writing to the line would be limited by the sample rate. For example, with a sample rate of 44.1kHz each sample in the buffer would be played every 1 / 44100 = 22675ns, yet loading the buffer would operate significantly faster than this.

Q3) Would my example essentially crate a backlog of audio waiting to be written to the line?
If writing to the line operates on its own thread and loading the buffer executes faster than playing the buffer, it seems like the buffer would simply overrun the line with audio data and it would just keep getting backed up further and further. Thus making the program not very responsive if it tries to play something “immediately” yet there is still a lot of audio data in front of it to be played.

Q4) My idea is to have the program itself essentially operating “behind” the audio that is currently playing by the time that it takes to play one buffer (...I hope that makes sense), so would I need to slow down the process of loading the buffer in order to have it keep pace with playing the buffer?

I hope these questions make sense...its way too late and I’ve been frying my brain over this!

I’ll quickly reply to your latest response philfrei before my brain totally shuts off for tonight. lol

You certainly have quite an impressive list of audio projects there. It’s been a while since I’ve tried to load any java applets via my web browser, but is there currently some kind of problem with the java plugin? I’m seeing on the firefox page that they’ve blocked the current version because it has security vulnerabilities. I was trying to load your Theremin applet, but no luck. Sad

For Android’s AudioTrack, I found an Android metronome example that uses AudioTrack. It uses the identical write method that SourceDataLine uses, same parameters and everything.
http://masterex.github.io/archive/2012/05/28/android-audio-synthesis.html

I think I know what you’re referring to in terms of “the hook” for porting Java to mobile. I looked into a couple tools as well...the first one was Appcelerator Titanium, which uses JavaScript. The other one is Codename One, which uses Java (it has its own Java API that you use) and that’s the one I think I’m going with. Codename One is an impressive beast! It actually generates native code for most mobile platforms...Android, iOS, Windows Phone, and J2ME/RIM. However, one drawback is that it does not support low level audio, but it does allow you to write methods in native code so you can add additional low level functionality that Codename One doesn’t support. There are a bunch of videos by Shai Almog (the creator of Codename One) on Youtube.

Regarding the limiter, I’d prefer not to simply turn down the sounds as that might be strange to have the rhythms changing volume as more tones are added. I’d like them to stay at the same volume and simply prevent it from clipping, but as you said...that gets into the more complex DSP category there. But maybe there’s a way to plug in a simple-ish equation that would do the trick? It doesn’t have to be incredible and transparent sounding or anything since I’m not mixing complex music, basically just clicks and cowbells...I gotta have more cowbell in my metronome app! Cheesy

Thanks again for all your help!
Offline philfrei
« Reply #7 - Posted 2014-06-19 17:55:25 »

I'm not entirely sure of the implementation, but I think SourceDataLine and TargetDataLine both rely on a BlockingQueue or something very similar. Basically, the audio loop proceeds in its own thread. The write method for SourceDataLine blocks until the underlying code has had a chance to process the previous buffer's data and it is able to accept the new buffer array of data.

Yes, the loading of data into the buffer array should be significantly faster than the rate at which the sound data is played! The audio while loop basically should be spending most of its time in a blocked state (at the point where SourceDataLine is writing). If the loading of the buffer array ever takes longer than the write command blocks, you will probably hear drop-outs.

The api for SourceDataLine's write method refers to the blocking:
http://docs.oracle.com/javase/7/docs/api/javax/sound/sampled/SourceDataLine.html

There are methods to check availability on the line, but there is usually no need to do this. Mostly we just keep shovelling buffer arrays into the line as fast as it is willing to accept them.

This should answer all 4 of your questions!

**********

Yes, the whole deal with applets is messed up. I gave up trying to deal with it. I leave it to the user to figure out what they have to do with their browser & plugins to allow the applets to play. As far as I know, my programs are okay to trust. They've just been sitting on my website and protected from tampering by whatever security the provider (startlogic.com) has going.

Some of the links are downloads of jars. You might have more luck with them. Should I dig up the link to the theremin applet's jar file?
Here it is:
http://Hexara/VSL/JThereminApplet.jar
But you will probably have to run it in your IDE as an Applet, not as an Application. Be sure to try the Echo effect. I've been meaning to add a control for vibrato, but haven't managed this yet. Vibrato via a mouse is hard to make sound good.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #8 - Posted 2014-06-20 00:34:25 »

Thanks again philfrei! Yes, that does answer my questions...and naturally created more questions.  persecutioncomplex

Does the blocking occur during the first pass of writing audio to the line or does it block if it tries to write additional data to the line? In other words, in my pseudo code from before:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
buffer = new byte[1000];

while (run){
   // LOAD BUFFER
   for (int i=0; i<buffer.length; i++)
      buffer[i] = getNextAudioByte();

   // WRITE LINE
   line.write(buffer, 0, buffer.length);
}


Would this code...
a) Load the buffer and wait while that data is being written to the line.
b) Load the buffer, drop the data off on the line, go back load the next buffer, and then wait until the previous buffer finished writing on the line.

I know it seems like splitting hairs since this operation would likely be very fast, and if I were only dealing with audio then it wouldn't matter...but for syncing the beat counter visual display and the audio I need to know how much of a lag there is from when my program says to play a sound and when that sound actually ends up playing so I know precisely when to update the visual display.

Actually, I may be able to test when it blocks by using a really long buffer (like 10 seconds worth of audio) to slow the write process down and then printing out the system time just before it calls the load buffer method. That would hopefully tell me when it is blocking.

Also, I'll try downloading your Theremin applet when I get home. Those things are certainly odd instruments. lol I got to try one in a music course I took in college.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 816
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #9 - Posted 2014-06-20 07:05:54 »

You seem to be under the impression that the visual feedback is instant while the audio has significant latency. It's actually the other way around. At 60Hz your visual update interval is 16.6ms, but the driver will have most likely 2 or 3 frames it is still working on, and then the data has to be transfered to the monitor, adding to the latency. The visual latency is therefor a multiple of 16.6ms and then some, while the audio is pretty damn low latency with sub-millisecond latency. The only issue is that the OS scheduler is not realtime, so we need some buffering, but anything over 20ms worth of samples seems excessive.

Luckily, your brain compensates for most latency, or we'd all be confused by realworld latency caused by the speed of sound. If visual cause matches only barely with the audio, your brain will match the two up and tell your conscience it was a single event. This calibration happens within a second, and can even be exploited to make it seem like the effect (audio) happened before the cause (visual) if at the time your brain was tuned for high latency. But I digress.

Long story short: you actually need a fair amount of artificial audio latency for the visual side of the event to be in sync. It's fair to target 30-40ms delay between an ingame event and *all* feedback you get from your system.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Tekkerue
« Reply #10 - Posted 2014-06-20 08:49:15 »

Hi Riven, thanks for the reply!

I wasn't actually assuming the visual latency would be "instant"...but at the moment my focus is on the audio end of the program such as determining how I'm going to be mixing and playing my audio as well as trying to determine the precise moment that the audio will be playing and how far behind my program is operating when it says "play sound" and it actually plays that sound. I will eventually be doing the same thing for the visual display, but I haven't gotten to the point figuring out my visual method yet. Right now I'm just updating a JLabel with some text and color, but I'm sure that will change once I start actually working on the GUI.

Also, it seems like an audio buffer with even 1000 samples isn't nothing...1/44100*1000 = 22.6ms. So in regards to my previous question if it ends up blocking when the second buffer tries to add data, then my program would respond almost two buffers behind which could be around 40+ms. Which may actually put it in nearly perfect sync if the visual latency is about the same as this.....I totally meant to do that! Grin

It's great to know what kind of range I should try to hit with the audio and visual synchronization. My ultimate goal is to develop this app for different mobile devices and I'm sure the range of latencies of visual display and audio playback will vary so it'll likely be impossible to get all of them "perfect".

Thanks again!
Offline Tekkerue
« Reply #11 - Posted 2014-06-20 10:12:45 »

philfrei, I've tried running your Theremin applet within a couple IDE's (jGrasp and Netbeans)...jGrasp wouldn't even open it at all and while Netbeans did open it, the option to run it was grayed out. So still no luck on the Theremin. Sad
Offline philfrei
« Reply #12 - Posted 2014-06-20 10:42:44 »

Thanks for elucidating this, Riven.

Tekkerue, as to the question about blocking: if the BlockingQueue is asked to do something that it can do, then the code executes. If it can't do it, then it blocks. So, I pick your scenario (b). But the lack of real time guarantees in Java makes this moot if you want metronomic accuracy.

The article I cited earlier goes into this: "Real Time Low Latency Audio Processing". It's going to be vital that you understand the nature of Java's lack of real time guarantees if you are going to use javax.sound.sampled for this.

For accuracy on par with Java's MIDI sequencer, the only way I know of is to calculate the actual sound frame a note should play, count frames, and start the note on that frame (when the while loop is processing that frame). The moment in time at which sound data is loaded into the buffer is too variable relative to when it is actually heard to do otherwise.

Example: 44100 fps stereo, 16-bit ("CD Quality"). Let's say you want the metronome to play 120 bpm. I would calculate the interval in frames ( = 22050 ) and have the while loop count frames as it processes the buffers of PCM bytes, and start the metronome click sound on that actual frame. So, in the middle of some buffer when you get to frame number N * 22050, you start the loading of the PCM data for the click sound.

I'm guessing you may have to hear for yourself before taking the trouble to use the frame counting method. I first ran into the 'lack of real time guarantees' issue with the Theremin, trying to get the sound to accurately track the mouse movements via a MouseMotionListener. My rude awakening (and considerable resistance to the truth) can be viewed on an old JGO thread. Eventually I got a reasonable solution.

Java's MIDI package and sequencer can provide the needed accuracy--so maybe you should just use that, and schedule Midi 'control messages' to trigger your visuals.

Doesn't NetBeans or your other IDE have a way to run Applets? I use Eclipse.
Doh!  Clueless I forgot to give you the entry point:
Here's the HTML code that is used for that jar:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
<script src="http://www.java.com/js/deployJava.js"></script>
<script>
    var attributes = {code:'jTheremin.ThereminApplet.class',
                      archive:'JThereminApplet.jar',
                      width:810, height:400} ;
    var parameters = {fontSize:10} ;
    var version = '1.6' ;
 
    deployJava.runApplet(attributes, parameters, version);
</script>


You know, there is the program "appletviewer.exe" that is in the bin file of your JDK, probably.
http://docs.oracle.com/javase/7/docs/technotes/tools/windows/appletviewer.html
http://en.wikipedia.org/wiki/AppletViewer
I haven't tested this, but I think if you put the above html in a valid html file, and put it in the same directory as the jar, you can run it via a command line.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #13 - Posted 2014-06-20 11:33:16 »

Hi philfrei,

I read the article you provided, so I do understand what Java does to muck with the timing...but I also know that people have somehow been able to make it work. Smiley If I recall correctly, Y-Metronome was made with Java and I use this one quite regularly. I also have a few different metronome apps on my Android phone that work great, and as you know Android is done with Java. So it is possible to get great results using Java.

In terms of "counting frames", isn't that essentially what I'm doing by determining the time in nanoseconds that the sound should start playing? I'm specifically calculating the point that the audio should play...I guess I'm not totally sure what you mean by "frame"...?

I did look into the MIDI route, but unfortunately it doesn't appear that MIDI sequencers are supported on mobile OS's. Not even on Android. It appears there is very basic MIDI support in that you can play a MIDI file, but doing the kind of low level on the fly MIDI synthesis is not supported for mobile. Sad

I do have an appletviewer, so I will look into that soon...hopefully tomorrow.

BTW, have you had any luck getting my Random Chord Progression Generator to load? I can't load that either via the web browser.
Offline Tekkerue
« Reply #14 - Posted 2014-06-20 12:28:24 »

...D'oh, never mind! I see what you mean by "frame" as you have 44100fps. I'm too used to calling them SAMPLES instead of frames. lol

Ok, so if I understand what you're saying...

You keep track of every frame that plays by incrementing a frame counter by the size of the buffer every time you write to the line.
Then instead of doing math on an ever incrementing number of "nanoseconds" I would simply do that for an ever incrementing number of frames.

And since the GUI part wouldn't be running on "frames" per say...I could convert the frames to nanoseconds with 1 frame = 22675ns at 44100fps.

Am I on the right track here?...
Offline philfrei
« Reply #15 - Posted 2014-06-20 19:55:13 »

Quote
Am I on the right track here?...
Yes,

Quote
BTW, have you had any luck getting my Random Chord Progression Generator to load? I can't load that either via the web browser.
I haven't tried yet. Something about the way I'm wired, I will burn all sorts of time writing answers to questions and then get even further behind in everything else I am doing. Tongue Next window of opportunity will be Sunday. Will be looking forward to it!

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #16 - Posted 2014-06-21 03:30:06 »

Woohoo! Looks like I have my plan forward...now off to write some code! Thanks again for your help and I'll stop taking up all your time so you can get caught up on your own stuff now. Wink
Offline Tekkerue
« Reply #17 - Posted 2014-06-23 09:34:49 »

IT'S WORKING!!!

Click to Play
Click to Play
Click to Play
Click to Play
Click to Play


Just wanted to let you know that you are a genius philfrei! Smiley Using the method you described everything is working fantastically now. I even tried it at 2000bpm and it works!...no that's not a typo! Grin It does glitch if I try to do other things while it's running like minimize/maximize windows, etc but if I just leave the computer alone it plays perfectly at 2000bpm. Not that anyone would ever have a need for 2000bpm, but that is still way awesome it can do that! lol I could probably make it stop glitching completely if I increase the time between when it tries to load the sound and when it actually plays. I had to adjust it so that it played perfectly even at slower tempos as when I first got it playing it was taking to long to load the sounds so they weren't playing when they were supposed to.

And just to let you know how long I have been trying to create this metronome application...I recently came across an old thread of mine over on JavaRanch from 4 years ago when I was trying to make this metronome app using MIDI instead of audio...but I never got anywhere with it. Now it's working using audio and it should be able to work on mobile devices as well (in an ideal world of course! lol)...

I've had the idea to create the ultimate "one metronome to rule them all" for a while, but it seems like I've run into road blocks at every turn just trying to get well timed and consistent clicks that didn't glitch or drop out or whatever...and thanks to you, it works great now! Seriously, you have no idea how ecstatic I am right now!!

Thank you again for all your help!!

More happy dance....

Click to Play
Click to Play
Click to Play
Click to Play
Click to Play
Offline Tekkerue
« Reply #18 - Posted 2014-06-29 21:08:08 »

Hello again!

So, I just tried running the metronome on a desktop, recording it on a laptop, then checking the timing using my recording software...and I found that my metronome is speeding up ever so gradually. I also tried going back to a much simpler and straight forward metronome version I made from before (see code below) and tried recording that, but it too speeds up gradually by nearly the exact same amount (they were only a few frames apart after 10 minutes).

After exactly 10 minutes the simple version was 323 frames ahead and my current version was 314 frames ahead, these are approximately 7ms ahead of where the click should be. This is strange because I figured that if the timing was going to be off at all it would be slightly *behind* the beat if the process of loading the buffer added a little bit more time...but instead it is *ahead* of the beat and I can't figure that one out.

Unless I'm missing something, this metronome should be sample accurate and should not be drifting at all. I used 60bpm so a tone should be added exactly every 44100 samples thus there shouldn't be any cascading rounding errors or anything. I'm not seeing anything that I'm doing wrong....so could this be a hardware problem or a JVM problem?

I would really like to be able to record for an hour (or longer) and have it match the tempo grid PERFECTLY in my recording software the entire time. Is there anything more I can do to guarantee perfect timing or is this as good as it can get using Java?

Thanks!

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
115  
116  
117  
118  
119  
120  
121  
122  
123  
124  
125  
126  
127  
128  
129  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;

import java.io.*;
import javax.sound.sampled.*;

public class TimingCheck implements ActionListener {
   static TimingCheck timingCheck;

   JFrame mainFrame;
   JPanel mainContent;
   JPanel center;
   JButton buttonPlay;
   
   int bpm = 60;
   double frequencyOfTick = 800;
   int sampleRate = 44100;
   int bufferSize = 2000;
   
   byte[] buffer;
   int tickBufferSize;
   double[] tickBuffer;
   SourceDataLine line = null;  
   int tickLength = 4100;
   
   boolean playing = false;
   
   public TimingCheck(){
      tickBufferSize = (int)((60 / (double)bpm) * (double)sampleRate);
   }
   
   public static void main (String[] args) {      
      timingCheck = new TimingCheck();
     
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            timingCheck.guiCreateAndShow();
         }
      });
   }
   
   public void guiCreateAndShow() {
      guiFrameAndContentPanel();
      guiAddContent();
   }
   public void guiFrameAndContentPanel() {
      mainContent = new JPanel();
      mainContent.setLayout(new BorderLayout());
      mainContent.setPreferredSize(new Dimension(500,500));
      mainContent.setOpaque(true);
     
      mainFrame = new JFrame("Timing Check");            
      mainFrame.setContentPane(mainContent);            
      mainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      mainFrame.pack();
      mainFrame.setVisible(true);
   }
   public void guiAddContent() {
      JPanel center = new JPanel();
      center.setOpaque(true);
     
      buttonPlay = new JButton("PLAY / STOP");
      buttonPlay.setActionCommand("play");
      buttonPlay.addActionListener(this);
      buttonPlay.setPreferredSize(new Dimension(200, 50));
     
      center.add(buttonPlay);
      mainContent.add(center, BorderLayout.CENTER);
   }
   
   public void actionPerformed(ActionEvent e) {
      if (!playing) {
         playing = true;
         timingCheck.createPlayer();
         timingCheck.generateTick();
         new Thread(play).start();
      }
      else {
         playing = false;
      }
   }
   
   public void generateTick(){
      tickBuffer = new double[tickBufferSize];
      for (int i = 0; i < tickLength; i++)
         tickBuffer[i] = Math.sin(2 * Math.PI * i / (sampleRate/frequencyOfTick));
      for (int i = tickLength; i < tickBufferSize; i++)
         tickBuffer[i] = 0;
   }
   
   public void createPlayer(){
      AudioFormat af = new AudioFormat(sampleRate, 16, 1, true, false);
      try {
         line = AudioSystem.getSourceDataLine(af);
         line.open(af);
         line.start();
      }
      catch (Exception ex) { ex.printStackTrace(); }
   }

   public Runnable play = new Runnable() { public void run() {
      buffer = new byte[bufferSize];
      int b=0;
      int t=0;
           
      while (playing) {
         if (t >= tickBufferSize)
            t = 0;
         
         short maxSample = (short) ((tickBuffer[t++] * Short.MAX_VALUE));
         buffer[b++] = (byte) (maxSample & 0x00ff);        
         buffer[b++] = (byte) ((maxSample & 0xff00) >>> 8);
         
         if (b >= bufferSize) {
            line.write(buffer, 0, buffer.length);
            b=0;
         }
      }
     
      destroyPlayer();
   }};
   
   public void destroyPlayer(){
      line.drain();
      line.close();
   }
}
Offline philfrei
« Reply #19 - Posted 2014-06-29 23:00:45 »

You are talking about less than 1/100th of a second after 10 minutes, when comparing the timing mechanisms of two entirely different pieces of software.

Maybe this is normal, I don't know. Perhaps this is why we have things like SMPTE code, and other mechanisms for maintaining synchronization?

Is tickBufferSize calculating out to what you expect? Bouncing back and forth between doubles and ints is worrisome. You only need to have the denominator of the division be a double (I don't know if making the sample rate a double adds to the error or not). Maybe test for a simple tempo like 60bpm or 120bpm, and eliminate this calculation -- hard code in 44100 or 22050 as your tickBufferSize and see if the error still occurs.

It occurs to me the (int) cast at the end could be causing a slight truncation in the tickBufferSize, which would create a slightly faster than expected rendition.

How many seconds in 10 minutes? 600? How many tickBuffers in 10 minutes? (Depends on tempo.) Are you testing a tempo (e.g., slower than 60bpm, or some "awkward" number) that would create the shown error if the tickBufferSize was off by one due to the int cast?

Another thought: take a piece of music, a wav that you have in your DAW, and play it back with Java, and record that in the DAW and compare. Or compare with other software that can play back.

"It's after the end of the world! Don't you know that yet?"
Offline Tekkerue
« Reply #20 - Posted 2014-06-29 23:27:56 »

...just a quick addition.

For comparison I just tried recording one of the current metronome apps I have on my phone (Samsung Galaxy S4) called Mobile Metronome, which the top free metronome for Android (with over 1 million downloads) and after 10 minutes of recording it was 3,229 samples (or approximately 73ms) behind the beat. So mine is actually over 10x more accurate than the top freebie, not to shabby. Smiley So maybe this is just something I should be expecting and just have to live with? Although, my mind is going all kinds of places like trying to figure out how to now easily calculate the drift and then create a drift "offset" that would nudge back it enough to keep it on beat.....OCD and programming, not a good mix. lol
Offline Tekkerue
« Reply #21 - Posted 2014-06-29 23:53:43 »

Hi philfrei,

I realize it's a small delay and different software...but I've moved recorded music tracks around between different software and computers before with no shift in the timing. A friend and I used to bounce tracks back and forth between his Mac laptop and my PC desktop (he used Logic and I use Magix Samplitude) and there wasn't any timing drift problems as long as my project's tempo matched his project's tempo (or vice versa). But maybe any difference between them would have been too small to notice with instrument tracks?...it's very easy to tell exactly where the start of my metronome tone begins since everything surrounding it is all zero's, but that wouldn't be the case with instrument tracks.

I just added in a print statement for tickBufferSize and it does come back out to equal exactly 44100. The tempo I tested was 60bpm, which I did to avoid any weird cascading rounding errors as you mentioned just to see if it would stay "spot on".

I love your idea of trying to play a sound file in Java and recording that. I'll record 10 minutes of the metronome within my recording software Samplitude so that its easy to compare the difference between the Java playback version and the original. I'll need to figure out how to load the wave file into my program, because I'd like to use the same SourceDataLine method I'm using for my metronome.

Thanks again for all your help! Smiley
Offline Tekkerue
« Reply #22 - Posted 2014-06-30 05:38:20 »

Alrighty, I got the WAV file reader working. BTW, I found some great source code on Stack Overflow from a google search that I was able to use to get it going...it was posted by an extremely helpful guy! Wink
http://stackoverflow.com/a/6400178
...what a small world! Grin

I recorded 10 min of Samplitude's metronome at 60bpm as a mono wav file, played it through my Java application, re-recorded it via the laptop, and it too is drifting ahead of time by 268 samples or approximately 6ms after 10 minutes. So this method had a little less drift than my metronome.

EDIT: I should mention that I did check the initial recording of Samplitude's metronome (just to be sure) by loading it back into Samplitude and it was perfect down to the sample at 10 min. No drift at all. This is the file I played via Java.

Here is the code:

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
65  
66  
67  
68  
69  
70  
71  
72  
73  
74  
75  
76  
77  
78  
79  
80  
81  
82  
83  
84  
85  
86  
87  
88  
89  
90  
91  
92  
93  
94  
95  
96  
97  
98  
99  
100  
101  
102  
103  
104  
105  
106  
107  
108  
109  
110  
111  
112  
113  
114  
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;

import java.io.*;
import javax.sound.sampled.*;

public class WavReader implements ActionListener {
   static WavReader wavReader;
   
   JFrame mainFrame;
   JPanel mainContent;
   JPanel center;
   JButton buttonPlay;
   
   int bufferSize = 2000;
   byte[] buffer;
   SourceDataLine line = null;
   
   boolean playing = false;
   
   File fileIn;  
   AudioInputStream audioInputStream;
   
   public WavReader(){
      fileIn = new File("SamplitudeMetronomeMono.wav");
   }
   
   public static void main (String[] args) {      
      wavReader = new WavReader();
     
      SwingUtilities.invokeLater(new Runnable() {
         public void run() {
            wavReader.guiCreateAndShow();
         }
      });
   }
   
   public void guiCreateAndShow() {
      guiFrameAndContentPanel();
      guiAddContent();
   }
   public void guiFrameAndContentPanel() {
      mainContent = new JPanel();
      mainContent.setLayout(new BorderLayout());
      mainContent.setPreferredSize(new Dimension(500,500));
      mainContent.setOpaque(true);
     
      mainFrame = new JFrame("Timing Check");            
      mainFrame.setContentPane(mainContent);            
      mainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      mainFrame.pack();
      mainFrame.setVisible(true);
   }
   public void guiAddContent() {
      JPanel center = new JPanel();
      center.setOpaque(true);
     
      buttonPlay = new JButton("PLAY / STOP");
      buttonPlay.setActionCommand("play");
      buttonPlay.addActionListener(this);
      buttonPlay.setPreferredSize(new Dimension(200, 50));
     
      center.add(buttonPlay);
      mainContent.add(center, BorderLayout.CENTER);
   }
   
   public void actionPerformed(ActionEvent e) {
      if (!playing) {
         playing = true;
         loadWaveFile();
         createPlayer();
         new Thread(play).start();
      }
      else {
         playing = false;
      }
   }
   
   public void loadWaveFile(){
      try {
         audioInputStream = AudioSystem.getAudioInputStream(fileIn);
      } catch (Exception e) { e.printStackTrace(); }
   }
   
   public void createPlayer(){
      AudioFormat af = audioInputStream.getFormat();
      try {
         line = AudioSystem.getSourceDataLine(af);
         line.open(af);
         line.start();
      }
      catch (Exception e) { e.printStackTrace(); }
   }
   
   public Runnable play = new Runnable() { public void run() {
      buffer = new byte[bufferSize];
      int b=0;
     
      try {
         while (playing && (audioInputStream.read(buffer) != -1)) {
            line.write(buffer, 0, buffer.length);
         }
      } catch (Exception e){ e.printStackTrace(); }
     
      destroyPlayer();
   }};
   
   public void destroyPlayer(){
      line.drain();
      line.close();
   }
}
Offline philfrei
« Reply #23 - Posted 2014-07-01 17:41:27 »

Hi philfrei,

I realize it's a small delay and different software...but I've moved recorded music tracks around between different software and computers before with no shift in the timing. A friend and I used to bounce tracks back and forth between his Mac laptop and my PC desktop (he used Logic and I use Magix Samplitude) and there wasn't any timing drift problems as long as my project's tempo matched his project's tempo (or vice versa). But maybe any difference between them would have been too small to notice with instrument tracks?...it's very easy to tell exactly where the start of my metronome tone begins since everything surrounding it is all zero's, but that wouldn't be the case with instrument tracks.

I just added in a print statement for tickBufferSize and it does come back out to equal exactly 44100. The tempo I tested was 60bpm, which I did to avoid any weird cascading rounding errors as you mentioned just to see if it would stay "spot on".

I love your idea of trying to play a sound file in Java and recording that. I'll record 10 minutes of the metronome within my recording software Samplitude so that its easy to compare the difference between the Java playback version and the original. I'll need to figure out how to load the wave file into my program, because I'd like to use the same SourceDataLine method I'm using for my metronome.

Thanks again for all your help! Smiley

Interesting!

I think it is very likely that if you go back to some of those tracks you swapped, there might indeed be some drift. Ears have to be really good, and circumstances very clear to hear imprecisions when only a few millis are involved. I recall once trying to improvise against a friend's track and having the feeling be slightly uncomfortable, unable to get a sense of being "in the pocket". We took a closer look at the rhythms and found one of the drums offset by 4 millis. Straightening that out did a lot to settle my nerves. But I didn't hear it as a mistake, but rather just experienced some tension that I couldn't really explain. It could also have been an example of me just being nervous, and the fix working as a sort of placebo. But my experience was that the fix did feel more solid. (And yes, non-percussive sounds would be next to impossible to detect as being off at this level.)

I tested the idea of the int cast truncation, as well, and found that the "correct" number of frames was being generated, to the nearest frame. Later on, it occurred to me: what if a proper DAW is even more accurate than the sample rate? In order to handle a plethora of sample rates, maybe the DAWs work at a finer granularity than 44100 fps, and the metronomes naturally are tied to the highest level of precision available in that DAW. (My Cakewalk Sonar also handles 48000 sample rate, so it has to have something that is more precise than 44100 fps, but aren't there 96000 tracks now, as well?)

If a DAW's metronome is more accurate than the 44100 sample rate, then the fraction of the frame that is getting truncated in Java, but is handled in the DAW's metronome could cause some drift that is in the ballpark for what you are getting. Maybe?

I'm not sure how the fractions would be handled in the DAW. Maybe something like a Leap Year, adding or taking away an extra frame every now and then? As far as Java is concerned, there is no way I know of to take nanosecond readings at the precise time the frame is run through the digital-to-analog-converter, and the lack of real-time guarantees means the point when the frame is processed in the while loop could vary considerably!

In any event, your demonstration that the metronome is as or more accurate than other commercial metronomes seems like a good reality check. I wonder how your metronome, when running in Android, will compare.

Last bit of musing: if the DAW's metronome IS more accurate (to fractions of a sample), then a recording made from a DAW would most likely be a tiny bit longer than the calculated number of frames (using the truncation that Java does with the int cast). Perhaps that led to the drift in the plus, instead of minus direction, for that test?

I forgot about that post you dug up. I was contradicting the answer of one of my Java heros, Kay Horstmann! I was also doing a bit of venting. Maybe if you like that answer, and found it helpful, you'll give it a +?

Sometimes I wish I had minored in mathematics and/or sound engineering. I have a friend that is an engineer and used to work for Pro Tools. I'll run the idea of the metronome vs sample rate accuracy issue by him next chance I get.

"It's after the end of the world! Don't you know that yet?"
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 816
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #24 - Posted 2014-07-01 17:45:56 »

Surely there must be a way to query/bruteforce the maximum supported samplerate of the hardware.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Tekkerue
« Reply #25 - Posted 2014-07-02 06:38:21 »

It would have been interesting to have done some really fine comparisons between our programs. Can't really do so now because my friend no longer lives in the area. Also, in Samplitude when you zoom all the way in on a waveform it actually chops it up using vertical lines to show the value of every individual sample...I'm not sure if Logic does that.

Being more accurate than 44100 was my initial thinking as well, which is why my thinking was to use nanoseconds for timing...but then Java wouldn't allow that to work. Undecided However, that seems like it would have helped more with the odd tempos like 61bpm or similar since there would have to be "rounding" involved. But with 60bpm, that is one beat every second so it seems like each beat should theoretically be in the same place regardless of the sample rate...44100 or 96000 frames is divided over the span of a second, but it seems like that very first frame should always be in the same place. It's strange for sure! I'm very interested to hear what your friend who worked for Pro Tools has to say. Was he a programmer who worked on coding Pro Tools?

I too wonder how well this will work when I move over to Android...hopefully the difference is in the coding, not the platform. I'm gonna be bummed if it ends up getting worse when moved to mobile.

Maybe the best solution is just to provide an "offset" value in the settings? Just in case the accuracy does change when moved to mobile, then for those who want it "dead on accurate" they could calibrate it for their specific device and punch in the offset value. Now it would be way cool if I could figure out how to do that automatically! Such as automatically record 1 min of click internally, calculate the drift, and then adjust the offset all at the press of a button! Man that would be slick! Grin

I tried to give your post on StackOverflow a "+", but it said I have to have 15 reputation points...I'm only at 11! Pffffft!

I minored in physics....which is essentially math + additional headaches. LOL
Offline Tekkerue
« Reply #26 - Posted 2014-07-02 07:20:26 »

Riven,
I'm not sure that the sample rate is the problem for my tests at 60bpm. Even if the sample rate was ridiculously low at like 100Hz, the beat should still start on the very first frame at the start of each second. Higher sample rates would certainly help for beats that start in between, but it seems like that first frame should always start at the start of the second on the dot.

Internal clock issues maybe? Hmm....brain storm! I think tomorrow I'll try recording Samplitude's click straight out of Samplitude (no Java) and record it via the laptop in Reaper. Two different DAW software on two different computers. If this comes out to be slightly off time as well, then I think we can conclude that this little bit of timing drift is to be expected.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 816
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #27 - Posted 2014-07-02 08:15:15 »

I'm fairly sure philfrei meant that you might be sending 44100 samples per second to hardware that outputs 48000 samples per second.

This means that every sample is 'scaled in time' by factor 1.0884353741496598639455782312925 (48000 /44100)

Some transformation has to happen in hardware, and it might not be optimal, leading to a drift of a few millis over a timespan of 10 minutes. A drift of 7ms over 10*60*1000ms means it is 99.9988333% accurate, which seems reasonable for consumer grade audio hardware.

That's why I suggest to use the native sampling rate of your audio hardware, as to get rid of this truncation/inaccuracy altogether (hopefully).


Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline princec

JGO Kernel


Medals: 404
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #28 - Posted 2014-07-02 10:14:49 »

ISTR that since Windows Vista the entire Windows audio pipeline is now implemented purely in software. I may be wrong.

Cas Smiley

Offline Tekkerue
« Reply #29 - Posted 2014-07-06 00:14:37 »

Ok, I have some new test results! I also re-recorded 44.1kHz just to see if it was the same as before, and it actually had slightly less drift this time...

TimingCheck.java:
44.1kHz = 295 samples (approx. 6ms) ahead of the beat
48kHz = 1,232 samples (approx. 27ms) behind the beat
96kHz = 297 samples (approx. 6ms) ahead of the beat

NO Java - metronome track played in Samplitude:
44.1kHz (both devices) = 289 samples (approx. 6ms) ahead of the beat

For some reason 48kHz was off by quite a bit more than the others and it was the only one to drift behind the beat. So that’s interesting.

I think you were right on the money philfrei, the drift is simply because it's two different applications running on two different devices. Even two different DAW’s on different computers were not in sync perfectly after 10 minutes and were off by approximately the same amount as the Java program.

Guess I should stop worrying about this now...eh? Grin
Pages: [1]
  ignore  |  Print  
 
 

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

Longarmx (49 views)
2014-10-17 03:59:02

Norakomi (38 views)
2014-10-16 15:22:06

Norakomi (31 views)
2014-10-16 15:20:20

lcass (34 views)
2014-10-15 16:18:58

TehJavaDev (65 views)
2014-10-14 00:39:48

TehJavaDev (65 views)
2014-10-14 00:35:47

TehJavaDev (55 views)
2014-10-14 00:32:37

BurntPizza (72 views)
2014-10-11 23:24:42

BurntPizza (43 views)
2014-10-11 23:10:45

BurntPizza (84 views)
2014-10-11 22:30:10
Understanding relations between setOrigin, setScale and setPosition in libGdx
by mbabuskov
2014-10-09 22:35:00

Definite guide to supporting multiple device resolutions on Android (2014)
by mbabuskov
2014-10-02 22:36:02

List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!