Here it is:

http://ff123.net/abchr/abchr.zip

(145 kB download)

Just construct a configuration file as shown in the example config file. Up to 8 samples are allowed to be compared. Load the configuration file to start, and save your results to a text results file to finish up.

Let me know if there are any serious bugs before I say it's ok to use.

ff123

Seems to work pretty well from what I can tell. Good job.

One small thing, which I'm not sure is a bug or a feature, is that when clicking the about item in the menu, it plays back one of the samples.

Btw, do you plan on including a "fast switch" playback feature and seeking, etc, (basically more playback controls) in future versions of this?

Notes to myself:

1. program crashes if I use a results text file as the config file.
2. should add text to say which file is currently playing, or has just been played.

BTW, for anybody mystified by the purpose of this tool: Each group (labelled 1 through 8) consists of two files, the reference and an encoded file. The reference is either on the left or right side, but you don't know which one it is. The ordering of the groups are also randomized.

When you rate something less than 5.0 by pulling down the slider, I allow you to comment on the sound (button above each slider). I infer that you think the other file is the reference, so I place that rating back up at 5.0 if you've previously rated it less than 5.0.

The idea is that every time you rate a sample, you're forced to choose against the original file. If you rate the reference file less than 5.0 too many times, your data can be discarded in a group listening test.

Also, in something like a low bitrate test using a non-ABC/HR utility such as Ivan's waverate, listeners may have a tendency to rate everything less than 5.0 because they "know that everything must sound worse than the original." But the thought of being wrong and rating the original less than 5.0 may help to keep everybody honest. If something can't be distinguished from the original, it is more likely to get rated 5.0 than 3.0.

ff123

That's a feature. It's a short section of castanets.wav, which was one of the first things I implemented to make sure I would be able to play wavs.



I plan to add a control to allow playback to start from within a file, but that may take some work. Ivan suggested I investigate using a separate thread and MCI.

ff123

Fixed the crashing if a malformed config file is loaded.
Added current file playback status.

Uploaded to same place. Version is now 0.3.

ff123

I'm working (sort of, because it is a little bit stopped at this moment) in a kind of ABX comparator with additional features over PCABX comparator, and have the quick switching and random positioning problems solved, and yes, I used a thread and Win32 waveform services. My comparator actually works (more or less) at its basics, but I'd like to refine it and add some features. You can contact me in private if you want more info.

Version 0.4: allows configuration file to have extra stuff like blank lines and comments without problem.

Thanks KikeG, I told Ivan I pretty much wanted to code stuff for myself because part of this is for my entertainment. But if I get really stuck, I'll look you or Ivan up for hints.

ff123

Version 0.5 -- fast switching

http://ff123.net/abchr/abchrBinary0.5.zip

As of this version, I have switched over to using the Media Control Interface commands instead of the basic PlaySound function. That means I am able to monitor the position in the wav file. The first thing I did was to add fast switching. This means when you hit various play buttons while the sound is still playing, the program will not start over at the beginning of the sound (hitting the stop button will get you back to the beginning).

There is a little bit of latency in these commands, but it seems to be no worse than the implementation in Arny Krueger's PCABX. I estimate, using my calibrated ear that the latency is on the order of about 100 msec.

One way I reduced latency was to open all the files when the config file is loaded, so this version won't actually enable any buttons until real files exist to open up.

Any suggestions/bug reports welcomed.

Next planned feature: start playing from specified position.

ff123

note to self:

1. when reloading config file, reset the sliders
2. "current file" status is stuck on 1L or 1R
3. verified 48 kHz and 24-bit playback works fine for me.

sorry, but what is abchr?

http://www.ff123.net/abchr/abchr.html



It's a software used for blind listening testings.

I haven't updated the page yet. Will probably do so with a version 0.5a tomorrow after I correct the minor bugs I listed above.

ff123

That's brilliant!

One suggestion: If I do a test, and then exit the program, can you make it prompt me to save the results (if I want) -you know, the usual Save? yes/no/cancel. At the moment, the program just exits and wipes all my data.

btw, a "reset all" might be useful on a menu too. With an "Are you sure?" box.

Cheers,
David.
sorry - that was 2 suggestions. the second one doesn't matter.

Ah, I forgot about the "Are you sure you want to exit?" thing. Thanks for reminding me. The "reset" function is currently handled by reloading the configuration file, but an "Are you sure..." message wouldn't be out of place here either. I'll take care of these things soon.

BTW, didn't you mention at one time that it wouldn't be too hard to find an algorithm to align two wav files in time? Such a thing would be useful in conjunction with fast switching.

ff123

Program now prompts user to save before reloading or exiting (version 0.5b).

I may opt to add a quick and dirty slider control for the user to specify the start point of the playback. It would be kind of kludgy though if I also wanted to be able to allow the user to specify the end point by using yet another slider.

The other option would be to delay adding this feature until I can construct a custom widget to select a portion of the wav to play back.

ff123

Version 0.6 is out, now with the capability of selecting a portion of the files to play.

Left click and drag in the rectangle at the bottom to select a portion (default is to play the entire file). I'll add the capability to change the start/stop using the edit boxes later.

http://ff123.net/abchr/abchr.html

As always, let me know what annoys you about the app, or what needs fixing or adding.

ff123

Probably I'll try to concentrate on offset correction next. The easy way to do this is to make the correction specifiable from the configuration file. The correction would be looked up by the test administrator in a database of common encoder/decoder combinations.

ff123

Offset correction is almost finished.

I'm thinking ahead to future versions. The next thing I plan to do is to make some sort of repeated trials mode. My first thought has been to make an ABX mode, but I wonder if it would make sense to make a repeated ABC/HR mode instead, where actual ratings are averaged?

Whichever mode I choose, I'll probably implement it as a separate screen from which the listener chooses two files to compare. I would keep track of progress in the logfile, and the ABX or repeated ABC/HR results would not change the ratings given by the listener on the main screen. However, the listener could use the ABX results to train himself or to verify to himself that he is hearing an actual difference. And of course, the test administrator could use the ABX results for his analysis.

In some future version after that, the program could be modified to allow the test administrator to specify a certain confidence level, for very strict tests. For example, the test administrator could specify that only results which have been verified to be ABX'd at 95% confidence against the original would be accepted.

I would probably still administer listening tests with coded filenames (eg., file325.wav) because there is one source of bias which the program does not eliminate, and that's the "embarrassment" bias, in which a user is so embarrassed to have chosen WMA8 as his favorite codec that he forgets to send in his results.

Which leads me to encryption. I was thinking that even a brainless option like ROT13 could work to curb most temptations to peek at results before sending them to the test administrator.

Comments?

ff123

q: is there a list of offsets for different encoders, or is that just a case with mp3?

edited:
so far i got this (didnt test with playback yet):

mp3 +25 ms
aac -23 ms
ogg 0 ms (1 sample?)
mpc 0 ms (1 sample?)

I'll have to compile a list after I've got the offset correction working. So far, the worst offender seems to be mp3pro, which I estimate has an offset of about 100 msec, which is clearly audible.

ff123

Version 0.7 is out, with the ability to correct for time misalignments by specifying a correction offset in the configuration file. For example, the correction offset for an mp3pro file is 105, where 105 milliseconds is the extra amount of time that a decoded mp3pro file has at the beginning compared with the original wav file.

http://ff123.net/abchr/abchr.html

ff123

Version 0.7a allows you to tweak the portion selection. Move your cursor over one of the edges of the selection box and the cursor will change into a horizontal, double-headed arrow. Then you'll be able to move that edge without affecting the location of the other edge.

There's an interesting, but non-fatal bug I noticed in this version after I uploaded, and it's that you can also tweak the portion selection by moving your cursor over other areas in the program window (see if you can find them!)

I'll fix that soon.

ff123

Fixed. Version 0.7b

http://ff123.net/abchr/abchr.html

ff123

So let me get this straight:

You're using MCI for wave playing in ABCHR? That seems mighty kludgy to me. The way I'd implement it if I were going to would be to spend the extra time implementing the API waveOut functions. That way, you could implement your fast switching feature by updating all the non-played buffers when a button is clicked. It'd be tricky to get a good algorithm to do that without interfering with Windows' playing of the stream, but it would let you switch without having the audible clicks and pops.

It's on my list of things to do. MCI was extremely easy to implement, which is why it's in there now. Peter (Winamp developer) has some code I might snarf when I want to update my code.

ff123

*understands* It took me over 50 hours of coding to figure out what the hell was going on when I first started messing with waveOut... Then I realized you have to leave the buffers alone after they're prepared and copy data into them. I came /this/ close to giving up. Heh, the project I did that for isn't even close to finished, though... Anyhow, I could likely walk ya through it if you need a guru, although I'm not much of one.

Version 0.9 is available:

http://ff123.net/abchr/abchr.html

Changes:

Fixes bug when changing the portion select box while performing quick switching. Hopefully cures simultaneous playing of samples (not reproduceable on my machine). The portion select box now has a double-click feature (selects the entire sample). Comment boxes now do not steal the focus (they are modeless) and they have word-wrapping. A general comments box has been added. Now able to set up a test from within the program instead of from a configuration file. The program name and version is now added to the top of the results file.

I'll probably go for forced ABX next.

ff123

The obligatory bug fix version

version 0.9a

The setup menu was setting all the offsets to 0.

ff123

Hi ff123, remember me?

I used abchr for the CT test, it worked flawlessly, maybe some more options can be added to make it even more all-purpose:

- A non-blind mode, with single instead of paired sliders, and the file sequence not random but numbered as configured in Test Setup.

- A 'repeat forever' option of playback.

- A 'rewind before play' option, to compare the same sample portion more quickly.

Hans

Hi Hans. Of course I remember you

The "repeat forever" feature is already on the wishlist (I termed it "looping"). When I implement looping, it will probably also come with a specifiable delay between loops. The "rewind before play" feature is also already on the wishlist, and was requested by someone else earlier. I refer to this as "disabling the quick switching."

Both of these features will probably be added as listener preferences in a separate dialog box. However, I'll probably want to write to the Windows registry to save these preferences. Which means that I really should have a program installer of some sort so that an uninstaller knows how to clean up the registry when the program is removed from someone's hard disk. I'd really like to get the installer program at the same time I implement writing to the registry.

I don't have a non-blind mode on the wishlist though. I can add it, but it will probably be pretty far down in terms of priority.

ff123

Why not just write to a configuration file?

I personally hate programs that fill up the regsitry with stuff that could have just as easily gone into an .ini/.conf file in the directory with the program. With a configuration file, it's much cleaner, much less complex to implement, easier to make modifications to by hand, and easier to remove entirely.

If you do it this way, an uninstaller would only need to delete the directory. You wouldn't have to worry about the mess of registry issues between different windows OS's and whether or not the uninstaller is intelligent enough to remove all of the stuff in there, especially if some modifications where made to it.

Just my .02

True enough. It's not likely there needs to be different preferences (for different accounts) on the same machine. I'm sold. an INI file it is!

ff123

I'm missing a "save as configuration file"-option for test-reuse.
Apart from that, the new version made setting up tests a lot easier.

0.9b is now available with a different random number generator (the same one Garf found for the bootstrap program).

It's not certain that ABX was affected before, but at least one run sent to me did show moderate evidence against randomness in a runs test.

http://ff123.net/abchr/abchr.html

I added text printout to the status window in the ABX dialog box, but I'm in the process of revamping the ABX statistics, so this will change.

The Setup and ABX screens may not look quite aesthetically right; that's because I added some entry forms for the ABX profile, but made them invisible for this version.

ff123