Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Naim: unfair 24/96 vs. 16/44 presentation (Read 33348 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Naim: unfair 24/96 vs. 16/44 presentation

Check out the "High Definition" versus "CD Quality" "test files at Naim Hi Definition versus CD Quality test files (free downloads)

I compared the spectral contents of  both sets of "High Definition" versus "CD Quality" files within the 20-20 KHz range and found *dramatic* differences. Major roll-offs below 100 Hz and above 8 KHz, as well as level differences.

Nothing like feeding naive listeners what they think they are supposed to want to hear, eh?

Naim: unfair 24/96 vs. 16/44 presentation

Reply #1
I can confirm that - upsampling to 384khz and sample-aligning yields a null-out to only -21.4 db - but I don't see anywhere on the page that implies that the files are meant for performance evaluation; only compatibility evaluation.

Where's the beef?

Naim: unfair 24/96 vs. 16/44 presentation

Reply #2
I can confirm that - upsampling to 384khz and sample-aligning yields a null-out to only -21.4 db - but I don't see anywhere on the page that implies that the files are meant for performance evaluation; only compatibility evaluation.

Where's the beef?


I was tipped off to these file's existence by someone who cited them as an example of how 24/96 sounds so much better than 16/44.  So, I'm not imaging that people naturally assume that there's no difference between the files other than that which is due to sample size and rate.

I suspect that most people on HA wouldn't have been fooled, but not so much for joe average audiophile.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #3
Very interesting.

I made a comparison of the classical pieces with iZotope Ozone and their "Matching EQ" which is normally used to apply the frequency response of a certain audiofile onto another one (i.e. for samplers). But it can also serve to find out frequency differences.



The 24/96 was used as the reference file and you can see that it amplifies frequencies for 1 dB starting with 1000 Hz (there wouldn´t be any difference if the red line would be flat). Then the phase of the 16/44.1 file was the opposite compared to the 24/96. It´s disturbing that people use these files to say that 24/96 is better when clearly this argument isn´t valid. From the frequency differences I can see one can judge that the differences don´t result from improper downsampling - they seem to be a result of different mixing for different samplerates. Which - if true - forbids any sonic comparison by principle if that comparison is used to explain differences between samplerates.
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #4
Apparently What Hi-Fi used these files in a blind test of three subjects who preferred 24 bit, as reported in the current issue (which I've not seen myself and report only as hearsay).

P.S. this makes me embarrassed by my username.

PPS without seeing the magazine I realize I'm now not sure what tracks were used--it was just "three tracks from the Naim/Linn sites" so maybe not at all the same as the link which started this thread.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #5
There's something even more disturbing about this. The website is for Naim's music label. It is entirely reasonable to assume that these test files to check compatibility are extracts from the actual files offered for purchase. And that means that Naim have deliberately nobbled the 16/44.1 versions in order to make the "hi-res" files sound better, thereby encouraging customers to pay a 70% price premium to get the "best" versions. This is cynical and, quite frankly, immoral.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #6
Just for comparison:


Again, the 24/96 served as reference. This looks like a frequency difference - but it isn´t. I compared the CD-layer of one of my SACDs with the SACD layer of the same disc. The CD-layer was grabbed with EAC, the SACD layer was recorded analogue with my ASUS soundcard while it was played back by my Pioneer.

1. The CD-layer is 1.06 dB louder than the SACD layer
2. The difference you can see results from the "Matching EQ" interpreting the dithering noise of the CD layer as sound. The slightly attentuated bass is only attentuated by 0.1 dB and could have its reason in the output stages of my Pioneer player.

When one considers that the CD layer is louder then it would have the sonic advantage for the casual listener. Audite and their mastering engineers obviously didn´t care about comparisons at all. BTW, I would get the same frequency response if I would compare the CD layer of TELARC releases to their SACD layer counterparts. There are honest people left - but Naim sadly isn´t one of them.
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #7
There's something even more disturbing about this. The website is for Naim's music label. It is entirely reasonable to assume that these test files to check compatibility are extracts from the actual files offered for purchase.

Why would you not assume it was part of the marketing to sell the files?

And that means that Naim have deliberately nobbled the 16/44.1 versions in order to make the "hi-res" files sound better, thereby encouraging customers to pay a 70% price premium to get the "best" versions. This is cynical and, quite frankly, immoral.

Have Naim stated that the signal is the same apart from the sample rate? Without such a statement, surely a rational person is going to expect the files to be processed to sound a bit different and with the more expensive ones sounding a bit "better" at least to the ears of the intended consumers.

Naim is an "audiophile" rather than a "high fidelity" company/brand. The products are designed and marketed to appeal to people that highly weight audiophile attributes and are prepared to pay a premium to own them (the premium price almost certainly being part of the attraction). In this case it is music with higher sample rates but the other products are similar.

There is nothing wrong with producing expensive luxury goods that only appeal to the subset of the population that have freely chosen to adopt audiophile beliefs in preference to other competing beliefs like, for example, scientific beliefs. In fact, since the audiophile thing became part of the mainstream of home audio in the 1970s, UK companies like Linn and Naim have done well whereas the established "high fidelity" home audio companies pretty much all disappeared although most of the brands are still around. I may have no interest in purchasing the products of companies like Linn and Naim because of their poor value for money given my beliefs but I would judge the presence of these luxury goods companies to have been a good thing for the UK.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #8
Have Naim stated that the signal is the same apart from the sample rate?


Implicitly, yes. They have given them the same name, they've advised that they be used for testing, and they haven't warned people that they are equalized in the 20-20 KHz range in a way that is bound to make them sound different,

Quote
Without such a statement, surely a rational person is going to expect the files to be processed to sound a bit different and with the more expensive ones sounding a bit "better" at least to the ears of the intended consumers.


Surely a rational person will assume that the same music in different packages differs only in terms of any limitations caused by the choice of packaging.

Do you expect a quart of milk to taste different than a gallon of milk when they have the same name and come from the same dairy on the same day?


Quote
Naim is an "audiophile" rather than a "high fidelity" company/brand. The products are designed and marketed to appeal to people that highly weight audiophile attributes and are prepared to pay a premium to own them (the premium price almost certainly being part of the attraction). In this case it is music with higher sample rates but the other products are similar.


What Naim did is IMO the equivalent of building cheap amps with mis-set tone controls to induce people to buy more expensive amps.

Quote
There is nothing wrong with producing expensive luxury goods that only appeal to the subset of the population that have freely chosen to adopt audiophile beliefs in preference to other competing beliefs like, for example, scientific beliefs.


I agree. This is about truth in packaging. Can we infer from this example that Naim's less expensive CD players have their bass rolled-off below 100 Hz so that people who are interested in sonic accuracy will be forced to buy their more expenisve models?


Naim: unfair 24/96 vs. 16/44 presentation

Reply #9
There is nothing wrong with producing expensive luxury goods that only appeal to the subset of the population that have freely chosen to adopt audiophile beliefs in preference to other competing beliefs like, for example, scientific beliefs. In fact, since the audiophile thing became part of the mainstream of home audio in the 1970s, UK companies like Linn and Naim have done well whereas the established "high fidelity" home audio companies pretty much all disappeared although most of the brands are still around. I may have no interest in purchasing the products of companies like Linn and Naim because of their poor value for money given my beliefs but I would judge the presence of these luxury goods companies to have been a good thing for the UK.
Frankly, that isn´t the point here. I think that no one holds a grudge against companies like Linn / Naim. They appeal - as you say - only to several people, and these people represent a minority. Also, Linn / Naim over the years produced relatively no-nonsense components (if one ignores this ridiculous expensive turntable from Linn) and from what I hear they are also quite reliable. May that be as it is, it looks different with Naim´s musical releases.

The key to this thread is responsibility. They have to know that naive audiophiles will take these files to do comparisons such as "Why 24/96 is better". By using what I would call "improper practices" they ridicule themselves & the audiophile community. They may have done it in the first place to sell more high resolution downloads but by simply using altered mixes for the other format they present themselves not in the best light IMO.
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #10
I know what we should do. Let's record some song at 16/44.1, then record the same song at 24/96, but this time in somebody't garage by amateurs who can barely play. Post both versions, then some magazine will do a listening test with these samples and "prove" that 16/44.1 sounds much better than 24/96. 

P.S. Thank you Cavaille, I love those plots.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #11
And that means that Naim have deliberately nobbled the 16/44.1 versions in order to make the "hi-res" files sound better, thereby encouraging customers to pay a 70% price premium to get the "best" versions. This is cynical and, quite frankly, immoral.

Have Naim stated that the signal is the same apart from the sample rate? Without such a statement, surely a rational person is going to expect the files to be processed to sound a bit different and with the more expensive ones sounding a bit "better" at least to the ears of the intended consumers.

I have no problem with Naim offering two different versions. BUT: Naim have always presented themselves as striving to provide the best possible sound quality. (Whether they succeed is another debate). Therefore I would expect any rational person to assume that when they offer music for sale, then it would be the best it could possibly be subject to any constraints of the delivery format used. Their typical customers *will* assume that the 16/44.1 files being sold by Naim couldn't be any better than they are. But it appears that Naim have deliberately engineered them to be of a lower quality than they could be. *That* is what I find unacceptable.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #12
Has anyone Naim already about this issue ?

Naim: unfair 24/96 vs. 16/44 presentation

Reply #13
There is nothing wrong with producing expensive luxury goods that only appeal to the subset of the population that have freely chosen to adopt audiophile beliefs in preference to other competing beliefs like, for example, scientific beliefs.

Frankly, that isn´t the point here. I think that no one holds a grudge against companies like Linn / Naim. They appeal - as you say - only to several people, and these people represent a minority. Also, Linn / Naim over the years produced relatively no-nonsense components (if one ignores this ridiculous expensive turntable from Linn) and from what I hear they are also quite reliable. May that be as it is, it looks different with Naim´s musical releases.

The key to this thread is responsibility. They have to know that naive audiophiles will take these files to do comparisons such as "Why 24/96 is better". By using what I would call "improper practices" they ridicule themselves & the audiophile community. They may have done it in the first place to sell more high resolution downloads but by simply using altered mixes for the other format they present themselves not in the best light IMO.

Cavaille, your second paragraph appears to contradict your first paragraph. I don't think honestguv's point should be dismissed. The audiophile industry works the same way as the fashion industry. They both are selling exclusivity and style more than substance. Consumers and designers in both cases may not fully aware of the game. Enough hedge and mystery is put into the pitch by the businesspeople to keep them clear of misrepresentation charges. Looks like Naim may have cut it a bit close to the misrepresentation line on this one.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #14
So what Naim are essentially saying is - our standard goods are worse than our luxury goods. And in this case, our standard goods are worse than everyone else's. Nice marketing. I wish Naim the best of luck with that one.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #15
There are a few free downloads (see right hand side of front page) of complete tracks from various albums which are also available in multiple formats (mp3/cd/hi-res) - has anyone checked those?

Like cliveb, I'm curious whether this is a one-off (i.e. a single intentionally rigged comparison), or standard on all their releases!

Cheers,
David.

Naim: unfair 24/96 vs. 16/44 presentation

Reply #16
Cavaille, your second paragraph appears to contradict your first paragraph. I don't think honestguv's point should be dismissed. The audiophile industry works the same way as the fashion industry. They both are selling exclusivity and style more than substance. Consumers and designers in both cases may not fully aware of the game. Enough hedge and mystery is put into the pitch by the businesspeople to keep them clear of misrepresentation charges. Looks like Naim may have cut it a bit close to the misrepresentation line on this one.
Really? I don´t think so if one considers that the components from Naim are produced by different people than their audio releases. They hire people from the outside to do recordings, do you actually think that the person who designs the components does the recording itself?

IMO, we have here two separate entities: one produces components, the other one produces music. In most companies, those businesses are strictly divided. In view of that, my comment isn´t contradicting.
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #17
Just for comparison:


Again, the 24/96 served as reference. This looks like a frequency difference - but it isn´t. I compared the CD-layer of one of my SACDs with the SACD layer of the same disc. The CD-layer was grabbed with EAC, the SACD layer was recorded analogue with my ASUS soundcard while it was played back by my Pioneer.

1. The CD-layer is 1.06 dB louder than the SACD layer


I don't see this in your difference graph.  Or was it something you determined by other means?  Since the SACD was captured as A/D, how did you compare overall levels of it to the digital rip?  Peak normalization followed by crest factor calculation?

Naim: unfair 24/96 vs. 16/44 presentation

Reply #18
There are a few free downloads (see right hand side of front page) of complete tracks from various albums which are also available in multiple formats (mp3/cd/hi-res) - has anyone checked those?

Like cliveb, I'm curious whether this is a one-off (i.e. a single intentionally rigged comparison), or standard on all their releases!

Here are the results of an invert mix paste and a frequency response difference plot in Cool Edit 2000 with the track "Perfect Fit" (note that this track, as well as one other, are offered only as 16/44 and 24/44 - there are no higher sample rate versions available; the third free track is offered as 16/44 and 24/48):

Frequency response difference (yes, that magenta dot to the left of center is it):


At least there appears to be no "malicious" intent on this one...
"Not sure what the question is, but the answer is probably no."

Naim: unfair 24/96 vs. 16/44 presentation

Reply #19
1. The CD-layer is 1.06 dB louder than the SACD layer


I don't see this in your difference graph.  Or was it something you determined by other means?  Since the SACD was captured as A/D, how did you compare overall levels of it to the digital rip?  Peak normalization followed by crest factor calculation?
Crest factor calculation? Sorry, I don´t know what that means - don´t forget, I lack some basic knowledge    You can´t see level differences with iZotope Ozone. Therefore I determined level differences with ReplayGain; then I corrected the volume with WaveLab (I lowered the volume of the CD-layer).

I determined the input levels before I started recording with some tracks from CD with the help of their EAC grabbed counterparts. As it turned out the analogue combination of Pioneer/E-MU produced level differences of around 0.05 dB (compared to the EAC rip).

I did the same test with the ASUS: it records approximately 3 dB louder - so in that case here I had to lower the volume of the SACD rip for around -3 dB. Only that way the ASUS recordings matched their EAC CD-counterpart. Since I pre-determined that -3 dB correction I used it on all my SACD recordings. And in this particular case it turned out that the CD layer was louder.
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #20
1. The CD-layer is 1.06 dB louder than the SACD layer


I don't see this in your difference graph.  Or was it something you determined by other means?  Since the SACD was captured as A/D, how did you compare overall levels of it to the digital rip?  Peak normalization followed by crest factor calculation?
Crest factor calculation? Sorry, I don´t know what that means - don´t forget, I lack some basic knowledge    You can´t see level differences with iZotope Ozone. Therefore I determined level differences with ReplayGain; then I corrected the volume with WaveLab (I lowered the volume of the CD-layer).


Crest factor is the difference (technically the ratio) between peak amplitude level and RMS (average) amplitude level, in dB.  It is a metric often offered on audio forums as a rough idea of 'dynamic range' of a track.


Quote
I determined the input levels before I started recording with some tracks from CD with the help of their EAC grabbed counterparts. As it turned out the analogue combination of Pioneer/E-MU produced level differences of around 0.05 dB (compared to the EAC rip).

I did the same test with the ASUS: it records approximately 3 dB louder - so in that case here I had to lower the volume of the SACD rip for around -3 dB. Only that way the ASUS recordings matched their EAC CD-counterpart. Since I pre-determined that -3 dB correction I used it on all my SACD recordings. And in this particular case it turned out that the CD layer was louder.


Sorry, I 'm still not sure I understand what you've done here. 

For a digital rip, the peak amplitude level of a track is 'set', it's 'hard coded' at the mastering studio.  Ideally you want to compare the levels of two rips to see what the 'hard coded' level differences are.  Otherwise you have to account for possible confounders like player output level difference for CD vs DSD D/A playback.  Furthermore for an A-D recording, you arbitrarily set the peak level (which can be anything up to the digital limit of 0dBFS).  This won't necessarily be what the peak level would be if you could 'rip' the track. 

So the problem becomes comparing apples to apples, when comparing a rip to an A-D capture.

If I read what you wrote correctly, you first calibrated your Pioneer/E-MU recording setup by comparing peak levels of an A-D capture of a CD, to a rip of the CD, and found that your chosen peak recording level happened to give you a very close level match to the corresponding rip (0.05dB difference calculated by comparing Replaygain values).  Then you used a *different* setup (Pioneer/ASUS) to do the same thing, and found that  ASUS required a 3dB cut of your initial chosen input level, to get A/D capture to match the rip (again by comparing Replaygain values)  Then you extrapolated that number (-3dB) to DSD A/D capture as a correction factor (i.e., you lowered your ASUS input level by 3dB compared to your original input level).  Then you compared the  Replaygain value of the A/D recorded DSD track to the Replaygain value of its ripped CD layer counterpart and found that they differed by 1.06 dB.  Did I get that right?










Naim: unfair 24/96 vs. 16/44 presentation

Reply #21
If I read what you wrote correctly, you first calibrated your Pioneer/E-MU recording setup by comparing peak levels of an A-D capture of a CD, to a rip of the CD, and found that your chosen peak recording level happened to give you a very close level match to the corresponding rip (0.05dB difference calculated by comparing Replaygain values).  Then you used a *different* setup (Pioneer/ASUS) to do the same thing, and found that  ASUS required a 3dB cut of your initial chosen input level, to get A/D capture to match the rip (again by comparing Replaygain values)  Then you extrapolated that number (-3dB) to DSD A/D capture as a correction factor (i.e., you lowered your ASUS input level by 3dB compared to your original input level).  Then you compared the  Replaygain value of the A/D recorded DSD track to the Replaygain value of its ripped CD layer counterpart and found that they differed by 1.06 dB.  Did I get that right?
Almost   

What you describe as "my chosen recording level" was (in case of the E-MU) the default input-level which would be 0 dBFS. I didn´t attentuate because I didn´t need to. The levels were correct right out of the box. The E-MU has analogue input levelers. I had to use the lowest input ("0") because anything louder would amplify the input. The "0" setting proved to be the right one.

The inputs of the ASUS recorded 3 dB louder with the same configuration (i.e. 0 dBFS). An example: I ripped a track from a CD with EAC (being digital). Then I recorded the same track with the Pioneer trough the ASUS. The latter was configured using the default configuration (0 dBFS for the input). Then I compared the digital rip (serving as reference) to the analogue one and found that the analogue recording done with my ASUS was too loud. I replaygained both files and the difference was roughly 3 dB (to be precise: 2.8 dB).

I continued all my recordings with the ASUS using the default configuration (0 dBFS again or 100 % inside the mixer panel). That means of course that all my recording done with the ASUS were roughly 3 dB too loud. When I had finished recording I converted the recordings to 32 bit floating point and THEN I lowered the volume by the pre-determined level for every recording.

Then I edited the files and converted them to WavPack. In case of i.e. my TELARC SACDs there weren´t any volume differences left: the SACD layer has the same level as the digitally ripped CD layer. With this SACD here it was different: the SACD layer was lower in volume (the only one in my collection btw.)
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #22
If I read what you wrote correctly, you first calibrated your Pioneer/E-MU recording setup by comparing peak levels of an A-D capture of a CD, to a rip of the CD, and found that your chosen peak recording level happened to give you a very close level match to the corresponding rip (0.05dB difference calculated by comparing Replaygain values).  Then you used a *different* setup (Pioneer/ASUS) to do the same thing, and found that  ASUS required a 3dB cut of your initial chosen input level, to get A/D capture to match the rip (again by comparing Replaygain values)  Then you extrapolated that number (-3dB) to DSD A/D capture as a correction factor (i.e., you lowered your ASUS input level by 3dB compared to your original input level).  Then you compared the  Replaygain value of the A/D recorded DSD track to the Replaygain value of its ripped CD layer counterpart and found that they differed by 1.06 dB.  Did I get that right?
Almost   

What you describe as "my chosen recording level" was (in case of the E-MU) the default input-level which would be 0 dBFS. I didn´t attentuate because I didn´t need to. The levels were correct right out of the box.



?  A recording device 'default input level' of 0dBFS does not guarantee that recorded peaks stay under 0dBFS (unless you are talking about a limiter).  It may happen in fact that the default level gives you a good (high but safe) peak level, but there's an element of risk.  Typically if I'm familiar with the material, I'll do a test recording of the loudest part, and see if peaks stay under 0dBFS when using the setup's 'default' input level.  If not, I adjust the input level downwards.


Quote
The inputs of the ASUS recorded 3 dB louder with the same configuration (i.e. 0 dBFS). An example: I ripped a track from a CD with EAC (being digital). Then I recorded the same track with the Pioneer trough the ASUS. The latter was configured using the default configuration (0 dBFS for the input). Then I compared the digital rip (serving as reference) to the analogue one and found that the analogue recording done with my ASUS was too loud. I replaygained both files and the difference was roughly 3 dB (to be precise: 2.8 dB).


OK, this is calibration to determine a correction factor. It works because you have a reference (the rip) to calibrate to.  But the difference you're correcting for -- playback-end level difference between a direct digital rip and an analog-to-digital capture -- has two potential sources:  your chosen recording level, and your player's output level.


Quote
I continued all my recordings with the ASUS using the default configuration (0 dBFS again or 100 % inside the mixer panel). That means of course that all my recording done with the ASUS were roughly 3 dB too loud. When I had finished recording I converted the recordings to 32 bit floating point and THEN I lowered the volume by the pre-determined level for every recording.



However, for DSD, there is no rip as reference...and while the ASUS would behave the same regardless of analog source, it's conceivable that the player output level is natively different for DSD format versus CD format.  This would throw your correction factor off, since it's calibrated to CD playback. 

This is why a difference or ratio-based number is better than simple peak level, as a comparison metric.  It scales. The  ratio of peak to average level for a given signal remains basically the same across the range of useful and 'legal' A-D recording levels.  Your method may suggest that a DSD rip of your track would really be 1 dB lower in peak amplitude level than the CD layer rip, but you can't know -- because you can't do a DSD rip.  But you can be pretty sure that the difference in crest factors would be the same regardless of method (assuming no compressors/limiters in the A/D chain.)

Naim: unfair 24/96 vs. 16/44 presentation

Reply #23
This is why a difference or ratio-based number is better than simple peak level, as a comparison metric.  It scales. The  ratio of peak to average level for a given signal remains basically the same across the range of useful and 'legal' A-D recording levels.  Your method may suggest that a DSD rip of your track would really be 1 dB lower in peak amplitude level than the CD layer rip, but you can't know -- because you can't do a DSD rip.  But you can be pretty sure that the difference in crest factors would be the same regardless of method (assuming no compressors/limiters in the A/D chain.)
Allright. Then you all may ignore my statement that the SACD layer of that particular recording is 1.06 dB lower in volume as I can´t be sure about my SACD player. It obviously misses some vital points (which I can´t redeem since I´m unable to do a digital SACD rip).

Can we now return to the topic at hand, please? 
marlene-d.blogspot.com

Naim: unfair 24/96 vs. 16/44 presentation

Reply #24
Arnold,

> Do you expect a quart of milk to taste different than a gallon of milk when they have the same name and come from the
> same dairy on the same day?

To me, your analogy seems a bit too far removed to work.

> Can we infer from this example that Naim's less expensive CD players have their bass rolled-off below 100 Hz so that
> people who are interested in sonic accuracy will be forced to buy their more expenisve models?

Again, I am not sure about your example but I think it is safe to assume that Naim will design their products so that they can hook into audiophile beliefs. It is one of the major things that is used to distinguish products that cannot be effectively distinguished on grounds of relevant technical performance.