Canuck Audio Mart Hifi and Audio ForumCanuck Audio Mart Hifi and Audio Forum
It is currently Sat Jun 23, 2018 5:40 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 1:48 pm 
Offline

Joined: Sun Oct 05, 2008 7:23 pm
Posts: 1098
Location: Victoria, BC, CA
http://www.diyaudio.com/forums/digital- ... oever.html

Hi all,

This same person has done midrange and tweeter tests, both of which are also super interesting.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 3:04 pm 
Offline

Joined: Mon Jan 31, 2011 6:58 am
Posts: 294
Location: Brampton, ON, CA
If you can't tell the difference then you can certainly save a lot of money. My feeling is that with audio, once you hear certain things (even if you believe you hear certain things), you can't 'unhear' them.

Again, my opinion based on fallible experience, is that some dacs, or really any source, can completely change the 'personality' of a system. Some dacs are hyper detailed, some more dynamic, some smoother, some more 'natural' sounding. I really don't know how someone can say they sound 'the same' if simply matched for 'output level'.

There just seems too much variance in dac design and choices by designers for this particular component to all sound the same. I struggle to see how a power cord makes a difference but these are much more complex than a power cord.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 4:25 pm 
Offline
User avatar

Joined: Thu Jan 07, 2010 3:45 pm
Posts: 930
Location: Rockwood, ON, CA
The only way to AB test is to listen to A for a week, listen to B for a week then go back to A. Dont know why people continue to think that differences can be detected instantaneously at the flick of a switch. You have to live with something long term to appreciate what it has to offer. A bit like a relationship.

_________________
Decware SE84UFO2... Naim Uniti Atom... Omega SAM's... TEO GC Liquid Cables... Equi-Core 1200 Power Conditioner... RaspberryPi... Hifiberry Amp/DAC2... Cambridge Audio WS30.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 4:46 pm 
Offline

Joined: Sat Sep 01, 2012 5:08 am
Posts: 628
Location: Montréal, QC, CA
RandyB wrote:
There just seems too much variance in dac design and choices by designers for this particular component to all sound the same.
+1
milesian wrote:
The only way to AB test is to listen to A for a week, listen to B for a week then go back to A.
+1

_________________
Nakamichi BX-300/Balance BW-1010 DAC, Balance MR-50R, Infinity RS 8 Kappa
Nakamichi Dragon/Onix Dac25b, Audio Research SP-9, Spectral DMA-100s/Nakamichi PA-7II, Martin-Logan CLSII/Balance Helicia


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 4:58 pm 
Offline

Joined: Thu Aug 29, 2013 5:48 pm
Posts: 399
Location: Vancouver, BC, CA
milesian wrote:
The only way to AB test is to listen to A for a week, listen to B for a week then go back to A. Dont know why people continue to think that differences can be detected instantaneously at the flick of a switch. You have to live with something long term to appreciate what it has to offer. A bit like a relationship.

Maybe because some people can hear the differences straight away with a flick of a switch? For me, I need to do 5 - 10 flicks of the switch to start hearing the differences, but when I do they are locked in. Having said that.... I also like to have a piece of new gear in my system for a week before making a decision. I like to do both :)

_________________
TT VPI Prime Signature/Benz Micro LP-S DAC Lampizator Golden Atlantic Preamps Mac C500T, Mac MX121 Amps Mac MC75 60th Anniversary (*2), Mac MC205, Glenn 300B Speakers Dynaudio C1 Platinum, B&W 804S


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 5:22 pm 
Offline
Premium User
User avatar

Joined: Tue Dec 19, 2006 11:44 pm
Posts: 6479
Location: Toronto, ON, CA
RandyB wrote:
If you can't tell the difference then you can certainly save a lot of money. My feeling is that with audio, once you hear certain things (even if you believe you hear certain things), you can't 'unhear' them.

Again, my opinion based on fallible experience, is that some dacs, or really any source, can completely change the 'personality' of a system. Some dacs are hyper detailed, some more dynamic, some smoother, some more 'natural' sounding. I really don't know how someone can say they sound 'the same' if simply matched for 'output level'.

There just seems too much variance in dac design and choices by designers for this particular component to all sound the same. I struggle to see how a power cord makes a difference but these are much more complex than a power cord.


If you can't hear a difference you might recognize a difference in build quality.

Build quality is very important to me...I don't want to be in a position where I am waiting for something to be repaired in order to hear it...if you know what I mean!!! :wink:

_________________
I Put A Spell On You!!!


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 5:32 pm 
Offline

Joined: Tue Sep 25, 2012 1:54 pm
Posts: 580
Location: Calgary, AB, CA
Sounds like they need to listen to better dacs lol.

Even the Eximus DP1, at it's pricepoint, was announced at the end of 2011. That's ~6 years ago. I'm sorry but dac technology has changed over that time. Not only that but for the price, you're spending a lot on a fancy case. Just a guess but the innerds are probably nothing to get excited about.

Take Schiit for example. Take their cheapest dac, the Fulla 2 ($99 USD) and compare against their flagship, Yggdrasil ($2399 USD). I had both of these and A/B'd them a lot just out of curiosity. The differences were obvious, way more metallic sheen and weaker, rounder bass were the biggest things on the Fulla 2.

That said, at the end of the day if I have to sit and A/B something and squint just to tell the difference, it probably isn't worth the cost differential.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 5:46 pm 
Offline

Joined: Mon Jan 31, 2011 6:58 am
Posts: 294
Location: Brampton, ON, CA
Voodoo Funk wrote:
RandyB wrote:
If you can't tell the difference then you can certainly save a lot of money. My feeling is that with audio, once you hear certain things (even if you believe you hear certain things), you can't 'unhear' them.

Again, my opinion based on fallible experience, is that some dacs, or really any source, can completely change the 'personality' of a system. Some dacs are hyper detailed, some more dynamic, some smoother, some more 'natural' sounding. I really don't know how someone can say they sound 'the same' if simply matched for 'output level'.

There just seems too much variance in dac design and choices by designers for this particular component to all sound the same. I struggle to see how a power cord makes a difference but these are much more complex than a power cord.


If you can't hear a difference you might recognize a difference in build quality.

Build quality is very important to me...I don't want to be in a position where I am waiting for something to be repaired in order to hear it...if you know what I mean!!! :wink:


My feeling is that a dac is typically one of the most reliable components in the audio chain and build quality in the sense of reliability is probably pretty decent if only a $30 fiio dac that was referred to in the OP's linked to thread. I would suspect some of the more expensive dacs may be more repairable, but also may have more to fail. Also, for $30, you could have a back up dac or replace it pretty quick from Amazon.

Personally, if I paid $3000 for a dac, it better sound a lot better to me than the $30 Fiio and I better appreciate something in the build quality, the design and parts used to justify something making it 100 times more expensive. I believe the OP's linked to thread was implying in an ABX blind test it is very difficult to distinguish between two dacs valued at $30 and another at $3000. If the implication was true (which I believe it generally is not), I could not justify the price paid just for build quality alone.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 5:47 pm 
Offline
Premium User
User avatar

Joined: Tue Dec 19, 2006 11:44 pm
Posts: 6479
Location: Toronto, ON, CA
RandyB wrote:
Voodoo Funk wrote:
RandyB wrote:
If you can't tell the difference then you can certainly save a lot of money. My feeling is that with audio, once you hear certain things (even if you believe you hear certain things), you can't 'unhear' them.

Again, my opinion based on fallible experience, is that some dacs, or really any source, can completely change the 'personality' of a system. Some dacs are hyper detailed, some more dynamic, some smoother, some more 'natural' sounding. I really don't know how someone can say they sound 'the same' if simply matched for 'output level'.

There just seems too much variance in dac design and choices by designers for this particular component to all sound the same. I struggle to see how a power cord makes a difference but these are much more complex than a power cord.


If you can't hear a difference you might recognize a difference in build quality.

Build quality is very important to me...I don't want to be in a position where I am waiting for something to be repaired in order to hear it...if you know what I mean!!! :wink:


My feeling is that a dac is typically one of the most reliable components in the audio chain and build quality in the sense of reliability is probably pretty decent if only a $30 fiio dac that was referred to in the OP's linked to thread. I would suspect some of the more expensive dacs may be more repairable, but also may have more to fail. Also, for $30, you could have a back up dac or replace it pretty quick from Amazon.

Personally, if I paid $3000 for a dac, it better sound a lot better to me than the $30 Fiio and I better appreciate something in the build quality, the design and parts used to justify something making it 100 times more expensive. I believe the OP's linked to thread was implying in an ABX blind test it is very difficult to distinguish between two dacs valued at $30 and another at $3000. If the implication was true (which I believe it generally is not), I could not justify the price paid just for build quality alone.


I agree with you...$3000.00 buys a LOT of $30.00 DAC's!!! :lol:

_________________
I Put A Spell On You!!!


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 6:55 pm 
Offline

Joined: Sat Sep 01, 2012 5:08 am
Posts: 628
Location: Montréal, QC, CA
Check on the net, wiki, specialized forums, humans have an aural memory on short time.
Few seconds... making useless AB tests... Listen A, when switching to B, the brain still has
(too) fresh memory of A. It's then very difficult to compare B to A.

Unless the difference be big, obvious, AB tests are useless.
ABX tests are even worst. The fact to not know X just 'mess up' the brain.
Impossible then to get an useful information from those tests.

Also another thing: our ears are more sensitive to change in level than any other change.
Then, since the components are calibrated at nearly the same level for comparison,
the test becomes even more difficult.

Milesian is right, long term testing is the way to go.
After all, we are talking about many $$ and the fact that we will have to live with that component.
It's clearly not the time to decide in few minutes of testing... right ?

Yes Voodoo Funk, build quality is not always obvious with quick tests.
However, it is as you said, very very important...

_________________
Nakamichi BX-300/Balance BW-1010 DAC, Balance MR-50R, Infinity RS 8 Kappa
Nakamichi Dragon/Onix Dac25b, Audio Research SP-9, Spectral DMA-100s/Nakamichi PA-7II, Martin-Logan CLSII/Balance Helicia


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 7:06 pm 
Offline

Joined: Fri Aug 15, 2008 12:09 pm
Posts: 37
Location: Edmonton, AB, CA
Maybe I missed it, as I wasn't prepared to read all 150+ pages, but what % does the tester consider a "Pass"?
He stated that every participant "FAILED to identify on every music excerpts.".

Does that mean:

The participants failed to identify "every" time, in other words, 100% of the time? That seems unlikely, as even guessing results in approx 50% correct.

Or..

The participants failed to identify "every" (as in "all") music excerpts correctly? In other words, could have gotten 9 out of 10 correct and one wrong, therefore "FAILED to identify on every music excerpts."?

It's unclear.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Sat Feb 10, 2018 7:48 pm 
Offline
Moderator
User avatar

Joined: Tue Feb 22, 2005 2:06 pm
Posts: 5103
Location: Montreal, QC, CA
That topic thread in the OP's link is 155 pages long... tl;dr

One of the very simple points which should always be made when the subject of blind A-B and A-B-X testing with short musical excerpts comes up, is that the test protocol is not a normal use of the equipment.

IOW we do not listen at home to snippets of music a few seconds long, then endeavour to identify the gear which played them. Instead, what we do is listen over longer periods to the same gear, playing a variety of pieces, some of the pieces more frequently than others.

Occasionally a pal may drop over and play a piece of music for us, then ask who we think the musicians are. That's about as close to A-B-X discrimination as we get in most of our lives.

Now that's not to say that the gear can't be used for A-B-Xing. I only want to clarify that its normal-use protocol is being bent into an outlier scenario. Consequently, any conclusion drawn from the A-B-Xing is a bent one, if it is applied to normal listening.

The range rule applies : equipment should be tested in the situation in which it is intended to be used.

Additionally, the mention of short auditory memory, the placebo effect and expectation bias, all of which the fast A-B-X protocol is intended to obviate, are relevant in other test situations, but are bent when applied to choosing audio gear.

There are folks posting to that topic thread who use fast A-B switching to earn a living. They are audio engineers and mixers. They develop, over years, the listening skills needed to work this way. For their purposes, a $30 DAC and a $3000 DAC may be equally useful. They are not normal home listeners.

Curiously, as I read one pro mixer's comments ( in the linked thread ) on how he developed his ability to use fast A-B switching to hear minute differences in sound, I thought of what I had read about the development of one particular fairly modern DAC. It was created by sound professionals and its design choices were supported by carefully developed technical and aesthetic positions. I remember listening to the DAC in question. At any given split-second, the sheer quantity of apparent detail was extraordinary, so much so that trying to hear it all actually became exhausting. The pro mixer I am thinking of makes the point that A-B-switch critical listening is exhausting. At my own listening session, after changing to another DAC, also designed by sound pros as it happens, the exhausting detail gave way to an emphasis on sonic flow and overall musical involvement. Also, the weakness in the high end of the first DAC became clear.

I wonder if the group which designed that first DAC made their most critical design choices by A-B switching. Only speculating, mind.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Thu Feb 22, 2018 9:08 am 
Offline

Joined: Fri Jun 05, 2015 3:01 am
Posts: 70
Location: Windsor, ON, CA
Differences between DACs are usually quite slight, when you look at it objectively. But it's hard to look at things objectively.


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Thu Feb 22, 2018 9:25 am 
Offline

Joined: Mon Apr 26, 2004 2:33 pm
Posts: 3185
Location: Toronto, ON, CA
milesian wrote:
The only way to AB test is to listen to A for a week, listen to B for a week then go back to A. Dont know why people continue to think that differences can be detected instantaneously at the flick of a switch. You have to live with something long term to appreciate what it has to offer. A bit like a relationship.


My experience has been exactly the same. The entire premise of instantaneous switching as an objective method of comparing how we perceive the performance of different components is, at least based on my own 40+ years; experience in the audio hobby, a major red herring.

Exactly as milesian has expressed it, listening to A for several days, then switching to B for a comparable length of time, then switching back to A is the minimum requirement for being able to intelligently compare two components—or for that matter, two separate systems.

So I don't place much confidence in blind ABX-type "testing" of audio components.

-- Thu Feb 22, 2018 11:26 am --

icevic wrote:
Differences between DACs are usually quite slight, when you look at it objectively. But it's hard to look at things objectively.


Not sure what led you to this conclusion. Care to share?


Top
 Profile  
Reply with quote  
 Post subject: Re: Blind DAC Tests
PostPosted: Thu Feb 22, 2018 10:01 am 
Offline
User avatar

Joined: Tue May 21, 2013 12:54 pm
Posts: 5921
Location: Dodge City, NB, CA
icevic wrote:
Differences between DACs are usually quite slight, when you look at it objectively. But it's hard to look at things objectively.


I went to visit a pal yesterday who wanted to hear some of my equipment as he is trying to decide the next logical step in improving his system. When we changed from his DAC to mine we had to put the bass port plugs back in his speakers. I think you can hear the difference between DAC's and I'm pretty sure he would agree. :lol:

_________________
BE KIND TO THE UNKIND...THEY NEED IT THE MOST.
Especially Trump supporters, nature already played a cruel joke on them.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 8 hours


Who is online

Users browsing this forum: two2the8 and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group