"...converts digital audio in graphics data and then performs effect calculations using the 3D architecture of Nvidia GPUs. Compared to the capability of just six GFlops of a typical CPU, Nvidia's chips can reach more than 40 GFlops."

Process Audio with Video Card

so your saying instead of using your CPU, you use your gfx card's cpu? that sounds hokey.. i've got a 3.2 Ghz P4 with 512 L2 cache, that is surely more powerful than my Radeon 9800 PRO's cpu. Nvidia's cant be any better, ATI makes the fastest gfx card right now.

The site I remembered had an actual SDK for this purpose, however this seems to be of a similar purpose:
http://www.gpgpu.org/

if you are so sure. then please tell how you know ?

Your CPU is way more powerful - at the majority of general purpose computing applications. It is designed from the ground up to be completely programmable. It contains a number of (very deep in the P4's case) pipelines for increasing instruction throughput and sophistcated branch prediction logic to keep them full. It supports multiple tasks, memory protection, a variety of IO operations and all sorts of other things the GPU lacks. In other words - your GPU doesn't have a hope in hell at keeping up with your CPU at general purpose computing.

On the other hand, your GPU is way more powerful at a selected number of operations specific to rendering graphics. It supports a limited level of programmability, extremely fast vector and matrix math, dedicated blending hardware and all sorts of other things that make it much faster than the CPU at being part of a graphics pipeline.

Many of the tasks that are needed to process sound fast overlap with those needed to process graphics fast - hence the GPU is very useful as a sound processor.

A GPU is what Intel's MMX failed to be

Not quite so.
But anyway. You don't seem to know how much MMX realy speeds up your every day aplications in your computer.

I remember when this was first posted. Anyone heard anything more or even better, tried it yet?

No, but I'd bet it will be successful.
128-bit floating point through the normal texel pipelines (how they'd use those I don't know), 16, 32, or 24-bit using shaders, and faster than a decent CPU in most, if not all, cases.
Typically 8-16 pixel pipelines and 3-8 shader units (soon to be 16+, if ATi isn't blowing smoke), running at 300-700MHz. Not a lot of power if decision-making is involved, like CPUs are made for; but for doing vector work, blazingly fast.

The only trick is how one deals with translating sets of data. It seems like it would have to translate the framebuffer. There can be several of them. Of course, it could also be possible to use the generic video processor thingy on the 6x00 series.

This is definitely a situation where PCI-E is going to make life very good, as it has bandwidth to spare and is full duplex.

It will also be an ironic day when you can recommend a Geforce 7800 Ultra to a guy who is using his PC for mixing music

Yeah, so I think it's time for APU (Audio Processing Unit) engineers to at last get to work and implement such technologies in the audio chips of our sound-cards. It's not hard to notice that audio-acceleration technology development is far behind technologies connected with video and general processing. In 10 years CPUs switched from 32bits to 64bits, from 40MHz to 4000MHz, from 32kbyte caches to 2MB ones, got MMX, SSE, HT and other stuff. In 5 years GPUs got shaders, are more and more programmable, I bet that in few years GPUs will have real ray-tracking technologies implemented, not to mention about programmable encoding & decoding of compressed video which more or less is already available in the newest ATI & nVidia cards. And what do we have in APUs? Hardware mixing. Sometimes simple environment simulation technologies like EAX. Hardware decoding of stuff like MPEG or Dolby is still rare and very expensive, hardware audio encoding is even more rare and expensive. One could say that all we got since 20 years is more speaker outputs, higher frequencies, and more bits. I should have a sound card with powerful programmable APU a long time ago, so all that encoding & decoding of MP3, OggVorbis, MPC, or OptimFROG wouldn't have to be the job of my CPU. If audio-processing technology development were more brisky, you could even dream about ray-tracking-alike technologies in your sound-card (air-tracking?), so primitive stuff like EAX would be obsolete ...then you could even program the shape of trumpet, the strenght of blow, and your APU would compute the sound of trumpet Instead of that I am going to have reverb performed by my Radeon Very interesting but wrong way.

To add to all this my 6 year old SB AWE 64 is still the best sound card I have. In the way of actual stereo sound quality for 16bit sources common sound cards have regressed over that time period. I actually wish my computer had an ISA slot .

Not likely. Even hollywood has gone to using raster shortcuts. However, video cards are catching up quite fast.Fully available in NVidia, but requiring special support and there are still some kinksYes, but with quality differing like it does...eh.Unfortunately the engineers that matter when into doing games.Aside from use in games, isn't it?
What would be equally nice would be for CPUs to have dedicated vector units, natively able to deal with 32- abd 64-bit floats, and natively manufacture 128-bit ones. A fully powered processor that did nothing but series of SIMD tasks on data. Or for NVidia and ATi to get into sound, and make it a commodity part of the GPU--integrate the shaders and generic processing to just be vector units with some other units off to the side that controled how the output was to be used.
The neat thing about using video cards is that the power is already there. It's merely a matter of unwrapping it and getting it to main RAM (something PCI-E should be a boon for). Also, you won't be doing 3d work near that complex at the time (at most, using Aero or Looking Glass for the desktop), so the power is just sitting there, waiting.

For those who previously mentioned CPUs being more powerful, here's a little, VERY high level, info on how it works:
A modern processor has pipelined units. ALU and FPU being the important ones for general tasks. The pipeline has stages, each stage being 1 clock cycle, and doing a portion of the work. A longer pipeline helps get the processor to higher speeds, though it does less work per clock cycle, and runs into power consumption issues (Prescott).
So, as an example, a Prescott Pentium 4 has a 31 stage ALU pipeline (arithmatic and logic--it does booleans and integers, basically), and an Athlon64 has 12 stages in that pipeline. So if you need to check the value of something, you may do several parts at once, using 3 or 4 of the stages, then at the end, use another few pipeline passes to put them together, then use one or two to test them (=, <, >, !=, etc.). Now this takes time, and is at the mercy of memory. Most of the CPU is made up of cache memory. If what it needs cannot be found in cache, it must go out to main memory, which is much slower. The longer the pipeline, the worse of a performance hit this is.
So, all the floating point must be balanced together. All the integer and boolean maths must be balanced together, even if what they need to do differs.
Well, you could put multiple cores on a CPU. Doing that. Hyperthreading helps a bit, too. But still, you've got one integer and one floating point math unit to do the work. In most tasks, any more would not net performance gains, since what you're most often waiting on are serial tasks, where one part must be completed to see what needs to be done next.

Now video cards are entirely different. They don't do much decision making. Shaders are changing that a little bit, but still not much. The vast majority of work requires very simple decision making, and whole tons of basic operations on large data sets. This means that those 170 or so million transistors of a new GPU can do real work, not all that programmed checking values and moving things around. Also, not having much to check, pipelines can be outrageously long, in the hundreds of cycles, to get higher speeds. For powering Word, a typical GPU would probably be comparable to a VIA C3. On the other hand, for doing high precision multiplication, division, addition, and exponential work on several large arrays of numbers, it would take a big iron machine, like a Power, to reach the kind of performance of a Radeon 9800.

There are hardware audio DSP cards that do quite a bit more than that, including extremely high quality synthesisers, reverbs, equalizers etc. Creamware, Powercore and UAD, for instance, bring pro quality sound to a common PC. It's true they haven't developed as quickly as graphic chips but they do quite a bit more than a high-end soundblaster is capable of.

I'm not sure what you mean here...

You didn't understand. What I meant by ray-tracking-alike technology in a sound-card (which I called "air-tracking") was a technology that would replace things like EAX. AFAIK in EAX you define things like reverbation, delays, and filtering to simulate for example underwater environment, open space, or a tunnel. In for example a race game you need to program these things and trigger them when the car enters the tunnel, and turn off when the car exits it. If you had that "air-tracking" technology, you would only need to define things like density of the environment (dense water or thin air), what material are obstacles made of (concrete tunnel, wood trees), and where they are and how big they are and what's the shape of them (which is already defined in the game's map). Having that data your APU would compute when and how things should sound like. You wouldn't need to sample a BOOM sound and trigger it when large rock falls on to the ground because APU would compute everything. Just like in ray-tracking you don't need to paint every single shadow, you only put an object and define where the light source is, rendering engine tracks the light rays and decides about the shadows. Something like that would completly replace EAX and other things, would give incredible possibilities and obviously could be used in games.

...to have another burden. That's wrong way. Things should be split between specialised processing units. 15-20 years ago PCs were useless for anything but databases, text processing and things like that because everything had to be done by one processing unit, a CPU. Even most simple platform games were hard to do becuase CPU was too weak to do side-scrolling, not to mention the other things you need for a game. You can compare any game made between 85 & 90 for Amiga500 and any PC, you would be surprised by the difference. Amiga500 had only 16bit 7MHz CPU without any caches, but also had separate processing units for video, audio, I/O and stuff, almost everything with DMA so CPU wasn't arsed even by data transfers. Current PCs are that powerful because things got split too. If you want multimedia you need to feed your 3 senses with 3 things, brain with CPU, eyes with GPU, and ears with APU. If you tried to feed them with CPU only you would still play MineField, to play HalfLife2 you would need a separate room for radiator of your 100GHz CPU.

Yeah, that's better than nothing. But specialised APU would still be better due to the reasons mentioned above.

Now let me rant about that too. I don't like Intel's way of increasing pipelines lenght & cycle frequency. AMD's way of implementing new technologies instead of more gigahertzs seems much nicer. And RISC processors (like G ones used in Macs) which have very plain instructions set, but all of these instructions are made in 1 cycle, are IMO a good example of how CPUs should be - do what you should do, and do it fast, don't be arsed with multimedia extensions like MMX or SSE, leave them for specialised GPUs & APUs.

I assume you are referring to wavetable synthesis? Yes the technique has come a long way from the 1980's when compared to analog synthesizers with various oscillators and filters (still nifty for electronic music though in my opinion). and FM synthesis which was pretty realistic for percussion instruments., but still just an approximate for everything else. Physical Modelling Synthesis is a lot more realistic in a sense especially for wood-wind and stringed instruments. I can't tell the difference. The only thing is it's very expensive and not everyone is going to use same card that has same instrument capabilties, etc god for bid we have to rally people these day's just to dump the damn integrated audio and buy a descent sound-card. If I have to yell a one more person that complain's about MIDI instrumentation without prior lack of knowledge I will throw a fit.



This does seem like very intuitve idea. Why not extend the APU for MIDI capabilities as well? Maybe even it's own programming language (just throwing idea's out their). ;-D.

Here I would disagree--on the extensions, anyway. Being stuck with x86, that's how it should be done, and the extensions have gotten a lot fo good use. That's what we've got, due to economic factors in the late 80s and early 90s. Performance keeps up with the RISC guys, though the few people I've talked to who do assembly work have nothing good to say about doing x86 assembly. On ramping frequency, Intel noticed it wasn't that good too, and instead of planning for 5-7GHz CPUs, killed the 4GHz P4 and said they'd make new ones based on the Pentium-M. They're still going to do dual-core Prescott 3.6GHz Xeons, though.