Magic? No, NeoMagic

Why is parallel processing important, and why are we dedicating a whole feature to it in the pages of Pen Computing Magazine? Because smartphones and other mobile devices are about to get a whole lot smarter. Because mobile digital TV (MDTV) is just around the corner and it could be a multibillion dollar market in just a few years. And because one company seems to hold the key to an important part of that future. That company is NeoMagic, headquartered in Santa Clara, California, right in the heart of Silicon Valley.

Who is NeoMagic?

Not all of that billion-plus cellphones will have next-generation multimedia wizardry built in, but a significant percentage will. Cool features in phones sells phones -- and lucrative service plans. Digital cameras are almost a standard feature in cellphones now. And we're no longer talking fuzzy VGA snapshots. Samsung recently revealed a 10-megapixel cameraphone (the SCH-B600), and it supports Korea's satellite digital multimedia broadcasting services. The future of cellphones, and not just cellphones, is in dramatically increased multimedia that surpasses much of what until recently was only seen in sci-fi movies. We're talking super-hi-res imaging, highly compressed video using the MPEG-4 and H.264 standards, 3D graphics for sophisticated gaming, and, of course, audio.

What is an application processor?

An application processor is not a run-of-the-mill general purpose processor. In a phone, for example, or some other wireless device, that task falls to a so called baseband processor that handles the wireless aspects, headphones, speakers, microphone and related functions. The application processor, on the other hand, has its own media engine, processor, memory and handles I/O, the display, expansion slots, imaging -- everything that requires maximum performance.

The trick then becomes to have application processors that support whatever operating systems, baseband processors, and wireless communications protocols vendors (and the public!) want. And what does the public want? Ever more powerful devices that can do incredible new things. And they want that in ever smaller, ever slimmer phones. And without giving up the battery life they've become accustomed to. In fact, they expect even longer battery life.

The battery life dilemma

Used to be that short battery life was blamed on ever larger color screens. But that argument doesn't wash with next-gen mobile phones as their screens are not very large. Much of the power is eaten up by the processor, so that is where the savings are to be found. But how are those savings accomplished? After all, more performance means more power consumption, and what we're after is more performance but less power consumption.

NeoMagic's magic

And for a good reason: in most processors, a faster clock speed is better. More cycles get more work done. The processor uses a lot more power and gets plenty hot, but it has fans to cool it down and it's plugged into the wall. Phones and other mobile devices don't have that luxury. They use a battery and they are not supposed to get hot. So simply jacking up the clock cycles isn't a very good idea. But are there alternatives? There are. NeoMagic has one.

Parallel processing

Why is NeoMagic's programmable parallel processing approach so important and revolutionary? Because the multimedia challenges are rapidly becoming overwhelming to most chip foundries. Digital imaging, two-way MPEG-4 video, interactive online gaming, etc., are so processor intensive that the only alternative to very high (and impractical) clock speeds is to add dedicated specialized hardware accelerators or shoehorn a co-processor, such as a DSP, onto the main CPU chip. Those approaches are not programmable, and if scaled to deliver more performance, power consumption again becomes a major issue.

How does NeoMagic's massively parallel APA work? It is based on a "single-instruction multiple-data" (SIMD) architecture that uses 512 data words of 160 elements each per clock cycle. Each of these elements has its own storage and logical processing. To make it all work requires some clever programming. For that, NeoMagic provides a suite of library functions and middleware to, among other multimedia tasks, implement 3D acceleration and video codec processing.

As a result of the MiMagic 6+ APA core's ability to run complex video algorithms, the video subsystem supports four modes: video conference, portable video playing, video camera, and still camera. It passes MPEG-4 tests and can run H.264, can do full-duplex 2-way video, adds video camera post-processing features, and also does 2D/3D graphic acceleration, with 3D acceleration approaching, or surpassing, that of dedicated game consoles of just a few years ago.

But what about the common concern about parallel processing, the fact that a lot of data simply cannot be processed in parallel and needs to be broken down into sequential streams, thus reducing the advantage of parallel processing or rendering it moot? Not the case with multimedia. Multimedia is just about perfect for parallel processing. Which means fewer clock cycles, and less power needed.

NeoMagic's vast patent portfolio -- a portfolio so attractive that techo-giant Sony has licensed it -- includes another neat trick. The company's on-chip embedded DRAM (dynamic random access memory) technology in graphics controllers was the industry's first commercially successful use of embedded DRAM in its graphics controller days, and it helps make the current MiMagic processors more efficient. NeoMagic found ways to merge processing and memory in a manner that eliminates moving data from local cache into the processor and back. A seven-layered concurrent internal bus handles internal traffic. That both lowers power consumption and increases performance. In essence, NeoMagic drew upon its experience and patents to merge computing and memory and thus eliminate bottlenecks. And all of this fits into a tiny 13 x 13 x 1.2 mm package.

And the company isn't standing still. The MiMagic 6+, a superset of the MiMagic 6, sports additional integrated peripherals and further power optimization. There is USB-OTG (USB On-The-Go; a USB 2.0 standard for connecting devices without the need for a host PC), Sony Memory Stick support, an LCD bypass mode and a generally optimized chip layout.

Handling programming complexity

Still, the question comes up whether the development tools are mature enough to allow programmers to quickly create their own routines. Syed Zaidi, Vice President of Corporate Engineering, says "Now we do have assemblers and C compilers available to program APA," but allows, "Having said that, it is true that APA is a massively SIMD parallel processor engine and optimization needs some experience." "Not a problem," says Jeffrey E. Blanc, the company's VP of Worldwide Sales and Corporate Communications. "Every client has full access to our team of APA programming specialists, and we deploy them to our clients." And while NeoMagic is not a large company, most of its employees are researchers, almost a hundred. Special assistance comes from NeoMagic's Israel facility where, under the leadership of Dr. Avidan Akerib, much of NeoMagic's APA R&D is being conducted. Akerib, a mathematician, has over 20 years of experience in image processing and parallel computing architectures.

NeoMagic's competitive advantage

So if Intel isn't NeoMagic's competition, who is? That would be the likes of Korea's Core Logic with its ARM-based Hera Multimedia Application Processors that use hardwired video codec technologies; Japanese Renesas Technology Corp. which is now in charge of the former Hitachi SuperH CPU core, develops the DSP-favoring SH-Mobile application processors and also presides over the SH-Mobile Consortium; STMicroelectronics' Nomadik family of multimedia processors that use McubeWorks' H.264 software video decoder; Texas Instruments with its hardware-accelerated OMAP application processors, and several others. Generally, decoding and processing is being done in hardware with dedicated accelerators and DSPs. That means extra overhead and extra power. NeoMagic's APA processor, on the other hand, is not only used for standard video decoding, but also for OpenGL-ES 3D operations, image enhancement such as color correction, stabilization, histogram equalization and so on. If anything else is required, it's a software routine away. No need to add more hardware.

Unfortunately, there aren't any standard benchmarks available to directly compare application processors. Alternate methods include running the video codec of choice -- mostly H.264 decoding in this field -- and then check video quality and power consumption. The MM6+ chip decodes H.264 at 30 frames per second on a QVGA screen at an extremely power-efficient clock speed of less than 100 MHz.

Mobile Digital TV solutions

Given that there are competing standards (DVB-H, MediaFlo, and T-DMB), flexibility, programmability, and generally being nimble on your feet will be crucial. The prize is a big one. British audiovisual research firm Screen Digest predicts that over 250 million MDTV cellular handsets will be shipped by 2012, with a 43% Asia, 32% US, and 25% Europe split. A NeoMagic powered handset with 30 frames-per-second video, blindingly fast Java apps, console quality 3D performance, and long battery fits beautifully into this scenario. And the company has already demonstrated audio/video processing solutions for the most promising mobile digital TV standards, including ISDB-T (a Japanese digital standard), DMB (already used in Korea, and soon in several European countries), and DVB-H (commercial launches expected this year in Italy and the US).

Here again, the trick will be for NeoMagic to convince some of the Taiwanese OEMs and ODMs to use the chip in their MDTV-enable PDAs, smartphones and other convergent devices. The first examples of such phones are now showing up. Taiwanese PDA and smartphone expert HTC has been showing their "Trilogy" platform, a digital TV and radio phone with a 2.2-inch screen, and a 1.3mp camera.

NeoMagic's answer is the NeoMobileTV product aimed at all global mobile TV standards. This will be available in mid 2006 and its goals are to add mobile digital TV not only to high-end phones, but even those selling in the US$150 range (with the NeoMobileTV part contributing less than US$10 to the bill of materials). What that means is that we'll be able to watch, and listen to, high-quality H.264, Windows Media 9, MPEG-4 or DiVX digital video almost anywhere. For how long? Well, the NeoMobileTV solution promises to use well below 100 mw of power under load. That means that depending on the hardware used, you can watch between four to eight hours.

"Our NeoMobileTV product focuses on the handheld space, mostly cellphones. It has H.264, MPEG4, DiVX, WMV and other video decoders running on APA and we're also providing a complete customized MDTV system software solution. Our customers are seeing better performance at lower megahertz, which means lower power consumption and lower cost," says NeoMagic's Zaidi.

Next-gen GPS devices

Does it run Windows Mobile?

The big picture

Contact: www.neomagic.com