Advertisement

Blog

Video Recorder on a Chip

A video recorder on a chip (comparable to the audio recorders on a chip but more complex, of course) would be a useful device. Even more useful would be a very low-power video recorder on a chip. You could build it into the frame of your glasses or the visor of your baseball cap. If the power were sufficiently low, operation from a single cell might be practical.

This seems like the sort of technology that appears in Star Trek movies, but such a device will likely be available soon as a system on a chip (SoC). In a recent paper presented by the IEEE International Solid-State Circuits Conference (registration required), a group of engineers present such an IC in prototype form. The IC features a very low-power microprocessor unit, very low-power ROM and RAM, provisions for an external memory interface (everyone always wants to add memory), provisions for external camera and LCD interface, and an H.264 video encoder. H.264 is a very common standard for video compression.

The prototype device was fabricated on a 65nm process. Recordings can be made at 32 frames per second in HD720 high definition. At reduced power levels (battery voltage at 0.48V), recordings can still be made at 30fps. At this point, energy consumption is estimated at 0.57nJ/pixel.

The design engineers tell us in their paper that they used various techniques — analog, digital, and software related — to get the desired performance and power consumption.

Energy efficiency has been improved through algorithm, architecture, and circuit techniques. At the algorithm level, a complexity and bandwidth-effective group of macroblock (GOMB)-based motion estimation (ME) with adaptive search range (ASR) is proposed. The bandwidth is reduced by ~71% at a negligible cost of 0.1dB quality loss as compared with the full-search ME in the H.264 reference software.
At the architecture level, a low-latency in-loop filter output buffer (ILFOB) and a block-based current/reference frame buffer (FB) organization are designed to reduce by 69% the number of cycles for external DRAM access. Combining these algorithmic and architectural techniques can reduce bandwidth by ~81% in total. Furthermore, ultra-low voltage circuit-level techniques… provide a ~70% energy reduction compared with a state-of-the-art H.264 encoder design.

To see how the designers have squeezed all this functionality into such a small space (approximately 4mm square), let's look at the block diagram:

Here is more detail on the H.264 video encode — both the architecture and the associated algorithm (click here for a larger version).

The authors provide more details on what their FB implementation does and why it's an improvement over other versions.

A 512Kb FB is included in the video-recording SoC for the H.264 encoder to store prediction data and macro-blocks from the reference frame. Compared with the state-of-the-art ULV SRAM design of 256Kb, our 512Kb FB halves the bitline (BL) loading (from 512b/BL to 256b/BL) to improve both the performance and the power dissipation.

Power draw at 1.0V VDD supply is 2.68mW — not especially impressive. Much of the power draw is due to leakage currents. This occurs with process technology this small. When operating at a VDD of 0.48V (the minimum operating voltage), power draw drops to an impressive 312μW

Here's a look at the die.

Have you worked on any video devices for HD applications? Anything at these power levels?

Related posts:

30 comments on “Video Recorder on a Chip

  1. eafpres
    August 7, 2013

    @Brad–this looks pretty nice.  By coincidence I recently saw a datasheet on a Fujitsu transcoder which they describe as enabling “placeshifting”–essentially smart compression to allow streaming to mobile devices etc. for high-quality video.  They don't have the rest of the recorder, but I thought their transcoder looked interesting:

    Fujitsu MB86M01/02/03

  2. Brad_Albing
    August 7, 2013

    Yep – I thought it looked pretty slick. I'll be interested to see how long it takes before we actually see a commercial version f the IC.

  3. Scott Elder
    August 7, 2013

    It seems that the power requirements would be dominated by the camera imager.  I've read Google Glass works for only 15 minutes or so.

    Perhaps it would be more efficient to simply transmit the live video via RF to your SmartPhone in the pocket with the 5WHr battery and record there.

     

  4. Brad_Albing
    August 8, 2013

    @Scott – that could work. Of course, the RF link would take some power, but maybe not too much. It would be very short range and could run in short bursts. Worth a closer look.

  5. fasmicro
    August 8, 2013

    >> You could build it into the frame of your glasses or the visor of your baseball cap. If the power were sufficiently low, operation from a single cell might be practical.

    I have been looking for Google Class tear-apart report to no avail. It is very possible that Google Glass has a video integrated chip as part of the main processor that powers it. I know it uses Android OS and possibly no traditional chipset as we have in phones and tablets. Sebastine Thrum has this expertise in video in Stanford and they might have done some cool video chip design in Glass. Any tear apart link will help.

  6. fasmicro
    August 8, 2013

    @Brad, there are small instruments doctors use these days. They put them at the tip of the endoscopes via larynscope during deep surgery. In Da Vince medical robot, they have these systems that analyse what those niddles see on display overheads. I think we have these video recorder chips, it is just that it has not gone to the consumer market. In other words, it is still expensive for the consumer market.

  7. Brad_Albing
    August 8, 2013

    @fasmicro – It surely would be expensive for some such devices. If the devices see more widespread usage and manufactured quantities go up, the cost should drop to what you and I would consider acceptable.

  8. Davidled
    August 8, 2013

    I guess that if Google glass has a video integrated chip, the size might be bigger than the current size since video data might be stored, even though data could be compressed with high pixel resolution.  Data compression is other area in the electronic industry.  

  9. PZman
    August 9, 2013

    Till then, you can use mu Sony DSR-PDX10, built in 2005 these units have both DV tape and 16 MB Memory Stick. I use the Memory stick to record while I change tape. Sorry I live in another century but these cameras are use for “B” roll. Over the shoulder shoot, shot of the interviewer, shots used for a minute or two.

    I have the Memory Stick set at 320×240 for 4×3 images, it can do 16×9 widescreen but not HD. The 16 MB card records for 2 min & 40 secs. in MPEG. There were cards that went up 128 MB, roughly 21 minutes. Not bad for nearly 8 to 10 year old technology.

  10. goafrit2
    August 9, 2013

    >> widespread usage and manufactured quantities go up, the cost should drop to what you and I would consider acceptable.

    Absolutely – there is that feeling these days that if a technology has not diffused into the consumer market that it does not exist. I think in aerospace and some of the advanced military systems, they have all these chips. The problem is that they do not have widespread adoption.

  11. goafrit2
    August 9, 2013

    >> I guess that if Google glass has a video integrated chip, the size might be bigger than the current size since video data might be stored, 

    Wish someone has torn apart the glass and we can get a good understanding of the chop that powers. I suspect the momeory will be different from the pixelated imager part. But they can have in-memory as some cameras do have.

  12. goafrit2
    August 9, 2013

    @Pzman . When you have a technology that slow >>There were cards that went up 128 MB, roughly 21 minutes>> you may be setting up yourself for disruption. We want seconds these days because time is precious for the young folks. The design issues is not just the capacity but the refresh rate

  13. PZman
    August 9, 2013

    Sorry if I mislead you. Playback is instaneous, the 128 MB chip has 21 minutes of record time. I can download in as fast as my USB allows me. If that isn't fast enough I going to start recording my video before it happens. As for News situation, the TV is prime and the internet is second, which means you probably see the news on TV before the net. So much for high speed net access getting you news faster.

    Curious though did anyone else use these Memory Sticks (that is their name)? I like what Brad is talking about if the chip comes to market. Memory Sticks, I believe, they have come and gone

  14. antedeluvian2
    August 9, 2013

    goafrit2

    Wish someone has torn apart the glass and we can get a good understanding of the chop that powers

    There are at least two teardowns here and here, but I doubt if you will get any understanding about the chips used. The is some discussion on this topic on MCC which is where I got the aforegoing links.

  15. fasmicro
    August 9, 2013

    >> As for News situation, the TV is prime and the internet is second, which means you probably see the news on TV before the net. 

    It all depends and you assume people are fixated on TVs. Largely, in coming years, we will see more breaking news on the web as users will break news and not the TV organizations. 

    I like your comment on Memory Sticks – these are transients systems. They will be done before we know about them. If we can have very affordable video recorder as chip, you will see new revolutions in mobile devices.

  16. fasmicro
    August 9, 2013

    @Brad, yes, it is Video Recorder on a Chip. That means we can have extremely small camcorders. Today you get recorders from systems which involve chips and other things. If we make the recorders to be wholly chips, that is another level of innovation.

  17. fasmicro
    August 9, 2013

    Yes, it is the problem. The thing is very expensive for most hobbyists to waste money just to see a masked processor which Google has worked hard to protect. Not sure there is any value in the teardown when the system is very integrated. Unless you are a layout engineer, you cannot see much. It could be a tiring endeavor for marginal value because they packed all things together to reduce footprint. Hence, you cannot cut a chip with razor to see the inside.

  18. RedDerek
    August 10, 2013

    This talks about the processor to handle and store the video, what about the imaging portion? Did I miss if that was co-located with the processor? And the other aspect is the lensing to focus the image. That could take up volume. It has been a while since I dealt with imaging systems – about 15 years. I am sure there have been some great developments.

  19. Brad_Albing
    August 11, 2013

    @PZMan, – why, that's practically from the last century. But you're right, not bad for when it was designed and manufactured. But it won't fit into your eyeglasses frame.

  20. Brad_Albing
    August 11, 2013

    @RedDerek – you're right about the lenses – the optics for this will present its own set of problems. But that goes well beyond my ken and beyond our charter here.

  21. fasmicro
    August 14, 2013

    >> Curious though did anyone else use these Memory Sticks (that is their name)?

    Memory sticks or USB was invented by Intel. I am not sure they will allow that to go slowly. But you are correct, we can get to the point where there will not be any need to have data out of the network. In that case, there will not be need for physical storage media since we can store in the cloud. With Dropbox, I do not do a lot of storing these days in any physical medium than the cloud.

  22. fasmicro
    August 14, 2013

    >> This talks about the processor to handle and store the video, what about the imaging portion? Did I miss if that was co-located with the processor? 

    It is very save to assume that if you can handle the video storage, you can essentially handle the imaging. Remember that it is the imaging part that will give you the video portion. You can take the images in frames. It is harder to store video than image; so focusing on the harder one may be the optimal strategy.

  23. fasmicro
    August 14, 2013

    Lenses only take space – they are not necessarily hard to incorporate in systems. Know the focal length and the right distance you are done. It is not an ASIC problem as is being discussed in the piece. More of optical and mechanical issue.

  24. Brad_Albing
    August 23, 2013

    @fasmicro – to your point – yes, you'd need to decapsulate the IC (very carefully) and very the chip. Eventually, you could probably figure out portions of it. Then, there would still be the matter of extracting the firmware….

  25. PZman
    August 27, 2013

    I have a new idea for recording images. Small & lightweight, not water about firmware and batteries. It is a film camera using 110 or smaller film size. Do we create for need or just for creations sake.

  26. Brad_Albing
    August 27, 2013

    @PZMan – Film?! Now that's just crazy-talk.

  27. PZman
    August 28, 2013

    What I am getting at is technology should match the application not always driving it. Technology is just technology, how it is used is the key. There is an application that the film camera is the best answer and likewise it's not. Not to pick on a product, the iPhone has great applications, sorry to say its only a high resolution wallet camera. Color balance is passable, almost no one can hold a level image and were is my real zoom. These items were designed into handheld digital camera as part of the design not added onto. I will take my phone (with camera built in) to shoot some of my work projects. Otherwise when I want real quality give me my baby Nikon.  I also once had a Bolex handwind up, 16mm camera. Great for shooting nature, and tv news. Not what you want to take family movies with.

  28. fasmicro
    September 8, 2013

    >> What I am getting at is technology should match the application not always driving it

    You may be missing a point that technology can create a new application space. If you see iPhone as a technology, mobile apps did not match it. Rather, they evolved because of the tech. Change can come from application or technology but even the application is tech-driven. Engineers solve problems because all policies are engineering problems. Take engineers out, hospitals close, soldiers cannot fight, etc.

  29. PZman
    September 9, 2013

    i agree with the thought that technology should match the application. At times technology is driven by application such as space travel & moon launches. These examples have a technology that not necessarily been proven but are on “trial” and must be driven by application. As for iPhones, these are technology. I have seen microscope adapter & application on the iPhone, really neat & has use in the field. I have also seen a SPL sound pressure level meter and was not impressed. The meter used the internal mic and had discussions with audio people that the mic is not a calibrated unit and the the audio circuitry is not the same gain & response from unit to unit. Here is an example of an application that technology could match but doesn't (what do you want its a phone) but its promoted as a professional piece. We have issues of technology meeting application, apps in phones not meeting their application and the marketing of said devices as an end all. I am still a fan of camera as a camera, love my Nikon small format digital; a phone that does phone & txt and can take a moderate picture.my stereo is still a stereo with inputs for everything. In the big picture; I believe I keep more engineers in work than the all purpose “vehicle” people do. I really appreciate your inputs & feedback.

  30. fasmicro
    September 16, 2013

    >> Then, there would still be the matter of extracting the firmware….

    That is why it is more tougher than many think. For example in due core systems, the innovation is not in the packing of the cores but the firmware and software that make them work. If you see the layout, without decoding the firmware, you have gotten nothing useful.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.