I have heard many times that the SOC used in the rPi has a closed video engine with a binary only driver. I have also heard that we really don't need to care about that. But then I saw this today in an Amazon listing for an rPi...
"GPU provides Open GL ES 2.0, hardware-accelerated OpenVG, and 1080p30 H.264 (requires license purchase from raspberrypi.com)"
None of this means a lot to me except for the part about having to purchase a license. What? And you have to purchase it from raspberrypi.com??? Are they only talking about the H.264? I want to say that is used for video conferencing, but I'm shooting in the dark here. On the rPi web site they list "codecs" for MPG-2 and VC-1 for a couple of bucks each.
What is going on? What happened to open source software? Are these codecs sold because there is a license fee with the format? I understand that JPG is also a licensed format, but it seems to be widely supported by any number of programs, both open source and commercial.
It's incorrect, the H.264 license is included in the price of the pi already. It's only the MPEG2 and VC1 codecs that require buying a separate licence.
Software patents are what happened... You can install software libraries to decode these formats and get away with it but if you want to use the hardware decoder you have to go by the book (or find a way to crack the hardware, but for the time and electricity you'd spend you may as well just cough up the license fee)
The Pi is an ARM CPU welded onto a GPU. The ARM side is open source, the GPU code isn't. GPU vendors are notoriously reluctant to open their code, because it would then be easy to work out the special magic of their GPUs.
The ARM makes requests of the GPU. This interface is documented. It can be 'please set the screen to 1080p' or 'my framebuffer lives here' or, via higher-level protocols like OpenGL/VG/Max, 'please draw a triangle' or 'plot this JPEG here'.
Old fashioned GPUs used to be fully hardware accelerators - with internal state machines for drawing lines, 3D rendering etc. In such GPUs you'd get whatever features shipped in the production silicon, and could never add to them. The VideoCore is a software GPU, in that you can upgrade the 'microcode' to make it do extra things. In fact, the microcode gets updated regularly for free.
Some of these extra features cost money, for two reasons. One is that they're over and above the base spec of the GPU - ie the silicon vendor made a retrospective upgrade to the features of their chip, and wants to get paid. The bigger issue is they have to pay for patent licenses, and these cost money.
So far there's only two chargeable features, which concern the VideoCore GPU accelerating playing video. You just hand over the video to the GPU and say 'please play this video stream', and it happens, with zero CPU overhead. But the MPEG-2 and VC-1 video codecs are patented, and you have to pay the licence fees to do that. The H264 codec doesn't have a patent fee (or Broadcom have swallowed that in the chip cost) so it's free for all RPi users.
In many ways, this is actually a nicer interface than many desktop GPUs, where the GPU/CPU interface isn't published. You can't easily say to an NVIDIA or ATI card 'please plot this JPEG here' because the CPU has to do lots of setup work before getting to this point, and neither the setup nor the GPU hardware interface is documented. Hence lots of reverse engineering of drivers before you can get anywhere at all.
On the flip side, it's essentially impossible for a normal developer to extend the functionality of the GPU. I can't add a function that says 'please plot this Photoshop file here' because that would need GPU-side code writing to understand Photoshop files, and I don't know the architecture or have access to the GPU code compiler. I can, however, have the CPU do the work and reduce it to ops that the GPU already supports, it's just less efficient.
The reality is that there's a crossover between hardware and software. Hardware (ie the chip layouts) aren't open source. Microcode isn't open source. Bootloaders sometimes are and sometimes aren't. Desktop OSes are often open source.
In the PC world, the CPU design isn't open, the microcode isn't open, the BIOS usually isn't, and the OS might be. RMS bangs on about open BIOSes, but even he doesn't ask for an open CPU design and open microcode.
The way things are going is more things are moving from 'hardware' (transistor interconnections) to 'microcode' (software deeply intertwined with the microarchitecture). It's a point of debate where the boundary between open and closed should be.
Theo
Declaration of interest: I work on a project where the CPU and all levels above that (will shortly be) open source:
I see that most of the platforms I am familiar with come with a paid-for OS which obviously included a license fee for any patented formats like this. But what about other open platforms? Do users of other hardware platforms have to pay license fees for these same formats?
Nope. They just download and compile if necessary the open source libraries that 'can't be supplied free by places that don't want to get risk getting sued'
e.g. from
formatting link
Are libdvdcss and libaacs legal? libdvdcss ========== libdvdcss is a library that can find and guess keys from a DVD in order to decrypt it. This method is authorized by a French law decision CE 10e et 9e soussect., 16 juillet 2008, n° 301843 on interoperability.
NB: In the USA, you should check out the US Copyright Office decision that allows circumvention in some cases. VideoLAN is NOT a US-based organization and is therefore outside US jurisdiction.
libaacs ======= libaacs is a research project and has an interoperability purpose (see above point).
Moreover, libaacs DOES NOT provide any decryption key. It is based on the official public AACS specification only.
Patents and codec licenses ==========================
Neither French law nor European conventions recognize software as patentable (see French section below). Therefore, software patents licenses do not apply on VideoLAN software.
NB:libaacs is not shipped in VLC.
--
Ineptocracy
(in-ep-toc?-ra-cy) ? a system of government where the least capable to
lead are elected by the least capable of producing, and where the
members of society least likely to sustain themselves or succeed, are
rewarded with goods and services paid for by the confiscated wealth of a
diminishing number of producers.
So this is possible on the rPi? I'm writing some code on bare-metal to display stuff, writing to the framebuffer works so far. Just the initialization is a little buggy sometimes. But letting the GPU paint a plain JPEG would be great! Do you have some information or links on that or other GPU features?
You need to use the OpenVG or possibly OpenMAX API to do it:
formatting link
formatting link
If you want to do it bare-metal you'll have to either marshall the data in the right format that goes through the mailbox, or drag in the Open* library bindings which will do it for you. I'd suggest the latter to make your life a lot easier. The library binding source is here:
formatting link
I don't know how much of an environment it needs that you'd have to provide.
You keep changing the topic of discussion. What about the MPEG2 and VC1 codecs? Most installations of Linux are in the form of bought software like on phones, tablets and commercial distributions. Do these distributions include any of these codecs? If they do, I assume they paid the license fee just as a cell phone includes the license for all the relevant technologies that have license fees.
I think you have a unique idea of the economics of selling hardware. When my laptop maker upgrades the BIOS they don't charge me for it. They make it a free update. When the GPU maker in my desktop adds new CODECs to their library, they don't charge for it, they make it freely downloadable to everyone using that card. The only limitation is that at some point they stop supporting the older hardware.
As soon as a hardware vendor starts charging for the software they provide that enables the features of the hardware they are selling, that makes the hardware less valuable and lowers the selling price relative to the competing hardware.
Yes, this is the part I am asking about. How do other distributions of FOSS OS provide this capability? Do they pay the license fee and pass that on embedded in the price of the product? What about freely downloadable versions where there is no product price? Do these packages not include the drivers for licensed formats?
I'm not sure what your point is. Any hardware has to have a published interface to be at all useful. I don't care about the internals. Even if that interface is post some software as long as it works and supports what I want.
I'm simply asking why is the rPi apparently unique in that they sell the hardware and then ask more money for the drivers for some of the formats the hardware supports? I haven't noticed this in other platforms that I have seen. Maybe it is just invisible to me (because I haven't looked very hard) or maybe I just haven't looked at the right systems. For example, does the Beagle Bone Black have a similar issue?
What form will this CPU be, a chip, an HDL for an FPGA, a spec?
They don't... To use the GPU you have to use a firmware provided by the raspberry pi foundation or presumably give Broadcom lots of money and sign an NDA.
Because they are using a system-on-a-chip that can hardware decode licensed formats. Since most people won't want to use these decoders the licenses aren't included in the price of the hardware but can be paid if you actually need them. They are not selling drivers but a license and unlocking the codecs in the GPU.
Sorry, my question was not clear. I didn't mean on just this hardware. I mean in general. If I have a tablet running Android I suppose they have paid the license fee. If I download a Linux distribution to a PC how do these CODECs get decoded, or I should say, paid for?
Ah, so these CODECs are not common? MPEG-2 is used when exactly? If there is MPEG-3 and MPEG-4, why do people use MPEG-2 if they have to pay additional?
Some commercial distributors have removed such code from their base package, so the relevant programs or libraries can only be downloaded from an auxiliary repository that is not under the formal control of the distributor.
Others have decided to pay the license money and include the software in their base package.
For example, the mpg123 MP3-decoder was removed from many Linux distributions and has to be manually downloaded or an additional software repository has to be added that has no problem with distributing unlicensed code.
We have a large oscilloscope on test at the moment. It runs Windows 7 on a touchscreen and has all kinds of funky features. The base feature set is included in the price. If you want it to decode (say) PCIe in real time, you pay $1000 (or whatever) for a licence key to unlock that feature. If you want it to do GSM validation tests, that'll be another $1000. It's 'just' an ADC in a box, and what you do with it is software. Sometimes software costs money.
You were sold a device that does X. If they later enable feature Y that wasn't on the standard spec sheet, then there's sometimes a charge for that. Just like buying an iPhone doesn't get you all the software in the App Store for free, even if it's written by Apple.
If it's open-source software, you get the software 'free', even if you pay for the media. There's no licence fee the vendor can squeeze out of a $0.00 price. Patent holders tend not to chase people who have no money - they may even explicitly give out free licences for open-source code. If you recall the Unisys GIF saga, they tried to charge for websites that hosted GIFs even non-profits - this made things awkward for open source users, with the result that people moved to PNG.
If money is involved, for example you buy a chip or a tablet that implements the patent, you have to pay the licence fee. I'm unclear exactly on the solidity of software patents (which varies a lot between jurisdictions), but selling a piece of hardware that is claimed to infringe patents is possibly more vulnerable to patent suits than just selling software. At the very least the handset makers will already be paying license fees for the GSM patents.
The other possibility is the vendor isn't paying the patent fees, and hasn't been sued yet. I suspect this is the case with small-volume or 'knockoff' vendors.
You can have a fully OSS system, but you also get no GPU. Nobody open-sources the code that runs on their GPU. You can have a dumb display controller (framebuffer, maybe with a bit of state-machine-based acceleration) but then there's no GPU code to open source.
Because nobody open-sources their code, all the other GPU vendors are afraid of open-sourcing their code. It's particularly acute because GPU architectures are often radically different from each other, so there's more for competitors to learn.
I don't know for sure, but I suspect other vendors roll up the licence fees into the price of the chip. If you're selling $20 chips then you can do that, if you're selling $2 chips maybe you can't. (I have no particular information on the price of the Pi's BCM2835, but there's only so much headroom in a $25 ticket-price).
Some other chips also push the decode into CPU code (ie don't accelerate it), in which case they get around it by open-sourcing the CPU code. But customers who sell it in products still need to pay.
HDL for an FPGA primarily, though anyone is welcome to make a chip if they like. It's primarily intended for people who want to modify their CPU rather than end-users who just want it to run fast.
That is of interest to me, but I expect your design will be similar to the many other open source CPU designs. What will set yours apart? Why design yet another open source soft CPU for FPGAs?
To be honest, most of my interest in soft CPUs is oriented to small, resource efficient and power efficient designs. I often work in FPGAs with 3k gates or less.
"If you?ve been following this website since we launched it last summer, you?ll probably be aware that we had to make some hard decisions about exactly what we could include on the Raspberry Pi if we were to meet our extremely low target price. One of the things that we had to regretfully dismiss as an option was an MPEG-2 decode licence for every unit. Providing that licence would have raised the price of every Raspberry Pi by roughly 10%, and we simply weren?t able to justify that when we held it up against the educational goals of the Foundation. Our initial expectation was that most of you would buy the Raspberry Pi for educational purposes, and that you wouldn?t mind that MPEG-2 wasn?t available. Our bad."
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.