Favorite VM for development?

So here I am, trying to (again) help (another) customer whose previous cons ultant didn't leave complete sources and tools, and who cannot reproduce th eir production firmware. Aarrgggg...

In past, we've required customers to install the tools and do the builds fr om our delivered sources, but that is time-consuming for them and doesn't y ield repeatable results for certain toolchain installations that wander off and fetch mysterious components from the net (and make it impossible to ba ck up an installation snapshot).

So, to avoid further repeats, I'm going to do development in a (windows 7 3

2-bit) VM with EVERYTHING required. Deliveries will be copies of the VM and I will ensure the customer actually rebuilds their production releases in the delivered VM. No more confusion about which version of tools, libraries , sources, etc....

I've avoided doing this in past because of VM problems:

- USB bugginess and/or performance interfering with JTAG dongles, and

- VM performance

- annoying 100GB delivery files

What VMs have you guys used successfully? The tools for this customer's projects include:

- Microchip MPLABX with ICD-3 debug adapter

- TI CCS for MSP430 (GCC version) using TI MSP-FET430UIF

- (phasing out) Rowley Crossworks for MSP430 using TI adapter

Host for the VM will be Win7 32-bit, and later Linux and Win7 64-bit.

I've used QEMU in past for developing inside a Linux VM, and VMware player for a WinXP machine. I tried VirtualBox but was stymied by USB issues. But all of these were at least a couple years ago...

Any advice, suggestions, warnings much appreciated! Thanks, Best Regards, Dave

Reply to
Dave Nadler
Loading thread data ...

I've used VMware for something like this and it worked pretty well though USB wasn't involved. Another idea might be to deliver on real hardware (a cheap laptop, say) instead of a VM.

Reply to
Paul Rubin

I've had excellent results with all of VirtualBox, VmLite and VMWare.

I'd give VirtualBox a slight edge because the documentation is quite good.

USB support seems okay - with the exception that I always have to do a forced remount ( -o force ) of spinning USB drives on an old SuSE 11.0 VM. I've no way of knowing right now if that's an artifact of SuSE, VirtualBox or the Win8 host. It's hardly a show-stopper and it may actually be fixable.

The VM files themselves are considerably less than 100GB in my case.

This being said, I'm not trying to connect a JTAG adapter. That's probably the main risk here.

--
Les Cargill
Reply to
Les Cargill

I've done USB JTAG things on VMWare Fusion: Mac OS host, Win 7 guest. It works fine, with the exception of USB3 devices which VMWare doesn't have a Windows guest driver for. Putting a USB2 hub between the device and the machine should fix it.

It's quite neat in that, whenever you plug in a USB gadget, it asks whether you want to connect it to Mac or choose a VM to connect to (you can of course remember this setting).

Theo

Reply to
Theo Markettos

For processors like the MSP430 and on up through ARM Cortex families I would suggest that giving the Forth VM a consideration might be worth your while. I am not sure the Microchip MPLABX (PIC?) is covered by any of the vendors of Forth but you never know until you start asking.. The two main vendors for Forth are:-

MPE Ltd. Forth Inc.

For the MSP430 there is 4E4th and MSP430Lite from MPE. There is also an ARM- Forth Lite for the ARM Core M0 and M1. You might have to speak to Steve Pelc about getting those. On the other hand, the professional versions will provide things like the USB and Networking stacks along with a host of other goodies.

In case you haven't come across Forth before, it is an extensible development environment that enables the construction of Application Specific Languages. It is based on a twin-stack VM that enables you to eliminate the need for a wide range of global variables and is an amplifier of the programmers capabilities. There is a simple philosophy behind Forth that enables you to construct solutions according to your preferred style.

My style is Component Oriented and I can fully certify my code to a level equivalent to hardware CofC. Many other Forthists utilise a mix of Functional and Object Oriented programming.

I will warn you, however, the Forth mindset, is quite a bit different from all of the Algol family of languages (C etc). So the Forth way might not suit you. There are a number of books available (some for free in digital formats) that will help you understand Forth and its Philosophy. Absorbing the Forth philosophy can be tremendously powerful or, if you don't get it well enough, be a complete disaster. It works very well for me though.

--
******************************************************************** 
Paul E. Bennett IEng MIET..... 
 Click to see the full signature
Reply to
Paul E Bennett

Dave wasn't asking about a VM to run on the target processor, but rather, a way to bundle up a completely configured dev environment involving a lot of third party Windows tools into a blob that the customer could host on their own computer, without having to individually install and configure all the different tools.

Forth is an interesting idea in the sense that it replaces all those separate tools with a single, relatively small "do everything" app, avoiding many of the install/configure hassles. But, by the same mechanism, it completely takes over the whole development process including imposing use of a programming language that has some nice properties but is rather far outside the current mainstream. It might be worth looking at but it's not for everyone.

Reply to
Paul Rubin

Hadn't considered he would be looking at the other level that is probably more suited to Eclipse and the like. Then, being the Forthist I am, I would not think to such a gargantuan level as a baseline. Hope he remembers to include the CM tools.

--
******************************************************************** 
Paul E. Bennett IEng MIET..... 
 Click to see the full signature
Reply to
Paul E Bennett

Yes, that is the idea. He wants to package up a complete Windows installation that's loaded up with the kitchen sink and everything else, in a way that the customer can run the whole shebang on their own hardware without a lot of hassle. The conceptually simplest approach is to just ship a full hard drive snapshot of the development system, but that can fail if the origin and destination computers aren't identical. That's why the virtualization is needed. In the Linux world, the approach has become very popular using lightweight containers like LXC or Docker rather than full-blown virtualization. I wonder if something like that might work for Windows.

Reply to
Paul Rubin

You're still not thinking big enough. He wants to take an entire virtual machine image, from the operating system on up, and hand that disk image file to the customer. Eclipse (or whatever), compilers, etc., would be installed *in* that virtual machine image. IOW, it's an image of an entire virtual PC with the entire development environment installed. The customer then boots that virtual machine under VMWare, Hyper-V or whatever hypervisor gets picked for this task, and they've got the exact, and complete, development environment.

Reply to
Robert Wessel

Amazing how often this happens! Even folks missing

*sources*! (oops!)

This *can* work. But, there are caveats.

Of course, one assumes customer has valid licenses, none are node-locked to *your* particular machine, etc.

I think the more practical issue is how completely you can (re)produce the development environment FROM THE CUSTOMER'S POINT OF VIEW. I.e., does *your* environment accurately reflect EVERYTHING the customer uses AT HIS END?

For example, my VC/CM system isn't replicated at any client. So, I have hooks to fetch sources, tools, etc. from *my* system built into my makefiles, etc. These simply don't make sense for the client.

OTOH, he may have *different* tools and hooks to perform similar operations.

As well as other tools with which they haven't bothered to involve you. E.g., I have a couple of projects with tie-ins to mechanical designs that I must be aware of; others where I am insulated from all that (along with the client's tools supporting those aspects).

Install a removable drive carrier, external "dock", etc. and just buy a bunch of small (e.g., 80-160GB) drives that you use AS IF "removable media". Any drive technology (other than PATA) should work well (I use FC, SATA, SCA and SAS drives).

If you are shipping media, you might opt for solid state drives and encourage your clients to "recycle" them (back to you! or, bill them for "media charges" in each deliverable)

I ended up moving my VM's (VMware) off my development systems and onto a host that *only* runs VM's. I found it was leading to flakey behaviour on the tools NOT running under the VM's on those systems (easier to move them all than fight the flakiness).

Reply to
Don Y

Or, more subtly, not the /real/ sources.

I've seen "source code" that was incomprehensible spaghetti C. Clearly the contractor had developed the FSMs in a development tool that emitted C which was then compiled by a standard toolchain.

Perfectly good way of developing a system, but not, IMNSHO, an acceptable way of delivering it. Good for job security, though.

Reply to
Tom Gardner

(sigh) No, I've had projects where I was given product from the factory with *ROMs* and told to make a particular set of changes.

"Um, where's the source code for these ROMs? (and the other programmable logic in the design that would enable me to figure out what thhardware actually *does*)"

"We lost it"

"Oh. Then, I can only give you a T&M contract..."

Deliverables should include everything that the client needs to "pick up where you left off".

If you "needed" a particular tool (hardware or software) to create your "key deliverable" (e.g., a binary for a piece of code), then that tool should be included (i.e., if it is something you wrote) or otherwise obtainable independently by the client (e.g., let him buy his own license for COTS tools).

[If I've created a tool, I am not obligated to provide the *source* for that tool -- unless it is specifically covered in the contract -- just a binary that the client can run to reproduce the items I've delivered]

However, I am free to throw together "informal" tools that are not included in the deliverables if those tools existed only to make me more efficient with my time.

For example, I build "screens" with a little kludge I wrote many years ago (WYSIWYG interface). But, the output of this tool is something that another developer could create and manipulate directly -- just not as *effectively* as I did.

OTOH, things like the rule compiler for the TTS system is effectively "essential" for building accurate rules (you could "hand compile" them but you would spend a LOT more time debugging your mistakes. So, the compiler is included in the deliverables.

But, the *source* for the compiler need not be. As long as I am willing to stand behind the rule compiler's accuracy and reliability (i.e., ensure that it does what *I* have advertised it to do; nothing more!)

Reply to
Don Y

Two points:

I have found VirtualBox to be excellent. It does everything you need here - USB works fine, and I've run several JTAG debuggers from within VBox machines. It is free (most of it is open-source "free", the USB bit is merely cost-free), which means you don't have to worry about licenses or other complications when installing it everywhere. And the dynamic VDI disk images are reasonably compact - though image files will still be big. It works fine on a Windows host, though Linux makes a better host for any kind of VM.

Secondly, try /very/ hard to make your virtual machine Linux rather than Windows. Windows licensing gets complicated when you start copying the VM or transferring it to a third party. If your customer worries about the details of such things (many don't), it will definitely be a problem. And if you try to use the same VM on different machines at the same time, with each one faffing around with updates and "calling home" to MS, you'll find your VM declaring you a pirate and de-activating itself. And if this is for long-term projects, you might find customers complaining that they can't use a Win7 machine because it is "insecure"

- their IT folk will only let them run Win9 or Win10 (or whatever it is called in the future). With a Linux VM, you pretty much avoid all this.

In a similar vain, try to make sure that none of the tools inside the VM require licenses, node locks, or anything of that sort. CCS with gcc should be fine, but I don't know anything about MPLABX.

Reply to
David Brown

Another vote for VMware Fusion. It works fine with several different guests. I'm primarily using different flavors of Linux, but BSD and Solaris work also (And Windowses, if you have to). The guest images work fine when moved to other computers with VMware or VMware Player.

Another nice property is that you can use a CD/DVD image on the host computer and show it as a CDS/DVD drive with disk to the guest, so there's no need to burn installation frisbees.

--

Tauno Voipio
Reply to
Tauno Voipio

That's just what I'm using. A complete Linux development environment with Eclipse IDE. No licensing hassles and it works well with VMware. The shadow side is that the images are pretty large (about 20 GiB).

--

Tauno Voipio
Reply to
Tauno Voipio

I have found VirtualBox on Ubuntu 12.04 to be almost seamless with what USB stuff I've tried. That's been limited to ST-Link, Olimex Tiny-USB and J-Link, but each one just needed me to select the USB device from the "plug things in" menu and click it.

The biggest problems I've had with it are that for all of these I normally use the USB device in question for my own work, and sometimes I'll waste a half an hour debugging a mysterious dongle problem that turns out to be that the VM is running and as a consequence has seized the USB device.

I think your idea of delivering the whole VM is a good one. I'm not sure how one can force a customer to use their own builds (I'm very happy with my one customer who insists on doing so, however), but the idea of being able to deliver one closed package and have it work is attractive.

I would add that you probably want to verify that builds work correctly when the VM's virtual ethernet port is disconnected, just to make sure that nothing is being fetched from Cyber-Land, although I suspect you've thought of that already.

--
Tim Wescott 
Wescott Design Services 
 Click to see the full signature
Reply to
Tim Wescott

I use VmWare - I haven't played much with QEMU, but like you I have had various device issues with VirtualBox.

So far VmWare has handled everything I've thrown at it, though if you have to support weird bus cards - or worse conflicting bus cards in different VMs - you may need to use a bare-metal version instead of an OS hosted version.

[Coincidentally, VmWare has released the vSphere bare-metal version as freeware: see
formatting link
It's officially unsupported, but help is available through the user community.] You can minimize VM images by not preallocating the virtual disk and by running the defrag/compact tool before archiving the VM image. Further, if multiple toolchains are based on the same OS, you can create a base VM for the OS and separate linked VMs for the unique toolchains [which saves delivering the OS multiple times].

WRT performance: it's a given that you should make sure your virtual disk files are contiguous (the notion that Linux doesn't benefit from defragging is bullshit). Generally a Linux host will outperform a Windows host given the same hardware. If you're on a 64-bit CPU, VmWare will let you run 64-bit clients even on a 32-bit host OS.

Hope this helps. George

Reply to
George Neuner

I second Tim's experience. I've been using VirtualBox under Fedora (10? up to the current 20) for several years and have no complaints (that I can remember...).

--
Randy Yates 
Digital Signal Labs 
 Click to see the full signature
Reply to
Randy Yates

I was on the delivery side of linux VM images for quite a while. What I did to keep the file-size in check was to run:

dd if=/dev/zero of=./nullfile && rm ./nullfile && shudown -h now

before delivery. This fills all available free space on your disk-image with zeros. Afterwards the disk/vm image compresses like crazy.

(good thing to do anyways - you may not want to leak any deleted data like source-code or emails that found their way into the VM to your customer).

Nils

Reply to
Nils

Yes, you can compress the disk image for delivery, but when you expand a preallocated image for use it will still include those zeroed blocks and be whatever size was configured for the VM.

If you don't preallocate, space for the unused blocks doesn't exist and the disk image physically may be much smaller than its logically configured size.

This is why I mentioned separating the OS and making the deliverable a linked image - you can quickly create a new linked VM and copy only the data that actually is to be delivered onto its disk.

A good practice if you intend to clone your actual development disk ... but I wasn't advocating that.

George

Reply to
George Neuner

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.