Keyboard / Mouse Input Device Design??

Peter,

You might want to look at

Don Cleveland

On any Unix system where the ImageMagick package is installed, one can use "import" to get an image from any part of the X display. See eg

-jiw

On a sunny day (Tue, 31 Oct 2006 10:40:55 -0700) it happened James Waldby wrote in :

This is correct, and you can get window info to simply with xwininfo. However the OP wants it for _all_ Linux GUI apps. So there is svgalib, others, and I can also in an X program draw a menu without using or informing the X server (of a subwindow), using fonts unknown to the X server etc. I have done this. And you can have the situation where a person was in X, and then did ctrl alt F1 to show a normal console, so simply testing if X is present does no guarantee that is what is on the screen, etc. Other languages, character sets, libraries, video fullscreen, we have it all.

What the OP told me he has, needs to read and evaluate what is on the screen. 'Impossible' may not exists, but it will be _very_ difficult to cover all cases in Linux, it is not a monolitic integrated GUI like MS Widows.

mouse

Is this a homework question? I haven't seen these devices for years! The old macro keyboards that inserted between keyboard and computer were probably under $100. Modern solution is use something like Macro Express which is much more convenient to use.

"Peter Olcott" schreef in bericht news:E5H1h.16764$ snipped-for-privacy@newsfe23.lga...

Hard to say because of the prerequisite constrains are unknown to me. I'd say 80 to 100 hours. Half of that time would be used for the real work. So making a ready to use proto (including enclosure, connectors, power supply and so on). The other half for gathering information, like details of the protocols, availabily of components and obtaining them and for making decisions. After all, when you want to go to mass production, component price and availbility are important issues.

For instance, when I need to be fast, I'd go for an ATmega162. It's like killing a mosquito using a canon but it's the smallest one with two UARTS I know for the moment and it's not cheap. And there you go. Looking for a cheaper solution requires time so you're slowed down. I should not be astonished if even this very posting starts a discussion about more simple, cheaper solutions. Which is, after all, the main value of the group.

petrus biybyter

mouse

really, REALLY easy... basic one way serial code - provide the correct key codes (these are the mappings for each key and not anything like ascii values - also may be upto 3 bytes each) and bob's yer kipper - google the PS2 interface adn that should give you everything you need.

that

any

you have a technology this advanced and yet you are having trouble with the PS2 interface!?

If this is something for future PC's, bare in mind PS2 and serial ports will cease to be included on new systems. This is because Microsoft has told PC manufacturers to remove all the leagacy devices on the LPC bus (internal ISA bus), which are mainly the built-in PS2 mouse/keyboard, parallel printer port, joystick port, serial ports, floppy disk.

So you can't simply get the pixels comprising the display screen on Linux?

I am already working 16 hours a day, what I need is someone that already knows this off the top of their head. Feasible / Infeasible ?

Good, what about sending keystrokes and mouse actions to Linux?

One of the applications that I am creating with my

Not sure if you are asking this question rhetorically, but yes you can, using XWD or, programmatically, using XGetImage on the root window.

The question is whether you intend to track all screen state, or take snap shots. If you intend to take only snapshots, then on both Windows and Unix, you're looking at 20 lines of code or less.

I noticed that your application allows the specification of font family, font face, etc., when scanning for text. This has implications on the scope of what you are doing. Naturally, you're not proposing to be able to recognize any piece of "graphical text", are you?

-Le Chaud Lapin-

Okay great so the project is feasible.

I see. Let me clarify for the other readers - you are _not_ proposing to recognize any arbitrary "text". It is possible to render text in Windows using a font not known by the system using bit-blt on internally rendered bitmap. Certainly you would not be able to recognize a custom "dingbat font".

That said, I can vouch for the value of your application (if it actually works. :])

Most macro recorders today are somewhat disfunctional. (Macro recorders monitor, in software, the input stream to Windows, and attempts to replay the stream back after the computer has been rebooted to get the application into the state it assumed when a human entered the input).

The reason that they are broken has primarily to do with timing - when it comes times to playback the keystrokes, they have no idea how rapidly the application is responding to playback. So the macro recorder might replay input too fast, playing keystrokes that were meant for a window that has not come alive yet. The keystrokes are then lost. The operator of the macro recorder will try to combat this by guessing how long it takes a Window to be "born" and "come to life", waiting that amount of time before playing input to that Window. But this is error prone. Raise or lower the CPU peformance, and it breaks.

So Peter's application apparently takes a snapshot of the screen, finds all the windows, finds the title bars in the windows, edit boxes, etc. and presents that data when it is requested by a function that wants it. In this case, the operator of the macro recorder will no longer have to guess how long it takes windows to pop up. He will simply say, "Wait until there is a window with "Google Talk" in the title bark".

If this is what you are doing, and you are not doing generalized OCR, which you said on your site you were not, then I am a bit puzzled, as it would be possible to do the same by intervention into the GUI subsystem of Windows. And unlike the frame grab method, where you watch the pixels and therefore cannot "keep up" with state transitions on the screen, you would know pretty much the "exact" state of the screen at all times.

-Le Chaud Lapin-

Not only is it feasible, but the more I think about it, the more I realize that you should do this in software. The reason is that, if you are already writing software that must exist on the host system, then you're already writing software on the host system, so you have control over what that system does.

There is no commercially available system where you cannot get as close to the hardware ports as possible in software. For example, on Windows, the keyboard hardware is controled by a keyboard driver (mone keyboard). Before a GUI window gets a pressed key, it must pass from this driver into another device driver called WIN32K.SYS. WIN32K.SYS gets the keys one by one in a RAW INPUT THREAD. So you have options. You can inject keystrokes using the Raw Input API, write a device driver (not trivial) to intercede KBHID.SYS and WIN32K.SYS, or write a driver that gets right up against the keyboard hardware and and feeds KBHID.SYS.

On Linux and other Unices, writing device driver borders on trivial compared to Windows, so you could do the same thing there.

I would seriously reconsider doing this in hardware, since the amount of effort to get 99% of your market is significantly less in software.

The ratio of material cost for hardware method vs software method is infinite.

-Le Chaud Lapin-

I can't tell what you are asking, if you are asking can my technology recognize text from screen shots, the answer is yes. Here are some more details:

Ok, so even though you're using a DFA in your algorithm, the overall model is still stochastic. I see in many places 100% recognition, which, naturally, makes anyone skeptical. To get 100% recognition of arbitrary text, you have to know a priori the Bezier sets of not only all font families currently known, but those that have yet to be made, which seems absurd.

I think you should be more clear about the effectiveness of your tools, as in how it works. Instead of saying, "it recognizes, say a bit more." Since you already have a patent, it does not hurt to be more complete in your description.

-Le Chaud Lapin-

Now that I have read your other posts I can more easily understand your question. My system can recognize any machine generated text. It must be provided with exactly what to look for, this is typically done by specifying one or more FontInstances: (a) Font Typeface Name (b) Point Size or PixelHeight (c) Style including (Bold, Italic, Underline, and StrikeOut) (d) Foreground Color and BackGround Color