"Device-to-smartphone" and "smartphone-to-device" technologies

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Nowadays the poor embedded developer faces new challenges that go  
outside the limited world of C/C++ program on a resource constrained  
hardware.
Now the hardware isn't so resource constrained (Raspberry and many other  
embedded Linux boards or SOM are just examples) on one side. And the  
device isn't isolated anymore, it is usually connected (directly or  
indirectly to Internet).

The Thing/device connected to Internet (IoT) is now a reality on all  
embedded hardware. With ESP chips you can have a WiFi connection for a  
couple of dollars. With many Cortex M3 you can have a wired 10/100Mbps  
Ethernet connection. lwip is the piece of software to have a working  
TCP/IP stack on those hw. Now we are at the application high-level layer.

Yesterday a typical request for an Internet-oriented device was HTTP  
server (I designed such a system one year ago).

Today the request is different. The end-user, at least in most consumer  
applications, doesn't want to spend time for DDNS and port forwarding.  
Moreover he wants to use his smartphone and not a typical desktop PC.

So now we have typical appliances that can be managed by a smartphone.  
You can configure the washing machine through your mobile phone and you  
can be notified when the clothes are ready.

This is a typical request for an embedded Internet-connected device. It  
should communicated with a registered smartphone.
The main question of this post is: what are the technologies that allow  
a device to communicated with a smartphone?

I understood this problem can be divided in two completely different  
challenges. The first is "device to smartphone" direction, with its  
solutions. The second is "smartphone to device" direction, with other  
solutions.

Regarding device-to-smartphone, I think the best solution is "push  
notifications". The push notification technology allows to reach  
whatever mobile phone in every situation, even if our app isn't running  
(I understood Android apps can be started automatically at startup, but  
iOS apps can't be started automatically at startup... so a push  
notification is the only solution). This technology allows to reduce  
traffic so increasing battery duration.

How to create a push notifications from an embedded device? I understood  
we need to register (and pay for) a notification service, for example  
Google Firebase Cloud Messaging (FCM). A notification can be sent with  
an HTTPs request to a server, so we need an HTTP client and the full  
TCP/IP plus TLS stack.
Of course we need a custom app running on the mobile phone: without a  
custom app, the notification can't reach the smartphone. This is a big  
issue, because apps and embedded fw are two completely different  
monsters that need different technical skills.

The smartphone-to-device direction is more critical, at least for me.  
The app running on the smartphone can't reach directly our device, that  
is usually behind a router and configured with a private IP address. The  
key technlogy is the "cloud", a public server that accepts requests from  
the smartphone and connections from devices on the field. The cloud  
server is able to link the smartphone request to a connected device.
Here the high-level view block diagram is clear, but I can't go down to  
the next level of details.

Which protocol do use the smartphone during communcation with the  
server? Maybe HTTP(s).
Which protocol do use the device during communication with the server? I  
think we have many solutions: MQTT, HTTP(s) and so on.
What are the software to use in the server? I don't know.

As a developer, I can think to rent a physical server (the  
infrastructer) from service providers and: install an OS (maybe a Linux  
variant), install a HTTP server (for smartphone requests), write and run  
server-side scripts to process smartphone requests, install a custom  
software that communicated with the embedded devices.
Wait wait, but I don't have the skills to create and maintain a serious,  
reliable and secure server that is reachable in Internet. Does it mean  
we need another guy in our team?

Another possibility is to rent (and pay for) a cloud service that let  
the smartphone to "chat" with the device. MQTT broker? PubNub? Amazon  
Web Services (AWS)? I don't know.

I also read about serverless solutions where you wrote the program and  
"launch it in Internet", without knowing the server where it really runs  
(you usually pay for processing time). In this case I needn't to have  
good skills to create and maintain a real server.

I'd like to know if you received those type of challenges in your daily  
embedded work and how did you try to solve them.

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 10.12.2018 ?. 00:49, pozz wrote:
Quoted text here. Click to load it

CPU power is abundant nowadays, so is connectivity. Just make your
device look like say an RFB (aka VNC) server and use the smartphone/
tablet/PC simply as a terminal.
We have been doing that for over a decade on our netMCA devices,
long before the "IOT" word had been coined I think.

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/


Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 10/12/2018 11:04, Dimiter_Popoff ha scritto:
Quoted text here. Click to load it

When the device *is* a server, you need at least Dynamic DNS and port  
forwarding. Too bad for many users.

Quoted text here. Click to load it


Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 10.12.2018 ?. 12:33, pozz wrote:
Quoted text here. Click to load it

If being a server is a problem make the device a client.
realVNC can be started in "listening" mode, so the device
connects to it, it can do that from behind a NAT router without
a problem.
This is how our netMCA series has been providing online
customer support for over a decade now.

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/



Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 10/12/2018 13:09, Dimiter_Popoff ha scritto:
Quoted text here. Click to load it

I'm not sure I got your point.

Where does realVNC run in "listening mode"? On a smartphone?

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 12/10/2018 15:51, pozz wrote:
Quoted text here. Click to load it

The android version cannot do that indeed.

So write your own viewer, RFB is simple enough, this is a huge
advantage.
Anything else you contemplate would take a lot more work for
android or whatever.

Dimiter

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/



Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 10/12/2018 21:50, Dimiter_Popoff ha scritto:
Quoted text here. Click to load it

You insist suggesting a point-to-point (device-to-smartphone) solution.  
I don't think this can be done.

For notifications, the device could alert the smartphone at any time,  
when the user is working at PC or playing tennis. In this case, the  
smartphone OS is sleeping, so all the services/apps aren't using  
Internet (all connections are closed). How the device could reach the  
smartphone/viewer?

Even for generic remote control your solution doesn't work. The user may  
want to connect to his home at any time, when the device behind a router  
doesn't know. How the smartphone could reach the device?

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 12/11/2018 15:27, pozz wrote:
Quoted text here. Click to load it

So how does the device reach anything you contemplate running on a
phone which is off. You seem to keep on trying to find a solution
to your quest "how do I have a DNS capable server without having one".
The answer to that is obvious enough I hope.

Quoted text here. Click to load it

Oh it works and has worked for about a decade now. The netMCA keeps on
attempting to connect to the listening VNC nodes (you can set multiple
of these) every 20 seconds or so.

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/







Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 11/12/2018 15:50, Dimiter_Popoff ha scritto:
Quoted text here. Click to load it

I usually don't power off my smartphone when working at PC or during a  
tennis mach bettwen hobbyist. The smartphone is in standby and this  
means all apps aren't active, in other words they can't communicate over  
the Internet (at least on iOS).

However there is a mechanism that let to push something to a sleeping  
smartphone, this is "push notifications". It's the same technology that  
creates a notification on your phone when you receive a Whatsapp message.

For what I know, this technlogy is based on a high-priority service  
running at the OS kernel level. It's a process that can always  
communicate over the Internet thanks to a persistent TCP connection.

Google and Apple have service for app developers that let them send  
notifications to one or mode smartphones. Many other services were born  
that let the developer to send notifications in a simpler manner.

I think this is the only way for device-to-smartphone  
alerts/notifications. IMHO here your RFB/VNC doesn't work.


Quoted text here. Click to load it

I can't see how. RFB is a TCP protocol. This means one is the server and  
one is the client. As we already discussed, the device behind a router  
can't be a server, so it must be a client.

Now, the server should be the smartphone. We can think to update our  
public address on a dynamic DNS service when the user open our app  
(myphone.dyndns.com). The device/client behind the NAT could try to  
connect to myphone.dyndns.com every 20 seconds. The app should be able  
to create a connection with the device in maximum 20 seconds. This could  
be ok in some cases.

However I'm not sure if the mobile operator will forward to your phone  
all the TCP packets, including the ones for TCP connection  
establishment. Are you sure the mobile operator assigns to your phone a  
*public* IP address, even if dynamic. In the latter case, you would need  
port forwarding on mobile network... much more difficult than port  
forwarding in my home.



Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 12/12/2018 0:34, pozz wrote:
Quoted text here. Click to load it

So in what will your app differ from a VNC
viewer app which "pushes a notification" in the respect under
discussion.

Quoted text here. Click to load it

You will likely find better answers regarding how to write android/IOS
apps on one of their forums, not sure many of us here know a lot about
that. At least I don't know much if anything.
But surely since there are apps doing it it should be doable, unless
there are services in these systems available to politburo members only
(no idea if this is the case, nor would I be surprised at all if it
were).


Quoted text here. Click to load it

Well RFB is widely used over tcp, I have not seen another use of it.

Quoted text here. Click to load it

Not at all, routers can do port forwarding; VNC (RFB) uses port
5900 in listening (viewer is server) mode and 5500 in active (viewer
is client) mode.
Quoted text here. Click to load it

Well I can't be sure what the phone operator does today let alone
what they will do in the future. Whether the operator provides internet
access or just some limited part of it is another matter.
But basically if they claim to give you internet access this is what
they should be giving you for the money you pay them.

On Android I can't even see the IP address my provider has given
me unless I look say at the logs of my domain when I visit it. May be
I just don't know where to look of course but I don't even know if
the phone has a real IP address, in fact I doubt that.
So when I want to get around the restrictions google and the rest of the
gang have put on the phone's net access I just use a server under
a domain I control in some way google can't yet block if they want
to stay on the OS market (http, ftp) so both devices can
find each other.
If you want to sell on the mass market (you mentioned Amazon
in another post I think) the $5/month such a server costs would be the
least of your expenses, and you will not even need a dynamic dns
as you will have your own domain included in the $5/month.
My hosting provider allows me to set subdomains such that some of them
are on their server and others are not, e.g. the online support
listening VNC runs on my desk via my home internet.

Dimiter

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/







Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Quoted text here. Click to load it

Do you actually want devices to communicate over the internet, or is
same-network communication sufficient?

For example, if the smartphone is on the same wifi network as the device,
you just need discovery protocols so you can find suitable devices.
Multicast DNS (aka Bonjour) is one such mechanism, but there are others.
There are also things like QR codes, NFC and Bluetooth which can be used as
bootstrapping (used to send an ID that's enough to identify the device on
the network and go from there).

If you do need internet connectivity (such as 'turn on my heating' when
you're out of the house) then you need some kind of public endpoint.

One thing you could look at is IPv6 tunnels, which allow your device to be
accessible if it's behind IPv4 NAT.  That solves the NAT issue (although you
would need an endpoint to host the tunnel - third party solutions for that
exist), and then you're back to discovery (where protocols like SIP allow
negotiation of peer to peer connections).  But you may decide you'd prefer
to go via a central server if you don't want your device listening to the
internet.

Once you're wedded to the central server model, the thing you should worry
about is resilience.  It's a single point of failure that can affect all
devices if something goes wrong - how will you handle that?  Some cloud
platforms will take care of hardware failure, but they're no help if your
buggy code causes the database to become garbled or the certificate expires
(cough, Ericsson), or one of a million different software things that could
go wrong.  What's the cost of all of your devices going down at the same
time?

I don't know if any ready made platform handles these things, although in
theory they should exist.  Next question: how much can you handle being
locked in to that platform?

Theo

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 10/12/2018 15:34, Theo ha scritto:
 >> Which protocol do use the smartphone during communcation with the
 >> server? Maybe HTTP(s).
 >> Which protocol do use the device during communication with the server? I
 >> think we have many solutions: MQTT, HTTP(s) and so on.
 >> What are the software to use in the server? I don't know.
 >
 > Do you actually want devices to communicate over the internet, or is
 > same-network communication sufficient?

Over the Internet, of course.


 > For example, if the smartphone is on the same wifi network as the device,
 > you just need discovery protocols so you can find suitable devices.
 > Multicast DNS (aka Bonjour) is one such mechanism, but there are others.
 > There are also things like QR codes, NFC and Bluetooth which can be  
used as
 > bootstrapping (used to send an ID that's enough to identify the device on
 > the network and go from there).

This is for smartphone-to-device direction. What about  
device-to-smartphone, for example when the smartphone is off? I think  
it's impossible without connecting to a Cloud service.


 > If you do need internet connectivity (such as 'turn on my heating' when
 > you're out of the house) then you need some kind of public endpoint.
 >
 > One thing you could look at is IPv6 tunnels, which allow your device  
to be
 > accessible if it's behind IPv4 NAT.  That solves the NAT issue  
(although you
 > would need an endpoint to host the tunnel - third party solutions for  
that
 > exist), and then you're back to discovery (where protocols like SIP allow
 > negotiation of peer to peer connections).  But you may decide you'd  
prefer
 > to go via a central server if you don't want your device listening to the
 > internet.
 >
 > Once you're wedded to the central server model, the thing you should  
worry
 > about is resilience.  It's a single point of failure that can affect all
 > devices if something goes wrong - how will you handle that?  Some cloud
 > platforms will take care of hardware failure, but they're no help if your
 > buggy code causes the database to become garbled or the certificate  
expires
 > (cough, Ericsson), or one of a million different software things that  
could
 > go wrong.  What's the cost of all of your devices going down at the same
 > time?
 >
 > I don't know if any ready made platform handles these things, although in
 > theory they should exist.  Next question: how much can you handle being
 > locked in to that platform?

More questions than answers



Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Quoted text here. Click to load it

OK - that wasn't immediately clear from your question.

Quoted text here. Click to load it

The device could store messages ready for the next time the phone is
connected.  There's not much you can do until the phone is turned on.

Quoted text here. Click to load it

I think this is actually a lot harder than people think.

Big question: what's the expected longevity of your system?

A lot of the IoT stuff is aimed at essentially disposable consumer devices -
after somewhere between 1 and 5 years they stop working.  If you want to
make your system still work after 5/10/20/30 years there are some big
challenges:

- will your node software be upgradeable, and how will you manage that?

- if your nodes aren't upgradeable, can you backend cope with all the
different versions of shipped firmware?  If they are, this issue won't
entirely go away (at some point, you reach the limits of a particular
hardware platform and further upgrades are infeasible)

- how will you test over all the models that might have been shipped?

- over time, communication protocols will change (eg SSL 2, SSL 3, TLS 1.0 /
1.1 / 1.2 / 1.3).  How will you handle that?  What should we do about a
product that's speaking what's now regarded as insecure SSL 2?

- a lot of IoT players are quite small.  There's a substantial chance that
any platform you pick goes out of business, gets bought out by someone who
isn't interested, 'pivots' to something else, or jacks up the price to
infeasible levels.

- running things yourself will insulate from changes at a third-party
platform.  But can you commit yourself to running the system in 5/10/20
years?

- what is the revenue stream that will keep the lights on?
Funding everything from an upfront purchase price tends not to work.


On smartphone it's even harder.  iOS is dropping support for 32 bit apps.  
So any app that hasn't been updateed in the last 5 years isn't going to work
any more.  Can you commit to keeping your software up to date (even if it's
no longer a shipping product)?

One option is to rely on simple technologies that are sufficiently baked in
to society that they aren't going away any time soon - for example voice
calls and SMS.  That fixes the smartphone end, but not the platform end
(still a need to provide an SMS gateway)


If this is not a product, but merely a means to an end (eg you have a
project for 6 months and don't care what happens after that) then a lot of
these headaches go away.

IFTTT is a possible option, although I have no experience of it.

Theo

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 11/12/2018 12:58, Theo ha scritto:
Quoted text here. Click to load it

Sorry, I mean when the smartphone is in standby (display is off while  
your are watching TV). Even if the device and the smartphone share the  
same home network, how to reach the smartphone to alert for some event?


Quoted text here. Click to load it

I know, this is the reason of my post.


Quoted text here. Click to load it

All those are good questions... from a technical point of view. However  
we are on the market and we need to sell products. Do you think the end  
customer asks so many questions when he is purchasing IoT gadgets on  
Amazon? I don't think. So here the main question is how to satisfy the  
stupid requirements of modern customers: playing with his smartphone  
*today* in couple of minutes. He isn't interested (and he will not be  
able to understand) if the gadget will work for 1 or 5 years.


Quoted text here. Click to load it

You will not sell any product with those technologies, even if they are  
more robust and should have a longer life.


Quoted text here. Click to load it

If I'm not wrong, IFTTT needs an already working cloud server. It talks  
to servers, not real devices behind a NAT.

Re: "Device-to-smartphone" and "smartphone-to-device" technologies

Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it
  
Quoted text here. Click to load it

never tried it but, Blynk ?

Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 10/12/2018 22:17, snipped-for-privacy@gmail.com ha scritto:

Quoted text here. Click to load it

It seems Arduino oriented... I will check better.

Re: "Device-to-smartphone" and "smartphone-to-device" technologies

Quoted text here. Click to load it


If you're going to use a public server to relay between the device and
the client (smartphone, or whatever), you have all sorts of ways to do
it depending on what you're expecting of the machines in question.

For example, if the device is smart enough to be a web server, you can
set it up to connect to the public server (via a TCP connection), and
then have the public server forward HTTP to the device.  Issues like
authentication and identification present a number of complications,
but these have been dealt with in various ways.  Specifically you'd be
doing something like a reverse proxy HTTP server for the public host,
which is something a number of web servers (including Apache) can do.
You might have to fiddle the back end connection to the device a bit.

If the device is not that smart, again, just establish a TCP
connection to the public server, and then let the public server handle
the clients.  If the clients are to see a web page, then you just
write the CGI (or whatever) pages to send requests to the devices as
necessary).  Any protocol you care to establish on the link to the
device.  If the client device is to be running an application, then
it's your choice what the protocol between the client app and the
public server is, although something SOAP-ish might be a good idea
(although hardly necessary).

In all cases authentication and identification are issues.

Re: "Device-to-smartphone" and "smartphone-to-device" technologies

Quoted text here. Click to load it

Yes, this is an experise area that in production quality requires
specific skills. My field is building this kind of systems from sensor
hardware to cloud and applications. I've done all of the parts to some
level to understand the basic challenges, but I don't even dream of
building a production quality system by myself.

I've worked with few developers that have been able to grasp embedded
and serverside, but this is quite rare and few.

Doing IoT securely is then an another matter - I've had the luck of
working with the best in this field and picked up enough from them to
know what are the main limitations and how difficult this is to do well.

As a minimum team for doing something you're describing, I'd pick a few
talented persons with skills in:
-Electronics/FW to get the HW done
-Design/mechanical, especially if it has to look nicer than typical
industrial stuff
-Cloud/server/mobile developer.

With these you could start, but to do something bigger you need to go
deeper in each part. As an example, mobile developers come mostly in two
flavours: iOS and Android. And then there's the possible web
interface. And I've only ever seen one single developer being able to
work well on web UI and protocol between device and cloud.

So yes, you'll need an another person.

Quoted text here. Click to load it

Check out the Google Cloud, Amazon AWS and Microsoft Azure IoT
architectures to get an idea. There's a lot of buzzword compliance (AI,
ML, IoT..), but by checking those three you can get the basic idea on
how it's usually done. There are a lot of product/application specific
stuff you'll need in addition, but you can start with any of those.

Quoted text here. Click to load it

Serverless is really nice approach for cloud, but finding true
experience in that field is difficult.

If I were to create a small scale mobile-controlled IoT device, I would pick
-Linux SOM or ESP32 (if you've got mains power - battery is more difficult!)
-Some of the commercial cloud stacks or my own MQTTS/AMQPS server with
 my own server.
-Firmware updates from the server (PKI signed with offline keys!)
-Web UI with some node.js based stack
-Mobile app to connect through the cloud - there are tools that allow
 reusing web development - not the best ones, but possible.
-Pay at least some thought on how your system will be attacked and how
 to mitigate

Some thoughts - now need to get back to work..

--  
mikko


Re: "Device-to-smartphone" and "smartphone-to-device" technologies
Il 12/12/2018 07:34, Mikko OH2HVJ ha scritto:
Quoted text here. Click to load it

Do you think a simple Cortex-M3 running FreeRTOS/superloop with lwip  
TCP/IP stack is not sufficient? This hw actually runs a HTTP server  
without big issues.

Quoted text here. Click to load it

Any real suggestions? I tried to read Amazon Web Services and Google  
Cloud Platform, but sincerely I lost the way.

Quoted text here. Click to load it

Do you mean a MQTT broker? So you imagine that the app on the smartphone  
is a MQTT client, as the device?

Quoted text here. Click to load it

Yes.


Web UI running in the app of the smartphone?

Quoted text here. Click to load it

Yes. The mobile app should connect to the cloud through MQTT or similar  
protocols, right?

Quoted text here. Click to load it

Thank you very much for your post. Those are precious info.



Re: "Device-to-smartphone" and "smartphone-to-device" technologies
On 12/13/18 4:51 AM, pozz wrote:
Quoted text here. Click to load it


Yeah, but it's not running HTTPS.  I think a CM3 would be woefully  
outclassed by having to slap SSL on top of things, and if you're not  
securing your traffic you're adding to the Internet of Crap problem.

--  
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.

Site Timeline