Plug&Play solution to reach an embedded HTTP server under a router

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
I'm designing a electronic device running an embedded HTTP server. I'm  
using lwip TCP/IP stack and the httpd application that comes with lwip.

Because the device is under an ADSL router and the public IP is dynamic,  
I need to register to a DDNS service, configure it in the router and  
open one port (NAT). These steps are too tricky for the most of my users.
So I'm thinking to a simpler solution.

I know I can create a cloud on the Internet that runs a standard HTTP  
server to the users, and a custom "IOT" protocol to the remote devices.  
However this way is too difficult for me and I have to change almost  
everything that now works well.

I'm thinking of a public HTTP tunnel that limits to forward the users  
HTTP requests to the HTTP server. Of course, the tunnel is made by two  
ends: one on the public server (and I could use everything, because I  
have to start from zero), the other on the embedded device (where I have  
many restrictions on memory size, computational power, no-OS and so on).

Any experience on implementing a HTTP tunnel on lwip?

Re: Plug&Play solution to reach an embedded HTTP server under a router
On Friday, April 6, 2018 at 11:51:17 AM UTC-4, pozz wrote:
Quoted text here. Click to load it

Did you ask on the lwip forum?
Looks like I'll have the same issue for an upcoming product...

Re: Plug&Play solution to reach an embedded HTTP server under a router
On 07.4.2018 ?. 17:47, Dave Nadler wrote:
Quoted text here. Click to load it

Don't know what lwip is but I did have this sort of issue 8-9
years ago when we introduced our netMCA series.
My problem was "how does the user know which IP address or host
name to enter in the VNC viewer and/or the browser" in order to
talk to the device. I think I posted here and got some suggestions,
none of which could work "out of the box" in an unknown environment
(i.e. the user might be using a wintel machine or a linux PC or
a phone etc.).

So I ended up leaving my original solution: when the netMCA boots
it does an ftp session and writes a text line (well, file) containing
its IP address. The file name contains the device serial number
which can be read on the box, so the user has to enter an URL
in their browser and see the IP address (which may be behind a router,
yet the user might be unable/unwilling to learn to see the router
dhcp list etc.).

Works OK - and I am not sure anybody uses it. I don;t think we had
a first time user manage that on their own, after all it takes
reading a paragraph in the manual...

So nowadays we ship a small router with the first netMCA for a new
user, the router being set to give fixed IP address(es) to the unit(s,
these being printed on a "quick start" paper. This generally works,
though typically new users still need some initial online
support.

Dimiter

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/




Re: Plug&Play solution to reach an embedded HTTP server under a router

[...]

I can't help you with the lwip TCP/IP stack, but consider to let the
owner of the device have the freedom to configure if a cloud server
should be used and/or which cloud server to connect to.

It will give the product a longer potential lifetime (in case you for
some reason have to take your cloud server down), and allow more
paranoid customers to avoid passing their data through a third-party
service.

Greetings,

Jacob
--  
"... while the C compiler will happily generate code for
 almost anything produced by leaning on the keyboard."

Re: Plug&Play solution to reach an embedded HTTP server under a router
Il 07/04/2018 17:20, Jacob Sparre Andersen ha scritto:
 >
 > [...]
 >
 > I can't help you with the lwip TCP/IP stack, but consider to let the
 > owner of the device have the freedom to configure if a cloud server
 > should be used and/or which cloud server to connect to.

Ok for allowing the user to choose between a direct user<->device  
connection and a "cloud" connected, but I think it's very difficult to  
allow choosing a "cloud server". Here there isn't a standard and the  
implementation is often proprietary. So the cloud server should be mine.


 > It will give the product a longer potential lifetime (in case you for
 > some reason have to take your cloud server down), and allow more
 > paranoid customers to avoid passing their data through a third-party
 > service.

I know, but nowadays users are more interested in the simplicity and  
"plug&play" functionalities instead of "paranoic" thinking.

Re: Plug&Play solution to reach an embedded HTTP server under a router

Quoted text here. Click to load it


Can you use something like SOCKS5 from the device?  Put the SOCKS
server on your public host.  You might be able to do that with some
implementations of SSH.

Re: Plug&Play solution to reach an embedded HTTP server under a router
Il 08/04/2018 06:27, Robert Wessel ha scritto:
 >
 >> I'm designing a electronic device running an embedded HTTP server. I'm
 >> using lwip TCP/IP stack and the httpd application that comes with lwip.
 >>
 >> Because the device is under an ADSL router and the public IP is dynamic,
 >> I need to register to a DDNS service, configure it in the router and
 >> open one port (NAT). These steps are too tricky for the most of my  
users.
 >> So I'm thinking to a simpler solution.
 >>
 >> I know I can create a cloud on the Internet that runs a standard HTTP
 >> server to the users, and a custom "IOT" protocol to the remote devices.
 >> However this way is too difficult for me and I have to change almost
 >> everything that now works well.
 >>
 >> I'm thinking of a public HTTP tunnel that limits to forward the users
 >> HTTP requests to the HTTP server. Of course, the tunnel is made by two
 >> ends: one on the public server (and I could use everything, because I
 >> have to start from zero), the other on the embedded device (where I have
 >> many restrictions on memory size, computational power, no-OS and so on).
 >>
 >> Any experience on implementing a HTTP tunnel on lwip?
 >
 > Can you use something like SOCKS5 from the device?  Put the SOCKS
 > server on your public host.  You might be able to do that with some
 > implementations of SSH.

It seems SOCKS is a "simple" TCP proxy between a client and a server,  
but the client stays client and server stays server.

In my application the user, through its Internet-connected smartphone,  
wants to connect to the embedded HTTP server behind a NAT. I think an  
intermediate SOCKS server doesn't help to reach the server behind the NAT.

Re: Plug&Play solution to reach an embedded HTTP server under a router
On Sunday, April 8, 2018 at 4:48:46 PM UTC-4, pozz wrote:
Quoted text here. Click to load it

Last time I had to do this I used OpenVPN from the device to a publicly visible
server. The user connected to the other side of the tunnel on the public server.
This would be overkill for lwip and a small device...

Can't you similarly use SOCKS to initiate a bi-directional connection
to your publicly visible server???


Re: Plug&Play solution to reach an embedded HTTP server under a router

Quoted text here. Click to load it


Sorry, my mistake - it's clearly been too many years since I looked at
it.  SOCKS5 does allow a "client" to establish a "bind" to a port on
the SOCKS server, but it's really only meant for use after an outgoing
connection is established to the other side - for example, it would
allow you to handle the data connection coming back from an FTP
server.

Re: Plug&Play solution to reach an embedded HTTP server under a router
On 07/04/18 01:51, pozz wrote:
Quoted text here. Click to load it

You want UPnP, specifically  
<https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal .

This is supported by basically all domestic routers, to
provide access from the Internet to e.g. gaming consoles.

It allows your device to find the router and ask it to
open a port from the outside.

Clifford Heath

Re: Plug&Play solution to reach an embedded HTTP server under a router
Il 09/04/2018 01:39, Clifford Heath ha scritto:
Quoted text here. Click to load it

This is interesting, even if I understood there are many similar  
technologies: UPnP, NAT-PMP, Internet Gateway Device Protocol, PCP (Port  
Control Protocol).  I don't know which one is the right choice.  It also  
depends if those are implemented in popular home routers.

Anyway, even if the NAT traversal problem is solved, there's another  
issue: the *dynamic* public IP address that could be solved with a  
Dynamic DNS account.  Anyway free DDNS accounts require manual  
confirmation every 30 days and this could be a problem for the end user.


Re: Plug&Play solution to reach an embedded HTTP server under a router
On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
Quoted text here. Click to load it

Very interesting, Thanks Clifford.
Some concerns: The article cited above says:
"UPnP is generally regarded as unsuitable for deployment in business settings..."
"... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."

Have you used UPnP successfully for commercial applications?

Thanks!
Best Regards, Dave


Re: Plug&Play solution to reach an embedded HTTP server under a router
Il 09/04/2018 14:36, Dave Nadler ha scritto:
Quoted text here. Click to load it

I made just some tests on a couple of home routers and both don't  
support NAT-PMP, that is a *different* protocol than UPnP/IGD.
Both routers replied with ICMP Port Unreachable.

I think they support UPnP/IGD because I see something in their web  
pages. However it's not simple to test in a short time.

Of course you can't expect all the routers support UPnP and that this  
functionality is enabled by the user.  I think UPnP is enabled by  
default, if supported. But the user can disable it, mostly for security  
concerns.

Re: Plug&Play solution to reach an embedded HTTP server under a router

Quoted text here. Click to load it

Most commercial (business/enterprise) routers filter most ICMP traffic
- by default some won't even allow pings.  Client applications are
expected to be using TCP or UDP.


Quoted text here. Click to load it

Yes.  UPnP can be a major security hole.  It's disabled by default in
most (all?) commercial units, and now even in some better SOHO units.

George

Re: Plug&Play solution to reach an embedded HTTP server under a router
Il 09/04/2018 19:06, George Neuner ha scritto:
Quoted text here. Click to load it

I was refering to ICMP packets *created* by the router as the reply of  
my NAT-PMP request (UDP port 5351).


Quoted text here. Click to load it

I think UPnP is *enabled* by default on most of home routers.


Re: Plug&Play solution to reach an embedded HTTP server under a router
On 09/04/18 22:36, Dave Nadler wrote:
Quoted text here. Click to load it

No, I have not needed to. But I know enough about networking
to have been curious how things like Skype, multi-player games,
and bit-torrent arrange inbound connections.

If customers can't reconfigure their networks, they need to
borrow a teenage boy for the afternoon :)

Re: Plug&Play solution to reach an embedded HTTP server under a router
On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath

Quoted text here. Click to load it

In ye olden days, Skype classified machines into two groups.  Machines
which could accept an inbound connection, and ones which could not.
The former were used as relay nodes for the latter.  That's a bit
oversimplified, as you actually could configure your Skype client to
not permit the relaying).  After MS took it over, they changed it so
that all Skype clients relay through a set of central servers.  Many
multi-player games work the same way.

Re: Plug&Play solution to reach an embedded HTTP server under a router
On 10/04/18 10:19, Robert Wessel wrote:
Quoted text here. Click to load it

Yes, but from very early on, it also supported hole-punching,
where you're not supposed to be able to accept an incoming
request, but with some forgery by a third-party server, many
NAT firewalls could be fooled into thinking that you'd originated
the request. I think it was that which made Skype special.

I started a project to build an API service for gaming companies
to do the same, but they're pretty keen to be middle-men, even
though the gaming experience is worse. Same goes for Microsoft.

Clifford Heath.

Re: Plug&Play solution to reach an embedded HTTP server under a router
On Monday, April 9, 2018 at 5:50:55 PM UTC-4, Clifford Heath wrote:
Quoted text here. Click to load it

I can't wait for the reaction from IT staff at fabs using our products ;-)

Re: Plug&Play solution to reach an embedded HTTP server under a router

Quoted text here. Click to load it

You don't need any of that if the devices make outgoing connections.


Quoted text here. Click to load it

HTTPS is the best choice for something that has to work in a security
environment because (unless an IP is blocked) all routers will pass an
outgoing HTTP(S) request.  Other protocols tend to be limited or
disabled and may require user/IT intervention to configure.

But if your <whatever> is something they want, the customers certainly
will arrange to pass your cloud traffic even if it's not HTTP.
  

Quoted text here. Click to load it

I'm assuming here that by "tunnel", you mean you need to pass binary
data in the HTTP request/response.  

Assuming your server IP isn't blocked, your devices should be able to
connect directly from behind a router.  You shouldn't need any kind of
tunneling proxy.

If you need to send a lot of data you may want to implement HTTP
stream transfers rather than having to ROT64 (or whatever) to embed
your binary data into the requests/responses.  


Quoted text here. Click to load it

Sorry - never used lwip.  But HTTP is just a data protocol over TCP
... it can be implemented on any TCP stack.


George

Site Timeline