Plug&Play solution to reach an embedded HTTP server under a router

Did you ask on the lwip forum? Looks like I'll have the same issue for an upcoming product...

I can't help you with the lwip TCP/IP stack, but consider to let the owner of the device have the freedom to configure if a cloud server should be used and/or which cloud server to connect to.

It will give the product a longer potential lifetime (in case you for some reason have to take your cloud server down), and allow more paranoid customers to avoid passing their data through a third-party service.

Greetings,

Jacob

Can you use something like SOCKS5 from the device? Put the SOCKS server on your public host. You might be able to do that with some implementations of SSH.

Don't know what lwip is but I did have this sort of issue 8-9 years ago when we introduced our netMCA series. My problem was "how does the user know which IP address or host name to enter in the VNC viewer and/or the browser" in order to talk to the device. I think I posted here and got some suggestions, none of which could work "out of the box" in an unknown environment (i.e. the user might be using a wintel machine or a linux PC or a phone etc.).

So I ended up leaving my original solution: when the netMCA boots it does an ftp session and writes a text line (well, file) containing its IP address. The file name contains the device serial number which can be read on the box, so the user has to enter an URL in their browser and see the IP address (which may be behind a router, yet the user might be unable/unwilling to learn to see the router dhcp list etc.).

Works OK - and I am not sure anybody uses it. I don;t think we had a first time user manage that on their own, after all it takes reading a paragraph in the manual...

So nowadays we ship a small router with the first netMCA for a new user, the router being set to give fixed IP address(es) to the unit(s, these being printed on a "quick start" paper. This generally works, though typically new users still need some initial online support.

Dimiter

====================================================== Dimiter Popoff, TGI

Il 08/04/2018 06:27, Robert Wessel ha scritto: > On Fri, 6 Apr 2018 17:51:11 +0200, pozz wrote: > >> I'm designing a electronic device running an embedded HTTP server. I'm >> using lwip TCP/IP stack and the httpd application that comes with lwip. >> >> Because the device is under an ADSL router and the public IP is dynamic, >> I need to register to a DDNS service, configure it in the router and >> open one port (NAT). These steps are too tricky for the most of my users. >> So I'm thinking to a simpler solution. >> >> I know I can create a cloud on the Internet that runs a standard HTTP >> server to the users, and a custom "IOT" protocol to the remote devices. >> However this way is too difficult for me and I have to change almost >> everything that now works well. >> >> I'm thinking of a public HTTP tunnel that limits to forward the users >> HTTP requests to the HTTP server. Of course, the tunnel is made by two >> ends: one on the public server (and I could use everything, because I >> have to start from zero), the other on the embedded device (where I have >> many restrictions on memory size, computational power, no-OS and so on). >> >> Any experience on implementing a HTTP tunnel on lwip? > > Can you use something like SOCKS5 from the device? Put the SOCKS > server on your public host. You might be able to do that with some > implementations of SSH.

It seems SOCKS is a "simple" TCP proxy between a client and a server, but the client stays client and server stays server.

In my application the user, through its Internet-connected smartphone, wants to connect to the embedded HTTP server behind a NAT. I think an intermediate SOCKS server doesn't help to reach the server behind the NAT.

Il 07/04/2018 17:20, Jacob Sparre Andersen ha scritto: > pozz writes: > > [...] > > I can't help you with the lwip TCP/IP stack, but consider to let the > owner of the device have the freedom to configure if a cloud server > should be used and/or which cloud server to connect to.

Ok for allowing the user to choose between a direct userdevice connection and a "cloud" connected, but I think it's very difficult to allow choosing a "cloud server". Here there isn't a standard and the implementation is often proprietary. So the cloud server should be mine.

I know, but nowadays users are more interested in the simplicity and "plug&play" functionalities instead of "paranoic" thinking.

Last time I had to do this I used OpenVPN from the device to a publicly visible server. The user connected to the other side of the tunnel on the public server. This would be overkill for lwip and a small device...

Can't you similarly use SOCKS to initiate a bi-directional connection to your publicly visible server???

Sorry, my mistake - it's clearly been too many years since I looked at it. SOCKS5 does allow a "client" to establish a "bind" to a port on the SOCKS server, but it's really only meant for use after an outgoing connection is established to the other side - for example, it would allow you to handle the data connection coming back from an FTP server.

You want UPnP, specifically .

This is supported by basically all domestic routers, to provide access from the Internet to e.g. gaming consoles.

It allows your device to find the router and ask it to open a port from the outside.

Clifford Heath

This is interesting, even if I understood there are many similar technologies: UPnP, NAT-PMP, Internet Gateway Device Protocol, PCP (Port Control Protocol). I don't know which one is the right choice. It also depends if those are implemented in popular home routers.

Anyway, even if the NAT traversal problem is solved, there's another issue: the *dynamic* public IP address that could be solved with a Dynamic DNS account. Anyway free DDNS accounts require manual confirmation every 30 days and this could be a problem for the end user.

Very interesting, Thanks Clifford. Some concerns: The article cited above says: "UPnP is generally regarded as unsuitable for deployment in business settings..." "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."

Have you used UPnP successfully for commercial applications?

Thanks! Best Regards, Dave

I made just some tests on a couple of home routers and both don't support NAT-PMP, that is a *different* protocol than UPnP/IGD. Both routers replied with ICMP Port Unreachable.

I think they support UPnP/IGD because I see something in their web pages. However it's not simple to test in a short time.

Of course you can't expect all the routers support UPnP and that this functionality is enabled by the user. I think UPnP is enabled by default, if supported. But the user can disable it, mostly for security concerns.

Most commercial (business/enterprise) routers filter most ICMP traffic

- by default some won't even allow pings. Client applications are expected to be using TCP or UDP.

Yes. UPnP can be a major security hole. It's disabled by default in most (all?) commercial units, and now even in some better SOHO units.

George

You don't need any of that if the devices make outgoing connections.

HTTPS is the best choice for something that has to work in a security environment because (unless an IP is blocked) all routers will pass an outgoing HTTP(S) request. Other protocols tend to be limited or disabled and may require user/IT intervention to configure.

But if your is something they want, the customers certainly will arrange to pass your cloud traffic even if it's not HTTP.

I'm assuming here that by "tunnel", you mean you need to pass binary data in the HTTP request/response.

Assuming your server IP isn't blocked, your devices should be able to connect directly from behind a router. You shouldn't need any kind of tunneling proxy.

If you need to send a lot of data you may want to implement HTTP stream transfers rather than having to ROT64 (or whatever) to embed your binary data into the requests/responses.

Sorry - never used lwip. But HTTP is just a data protocol over TCP ... it can be implemented on any TCP stack.

George

Exactly, this is how I have implemented the "online support" on our netmca series (they can be set to call "home" every 20 seconds or so).

The issue one might have is that there may be no universal listening application at the "home" side. In our case realVNC can be set to listen and this works fine. Users also get the ability to have their netmca call a set of addresses of their choice (and they can switch off the "support" call of course).

Dimiter

No, I have not needed to. But I know enough about networking to have been curious how things like Skype, multi-player games, and bit-torrent arrange inbound connections.

If customers can't reconfigure their networks, they need to borrow a teenage boy for the afternoon :)

In ye olden days, Skype classified machines into two groups. Machines which could accept an inbound connection, and ones which could not. The former were used as relay nodes for the latter. That's a bit oversimplified, as you actually could configure your Skype client to not permit the relaying). After MS took it over, they changed it so that all Skype clients relay through a set of central servers. Many multi-player games work the same way.

Yes, but from very early on, it also supported hole-punching, where you're not supposed to be able to accept an incoming request, but with some forgery by a third-party server, many NAT firewalls could be fooled into thinking that you'd originated the request. I think it was that which made Skype special.

I started a project to build an API service for gaming companies to do the same, but they're pretty keen to be middle-men, even though the gaming experience is worse. Same goes for Microsoft.

Clifford Heath.

I was refering to ICMP packets *created* by the router as the reply of my NAT-PMP request (UDP port 5351).

I think UPnP is *enabled* by default on most of home routers.