Web Servers and the IoT

The World of embedded  Webservers

This is quick write-up of some of the things that are keeping me busy lately.

In writing some hands-on notes on networking, using as an example a server providing HTML content, I realized there might a potential market for something I called in my head Aweb. 

 Why that name? I was looking for a wordplay on webapp: a Aweb is an app that provides a webpage! ;-)

This would be but a micro/nano-webserver serving static HTML content that eventually could be even hard-coded.

Such Aweb would be extremely-narrowly dedicated. By that I mean, think of a normal webserver, say Apache, or Nginx, that would be serving only one file, or only from one folder.

It would also have a minimal storage and memory footprint...That's it! I realized then where such a thing could be useful...

Of course, embedded systems of the Internet of Things (IoT)!

Before I dig myself further into writing such a minimal webserver, I figured I look around what's already out there.

It turns out, there is a niche business for it!

But these micro/nano-webservers have a bunch of non-trivial, unique challenges. The only Wikipedia page I found is this one in french, but it gives a nice overview.

• Lost of flexibility:
• They are usually written in Assembler or C. Not surprising, and that's fine, I'm writing these servers in C  myself.
• The reuse of code is practically inexistent. This goes with the extremely targeted nature of each of these embedded servers.
  
• Portability is completely forgotten. Ditto.
  
• The skill requirements in  programming expertise are very high. I hope my upcoming hands-on on networking helps you at least with the part on networking concepts involved. :-)
• Cost sensitive:
• "The purpose of this work is to create cost-sensitive, reliable technology that enablesWeb access to distributed measurement/control systems". See here.
• Performance:
• For IoT to be successful such that people adopt it both devices as well as software must be fast enough do deliver the usability required. These applies to such a micro webserver: it has be capable of handling many simultaneous connections. Due to the resource limitations of the devices such a server is running one, this is not a trivial challenge. Power consumption is one such challenges.

Some Solutions

 

So, what is then an industry, typical such IoT hardware?

An example could be an 8-bit processor with 40KiB memory -incidentally, that reminds me the Zilog81 in the form of a Spectrum48K we had at home in my childhood: 16KiB of ROM, 48KiB of RAM!

Surprisingly enough, at least to me, efforts have been devoted to come up with an embedded, underlying, specially-tailored Java virtual environment. Such JVM would provide the missing portability. 

 

Some embedded systems use Jini, originally a Sun Microsystem's project was transferred to Apache under the name of River. 

Another Java framework used in mobile systems is J2ME, the Java Micro Edition. targeting systems with less than 100KiB of memory.

So what are some solutions out there?

 

If one can have access to similar resources a typical OS provides, such a webserver can be designed not too different from what we may do using C.

 

However, if the limitations of the physical device are too tight, that may not work. In these cases such an Aweb and the OS functionalities must be blend into the same, unique software.

• With an OS
• Contiki: From the Wikipedia page: "Extant uses for Contiki include systems for street lighting, sound monitoring for smart cities, radiation monitoring, and alarms.[...] It provides multitasking and a built-in Internet Protocol Suite (TCP/IP stack), yet needs only about 10KiB of RAM and 30 of ROM. A full system, including a graphical user interface, needs about 30 kilobytes of RAM!!". Amazing!!
• TinyOS: "operating system and platform for low-power wireless devices, such as those used in wireless sensor networks (WSNs), smartdust, ubiquitous computing, personal area networks, building automation, and smart meters. It is written in nesC, a subset of C optimized for the memory limits of sensor networks. TinyOS has been used in space, being implemented in ESTCube-1". Cool!
• Without an OS
• That Wikipedia page only mentions SMEWS (Smart Mobile Embedded Web Server). It doesn't offer more details -except that the main developer was a co-founder of Contiki.
  

The TCP/IP Stack

Here is another interesting piece. 

 

The integration of the TCP/IP stack in one such small devices isn't evident. This network and transport layer was designed in the 80's free of the limitations these new devices come with. As such a common Linux system uses of the order of several 100KiB!!

It seems the solutions go then through stripping of the usual TCP/IP stack from parts not strictly necessary in the context of IoT, or at leas that one can do without.

A guy named Tao Lin did that by removing the time-out timer of TCP. As you know, when communicating through TCP, packets for which no acknowledgment is received -within a determined time- are resent. That's at the heart of the reliability achieved by the TCP protocol. Lin hard-coded a time and considered that a packet not acknowledged within 6 secs is never going to be so and can thus be forgotten. This removes the need to resend it. 

That's interesting. Somehow, it brings us closer to an UDP protocol. In some sense, we choose the way and content of what we want to communicate. The underlying assumption is that packets are rarely lost -and if they are, we suck it up and live with it. 

 

Notice that that's what's done with video and audio: if the image freezes for sec...you really don't care to see later what the second you missed before!

This means, these kind of shortcuts drive the solutions for IoT, and hence, the whole technology towards a real-time experience, and it makes sense I guess. It's also part of our time where everything is geared towards immediacy.

 

Embbeded TCP/IP implementations are :

• µC/TCP-IP
  
• openTCP 
• lwIP : Aside from the TCP/IP stack, lwIP has several other important parts, such as a network interface, an operating system emulation layer, buffers and a memory management section. The operating system emulation layer and the network interface allow the network stack to be transplanted into an operating system, as it provides a common interface between lwIP code and the operating system kernel.

These require of the order or several 10KiB and some kind of underlying OS. (Did you notice? Somebody else also thought of the "micro" prefix. ;-). However, there are other solutions that bypass that latter requirement and access the hardware directly.

The limitation on resources means that these solutions deal only with one segment at a time and

do not manage

• network congestion management
• sorting of incoming packets
• IP fragmentation
• urgent TCP data packets.

Specific examples

An example is WebCard,  a web server for JavaCard:

• Only considers packets to port 80
• Ignores any TCP option
• Only uses HTTP1.0 not ver. 1.1
• Only implements the GET method.
• It doesn't handle IP fragmentation: it is assumed the GET request fits and arrives in one single packet!

 

I hadn't thought on IP fragmentation yet. But I was conceiving Awebs as offering only the GET mechanism indeed. 

 

Another example is MiniWeb, which is a proof of concept. It uses 24-28Bytes of RAM for the executable code and C source is a mere 400 lines long. The author doesn't mean it as more than what it is though: "It is important to note that this approach of course is totally and utterly wrong; a system using this approach should not be connected to an actual network, let alone the global Internet!".

 

It is of course limited compare to usual web servers: it handles only one connection at a time. Obviously, it doesn't offer dynamic content either.

 

Additional  technological  shortcuts are, e.g.,  client-side execution,  as is the idea of AJAX, or co-design of IoT system and software.

Another project is Ethernut: This is an Open Source Hardware and Software Project for building tiny Embedded Ethernet Devices. Firmware, hardware and (programming) tools are all developed within the same project.

This wikipedia page is a trove on low-level IoT technology. It ends with a table of examples of different embedded webservers. Notice that in french, a byte is called an octet. Thus they say kilo-octet (Ko) instead of kilobyte (KiB).

Conclusions

The low-level world of IoT is fascinating and fun. At least I'm realizing that, now that I'm writing those hands-on examples of networking code.

Maybe I have to chance to have a closer look into one of those projects.

Among those apparently more developed and/or not abandoned seem to be

• lwIP
• Ethernut
• Contiki
  

at least they are the ones that got my attention. Ok, two more did as well, but I digressed already enough on this topic for me to devote it more time. Anyway, I'm curious about microTCP and openTCP mentioned above.