Store data on an IoT web server using HTTP POST requests

As explained in the first article of the series store data obtained by Internet of Things devices, although the data that is saved ends up on a server MySQL o MariaDB and language is used PHP to manipulate them at input and output, the flow of information between the electronic equipment and the database occurs using a Web server with whom you communicate according to the HTTP protocol.

At the beginning of the definition of HTTP protocol There were uses comparable to the one being described but the fact is that in the end it has not been fully exploited for various reasons, partly security and partly because progress was never made in defining a more specific or more efficient protocol, so Nowadays, especially in public servers, the most common thing is to use a connection HTTP what makes a POST request to the server to store the information or a GET to recover it, normally to display a web page that presents it and even on which you can interact.

The most basic text sent to the server in a request HTTP POST includes a line with the type of request (POST) the path to the web page that will store the information and the version of the HTTP protocol; another line with the host name (which allows virtual servers on the same server and/or on the same IP address) and finally another that contains the data that is recorded, separated from each other by the & sign and from the previous lines by one blank.

In the example above, a server called polaridad.es would contain a page in /iot/grabar_temperatura to manage the information using version 1.1 of the HTTP protocol

It can be seen that two & signs are used which shows that three fields are stored. The name of the fields is to the left of the equal sign and only two letters have been used to define them. As the name of the fields (or variables, if you prefer) of the request HTTP They are not related to those in the database, it is not especially important to use descriptive texts and short names are usually chosen (even numbered fields) to save text in communication with the server and speed up the data sending process.

The data that an IoT device normally sends to the server is of numerical type, mainly integers and simple decimals. When values ​​are sent in text format, as is the case with the variable "ne" in the example, unfavorable circumstances may arise that can be resolved, depending on the case, with more or less success and ease. On this occasion, plus signs (+) are used to separate words, replacing spaces that would otherwise alter the POST request. A generic way of sending the data that resolves most cases is by indicating the hexadecimal code of the characters, preceded by the percentage sign (%) Logically, it is not advisable to use this resource except when what is encoded is problematic since The length of what is sent is increased, which generally requires more resources, although it is certainly very small in size.

Although it is possible to operate a Web server For the Internet of Things only with the information from the previous example, many servers, especially public ones, add other data to the POST query (unfortunately not always limited to the protocol). The example below corresponds to the post request requested by the well-known server. public for internet of things ThingSpeak.

In addition to some personal data, such as X-THINGSPEAKAPIKEY (and which corresponds to the identifier of each client) in the previous example you can see that there are other headers that add more information to the request.

How to use a header in a POST request It simply consists of writing its name, a colon sign (:), a blank space and the value you want to assign it.

In order to test POST requests to the web server before completing the configuration of the other components, a connection to the server can be established and the data manually sent. For example, on a Linux computer it would be enough to use telnet polaridad.es 80 where polaridad.es is the name of the server and 80 is the port number on which the service responds. HTTP.

The cross-platform tool can be used on Linux, Windows or OS PuTTY, which was talked about in the article control UART serial devices from computer, to make the connection without using the console.

In the following HTTP headers list There are most of those that can be useful for a POST request to a Web server for Internet of things.

• Accept It is used to indicate the type MIME that the request expects the server to use in the response. It is expressed as tipo/subtipo which can be generalized using the asterisk (*) as a wildcard sign, for example as */* to refer to any or tipo/* to refer to all subtypes of tipo

  The most commonly used are:

  • text/plain Although it is the most basic, it is also the most used. It expects the server to return a simple (plain) text response that is sufficient to notify that the transaction has been correct and at most add accessory information such as the order number of the recorded data, the result of a comparison, the date of the server…

  • application/xml o text/xml Wait for the server to respond to the request in format XML. The meaning of choosing text instead of application allows for easier “human” (versus “automatic”) reading. This duality will present itself in others MIME types but the future trend of the standard is to prefer application against text The format XML allows a response that contains a lot of data to be structured in a very solid way, the drawback is that it adds a lot of artifice to the net data, which makes the response take up more than necessary, therefore requiring more bandwidth and probably more memory in the IoT device to process it.

  • text/html Used when the server response is HTML encoded as plain text (plain text) usually formatting a response. Since it is about storing data and the response would reach a small device without many resources, it is not common to use this type.

  • application/xhtml+xml Basically it is the version XHTML (HTML as XML valid) of the information in the previous section.

  • application/json It is most used when the server response contains several data including a more or less complex structure. The format JSON share with format XML a solid structure and adds the advantage of being more concise in the text added by the structure.

• Accept-Charset Determines the text encoding that is requested to be used in the response. To encode Latin characters, the UTF-8, the most complete, ISO 8859-15, which includes the Euro sign (€) and the ISO 8859-1, which is the most basic.

• Accept-Encoding Indicates the format according to which the server response can be encoded. Basically it serves to indicate a type of compression. Some of the most frequent are gzip deflate (which can be specified in more detail with deflate-raw o deflate-http) It is not common to use it in small devices connected to the Internet of Things since it requires a certain consumption of resources (memory and processing time) that are usually scarce in these types of equipment. It is not necessary to indicate that compression is not used with Accept-Encoding: identity since such a circumstance is considered by default.

• Accept-Language Expresses the language that can be used in the answer. For example, the Spanish of Spain would be specified as es-ES or the English of the United States of America as en-US

• Connection It is used to specify what should be done with the connection that has been established between the client (the IoT device) and the Web server once the data is received. Typically used with the value close in the format Connection: close to indicate that the connection should be closed after responding to the client.

• Content-Length It allows you to indicate the number of bytes occupied by the part of the request that contains the data, which is the one after the headers and separated by a blank line. It is very useful since it serves to verify the integrity of the information that is sent; If it does not measure what has been declared, it is not stored since it is considered that it has not arrived correctly. It is usually required by most public IoT servers.

• Content-Type It serves to indicate the MIME type with which the information sent to the server is encoded. The types are usually used text/html when the data sent to the server is expressed as a simple list of values ​​(something like a=3.6&b=4.8) Y application/jsonrequest (which would be the equivalent of the type application/json which is talked about in Accept) when a more complex structure is necessary, but any of the following can be sent list of MIME types.

• Cookie It is used to add a session identifier with which to maintain a chain of transfers (query, response, query...) that is more complex than a single request with which to send certain related data but obtained at different times.

Table of Contents


• Referer URL that originated the POST request, for example the web page from which it was sent. In the case of being used for IoT, it does not add relevant information since the information is sent directly, without a previous page, so it is not frequently used.

• User-Agent Reports the device making the request. In normal web traffic, the browser used in the Internet of Things allows the server to indicate the type of electronic device making the request. Identifying itself to the server allows the response to be formatted differently in each case, for example, returning a complex web page to a browser and a few warning data to a small IoT device.

It is possible to specify a comma-separated list of options instead of a single value in headers to indicate that several different values ​​are supported simultaneously. These values ​​can have an order of priority that is expressed according to a quality coefficient q for each one. Quality coefficients are separated by a semicolon (;) and asterisks (*) may also be used to refer to any type or subtype.

Accept: text/plain,text/xml,application/json;q=0.8,text/*;q=0.9,application/json

In the previous example the priority of the format JSON the largest (0.9) is that of plain text and that of formatted text. XML, which meet the specification text/*, is smaller (0.8) and equal between them. If possible, the server should react by encoding the response as JSON.

In the following example of a more complete POST request, the /iot/grabar_temperatura page of the server called polaridad.es is accessed using version 1.1 of the HTTP protocol. The client, called Sensoreitor-2000, sends the encoded data in JSON, expect the response as plain text in format UTF-8 using Spanish from Spain without using compression, which, by the way, is not necessary to indicate. The data sent to the server takes up 65 bytes. When sending the response the connection between the client and the server will be closed.

The following article explains how to configure MySQL database to store information sent by IoT objects