Scheduling¶
Importing data feeds¶
An import is made through using the standard HTTP POST method, with personal cluster credentials provided by Voyado. If the data feed is valid, a 204 OK response will be returned. If the data feed is invalid, an error will be returned with descriptions of the errors in the file. For more information about validation, see Validation. For more information about the import request, see Import.
Compression¶
The request body can be compressed with gzip if the header Content-Encoding
have the value gzip
. It is recommended that the imports are compressed.
Scheduled imports¶
When going live, products and updates to them must be scheduled to be imported on a regular basis. This is typically achieved by creating a program that queues and sends full imports and incremental imports at regular intervals.
Updating advanced custom attributes
When performing an import containing advanced attributes, each identifier must always be related to the same name for all products within the import. Relations are permitted to differ from already imported products that are not included in the current import. In this case, the latest id to name relation will be interpreted as an override, and apply for all products in the catalogue.
Full imports¶
Full imports, containing all product data is recommended to be sent on a regular basis for ensured data consistency.
For small databases (i.e. less than 10 000 products), full imports could be sent on a frequent basis, replacing the need for other types of imports. For an approach of only sending full imports, it is recommended to send the full imports once every hour.
For large databases, full imports are also recommended to be sent on a regular basis but at a less frequent interval, such as once a day or once a week. These are recommended to be complemented with more frequent smaller imports, modifying parts of the catalogue.
Incremental imports¶
There are two types of incremental imports: partial imports and attribute modifications. These types of imports should only contain the products where a change has been made since the previous import, and are therefore often significantly smaller than full imports.
Partial imports allows for structural product group changes such as adding or removing variants, products, or product groups. These must include complete product groups.
Attribute modifications allows for modification of individual attributes on variant, product, or product groups. These are typically useful for changes driven by separate systems such as price
and stock_number
changes from an ERP.
Queueing imports¶
Imports are processed one at a time
Incoming imports will be rejected while another import is being processed. This means that the data feed software must be set up to run imports one at a time.
Since imports are processed one at a time and for various reasons could fail, all imports should ideally be sent to a queue that aggregate imports, handles retries, and handle errors. A typical process for this is normally responsible for the following or similar tasks:
- Changes invoked by other systems are placed in a queue
- Once every e.g. 5 minutes, empty queue and if any changes, aggregate them and send as a single import
- Log and notify an appropriate person to take action upon errors
- Remove successfully imported changes from the queue
- Save unsuccessful import files for easier debugging
Long-running imports¶
Depending on the size of the import to Elevate, it might take some time. In rare cases, this means that a timeout may occur. Most likely, the timeout is caused by a network component in between the client and Elevate itself. In these cases, import requests using API 2 will return an error, whilst actually continuing to process the import in the background.
As of the introduction of Admin API v3, imports exceeding the maximum duration will instead respond with 202 ACCEPTED
and will return an import id that can be used to query the status of the import. The default maximum duration allowed for blocking imports can be overridden by adding the blockingTimeout
parameter to the request.
Example¶
The following example showcases the request flow for an import exceeding the blockingTimeout
Import using blockingTimeout
of 1 minute (=PT1M)
curl -i \
-X POST \
-H 'Api-Key: {api-key}' \
-H 'Accept-Encoding: gzip' \
-H 'Content-Type: application/xml' \
"https://{cluster-id}.api.esales.apptus.cloud/api/admin/v3/import/data?blockingTimeout=PT1M&force=false&name=name" \
-T request-body.file
After the blocking timeout, the server hasn't yet processed the file. The server responds with 202 ACCEPTED
and the body:
{
"id": "5537431447739019570",
"status": "IN_PROGRESS",
"reason": ""
}
The previously returned import id can now be used to periodically poll the import status.
curl -i \
-X GET \
-H 'Api-Key: {api-key}' \
-H 'Accept-Encoding: gzip' \
"https://{cluster-id}.api.esales.apptus.cloud/api/admin/v3/import/status?id=5537431447739019570"
The import status responds with 200 OK
and the status in the body:
{
"id": "5537431447739019570",
"status": "SUCCESS",
"reason": ""
}
The returned status can be NOT_FOUND
, IN_PROGRESS
, SUCCESS
, or FAILURE
. In the case of FAILURE
, a reason will be provided.
Error handling¶
All imports are atomic (i.e. either the entire import is processed or nothing will be imported), to make it easier to recover from errors. An error can be caused either by the content being invalid or because of other factors, such as network errors. If an error is received, the import can be safely rerun once the connection is restored or content of the data feed has been resolved.
The following types of errors can be returned:
Code | Description |
---|---|
4xx | The import contained errors. The response body contains more information about the error to help with corrections to the data feed generation. |
400 | The import was deemed dangerous according to Elevate. For more information, see Force imports. |
403 | Incorrect cluster credentials. |
404 | Cluster not found. |
409 | An import is currently in progress. |
5xx | Server error such as cluster unavailable or busy. If 5xx errors persist, contact Voyado Elevate Support and attach any information found in the response body. |
If a retailer experiences persistent network errors they should first contact their network administrator. If the problem seems to be on Voyado's end, please contact Voyado Elevate Support.
Temporary network errors
The data feed software should handle temporary network errors without manual intervention as this is usually done through retries.
Force imports¶
Potentially harmful imports
Do not override the built in protection in scheduled imports. Overrides should only be made for isolated and manually triggered imports.
Protection from potentially harmful imports is built into Voyado Elevate. If an import looks dangerous, e.g. it removes half of all products in a market it will be rejected with an appropriate error message and code 400. To override the protection and force Elevate to accept such an import, a force=true
argument can be added to the post request of the import.