Apps that help to fix the problems of your customers

Škoda Auto is innovative and always looking for new ways to develop its app. Therefore, solutions that help customers with their everyday mobility problems are a good way to increase customer satisfaction. Škoda Auto wanted to offer its customers an easy way to load the trunk using augmented reality, but the loading possibilities and the type of luggage are practically unlimited. This was a real challenge. Such problems are usually calculated at universities by large teams.

Today, the Škoda-LoadIn app is integrated into the main Škoda app and helps customers of all vehicle types to find an individual, optimal loading strategy for the boot. For this purpose, a 3D network was developed that corresponds to the boot. The algorithm was then trained to make a quick and at the same time as precise as possible statement about loading variants. For the benefit of usability, an intelligent text recognition system was integrated for objects that may already be labeled accordingly (e.g. furniture). It is also possible to adjust the size of the object via multi-touchgest and last but not least the user can measure the object with the help of augmented reality. The app was first tested as an internal release. The Škoda employees have given the app an A+ rating, which means that the app has been put into live operation. The selection for exactly this provider was based on design competence (Clutch Awards 2019) and high competence in iOS development.

In order to test the functionality, iteration had to be performed and potential scenarios had to be differentiated. The results of the calculation could be displayed directly in a matrix in 3D in order to precisely identify any defects in the concrete result.

For Škoda Auto, using augmented reality was a total novelty and nobody had experience with it. 

The app should be a showcase for new technologies. At the same time, the calculation had to be done in less than 60 seconds.

The app was successfully integrated into the brand strategy.

The calculation does not have a certain correct solution, but is potentially infinite. This means that the problem had to be broken down into individual possible solutions and then defined what the ideal solution could look like. Successively, several different strategies of the loading process were implemented, whereby the most effective one was ultimately chosen.

Škoda Auto provided various 3D trunk models for all supported vehicle models. These had to be converted into a uniform form. The first challenge was to standardize the documents in a format that could be used for calculations:

• The trunk of the car was divided into virtual small cubes with an edge length of 5 cm, which proved to be ideal in tests. Most of the luggage is loaded into this net and the calculation is faster than if something is manually stowed in the trunk. Computer scientists and mathematicians visualize this in the form of the so-called backpack problem.

Since the calculation using an app should not take longer than the calculation in the head, the algorithms had to be optimized and a balance between accuracy and speed had to be found.

• The load in the trunk is calculated in four ways. The most effective method of loading is then displayed directly in the mobile device using vector masks.

The input of the dimensions of the individual items should not degenerate into boredom. Therefore, ways were considered how to simplify and facilitate the associated routine work.

• Users can enter the baggage dimensions and any information on the fragility or softness of the load into the app themselves. However, for some packages - such as those for furniture kits - the dimensions are already given. In this case it is sufficient to pull out the camera and leave the input of the dimensions to the intelligent text recognition.

The app is maintained by the provider including the management of updates. No special skills were required on the part of Škoda Auto.

The task was to create an independent module of a mobile app for Android and iOS that can tell the user within 60 seconds whether the entered cargo fits into his car and if so, show a safe way to load it.

For the representation of the objects in the cargo area of the car, it had to be assumed that the respective cargo area with an open trunk was merely rendered in 2D. Therefore, a special way of calculating the perspective and projecting the objects into the 2D space was developed - including masks for displaying the hidden parts.

The models of the interior of the cargo area are provided in the standard triangular format. The task was to devise a discrete format suitable for the calculations, including a transformation of the coordinate system suitable for representation and calculation.

A mere manual numerical input of the size of the objects would be insufficient and not user-friendly. For this reason, the input of an object is based on visualization and animation, and it is even possible to handle the object including its size adjustment by means of multi-touchgest. With the help of OCR, the app can recognize an object and rewrite its size. The user has the possibility to measure the object with the help of AR, if he does not have a measuring stick at hand or does not know the exact size.

Technical facts

•  iOS already online as part of the My Škoda App, Android on the road
• The version for Android is based on MVVM architecture. The coding was done in Kotlin. The app for iOS is kept in Swift in MVVM-C architecture. Since Load-in is currently part of the app MyŠkoda, everything has been packaged into a library for use in other apps using common package management systems.
• The algorithm running on the backend was developed in the Python language, which has a very good set of tools for working with scientific computations and which allows fast iteration. For the final solution, the code is also translated into C using the Cython tool, which makes the calculation even faster. All in all, this runs on Azure Cloud - with intelligent scaling, any number of users can be served.
• To handle the trunk packing problem, several strategies were successively used - Residual Space and Minimal Space, both in two variants: lean and brute-force.

Where is the data stored?

The data is on-premise at the end customer