December of last year. The European Space Agency (ESA) is announcing the selection of the winning tender for the building of DIAS systems (Data and Information Access Services), i.e. special platforms for storing and analysing data from the Copernicus program, the largest Earth observation program in the history of mankind. Every day, Copernicus satellites send approx. 20 terabytes of different types of images to Earth. This is approximately as much as the storage capacity of 80 standard laptops. The DIAS systems are aimed at providing everyone with free access to the data.
The heart of each platform is meant to be the data storage cloud. We are not only talking about gathering the data, but also about sharing it and maintaining the entire system. The contracts go to the industry’s giants: Airbus (Orange is responsible for the cloud, Atos (T-Systems, i.e. a subsidiary of Deutsche Telekom, will prepare the cloud) and Serco (the cloud will be prepared by the company OVH). The fourth contract goes to… no, not to the giants, but to a group of small and medium tech companies (mainly Polish). Creotech is the leader, and the task of preparing the cloud was awarded to an enterprise called CloudFerro.
Just like in a student flat
CloudFerro consists of 30 individuals in an office building in the centre of Warsaw. They have been on the market for 3 years. Until recently, they have been cramped into a four-room flat. As they recall, the climate of the place resembled a student flat to a certain extent, and the best place for meetings was the kitchen, which was converted into a small pseudo-conference room.
“When we began implementing the first project for the European Space Agency, the technical director came to visit us. He saw our ‘headquarters’, which was in a rented flat, scratched his head and demanded us to lead him to our server rooms. When he saw them – what we have, how the equipment is configured, the way the cables are laid – he had no doubt that we will make it”, says Maciej Krzyżanowski, the president of the company and one of its founders.
Despite the fact that CloudFerro is small and young, its portfolio contains contracts that large companies can only dream about. Three years ago, they have created the EO Cloud for the European Space Agency as part of a consortium with Createch, where 7 petabytes of data from European satellites is stored. For the ECMWF, the European Centre for Medium-Range Weather Forecasts (an international organisation that forecasts weather and analyses our climate), they have prepared a system for the acquisition and processing of satellite data for the needs of monitoring climate changes. Their original technology of combining a private cloud with a public one has beaten solutions drawn up by 12 other competitors, including by the industry’s giants. How is it possible for a small and young company from Poland to acquire such contracts?
“It is simple. We have cool technology and great competencies”, Krzyżanowski answers.
OpenStack and 300 knobs
CloudFerro provides services in the field of cloud computing and data storage. They are not the only ones. The cloud computing industry has been dominated by a couple of players, mostly American. Each offers solutions based on their own technologies. It is a bit like a set of boxes – it is easy to build something out of them, less knowledge is required to handle them, but the amount of possibilities is highly limited. CloudFerro builds clouds based on the open-source OpenStack technology. This system requires specialist knowledge, but you can use it to configure many things.
“In a closed system, we have proverbial three knobs, and when we spin them, everything just works. But you cannot do anything other than what those three knobs allow you to do, and if you spin something the wrong way, nothing bad will really happen. In an open-source system, there are 300 such knobs. And if we spin knob no.1 and then spin knob no. 281 the wrong way, we will be in a lot of trouble. We know how to use OpenStack safety and favourably”, Krzyżanowski tells figuratively.
As Krzyżanowski emphasises, they are able to spin the knobs and make use of the possibilities of the technology in such a way that they are able to flexibly react to the need of their clients. For example, in the cloud prepared for the European Space Agency, they came up with an idea for the rapid viewing of images. Previously, in order to view large amounts of data, users had to download them to local services first. That is why many analyses were comprised of scientists downloading images onto their computers for two months and then performing the analysis, which only took two days. CloudFerro developers combined cloud services with a large repository of satellite data and tools for an easy processing of the required information. As a result, the user did not have to copy the data to their machine. Instead of wasting two months in order to download the images, the user is able to conduct the analysis straight away in two days. And the images themselves are available to the users within 3-4 hours from the moment that they are sent to Earth.
Thanks to the technology, they have acquired a contract with the ECMWF.
“When we became interested in the tender, we only had our private cloud. But the client wanted something completely different – for the devices to be located at his location, for them to be connected with the public cloud, for them to have additional features. We have prepared an offer in three weeks, and then we managed to create the cloud in three months. It not only meets all of the requirements, but it is so flexible that if the client wants some additional feature, we are able to give it to him”, Krzyżanowski tells.
The art of laying cables
As the president of the company emphasises, you have to have competencies to be able to spin the knobs. As part of DIAS, 30 petabytes of data are meant to end up on the company’s drives in a span of three years. If you wanted to gather so much information on laptops, you would need to place 120 thousand devices side by side. In addition, you would have to make sure that the laptops do not overheat, you would need to configure them in such a way that they cooperate, and when one fails, you would have to make sure that the user does not notice the failure. Those competencies include, e.g. what drives to use in order to “only” use 4 thousand instead of 120 thousand, or what plugs or cables to use so that they do not break off, so that they do not decrease the data transfer speed.
“This is not as trivial as it seems. Our network is 100 gigabits, so it either requires fibre optics or high-frequency cables. None of those cables like to be bent. In order to lay the cables, we need people who have reached a high level of technical sophistication. As they say, the kingdom fell because a horse lost its shoe. And in this industry, many problems begin only with a simple cable”, Krzyżanowski says.
The first satellite images are expected to land on the DIAS platform in June. The European Union hopes that, if people – scientists, companies, cities, private individuals – have free and open access to data, they themselves will come up with a way of using the data, and thanks to this, the number of innovative companies in Europe will increase. Krzyżanowski uses the satellite images himself. He checks if no one dumps rubbish onto his garden.
